build runtime library conformant with dynamic libraries on linux

thank you

actually, that didn't work. the GC needs another mode for determining the segments. new patch attached.

More problems: memory allocation works with the changed GC mode, but garbage collection crashes. I guess there are GC changes required that i don't know how to do. I can deliver a test-lib and app if required.

v2 didnt work with svn branch 0.99.4

I'm pushing this off because I don't know enough about *nix dynamic libs to fix this without some research.

I intend to fix this one but it's going to take some research. With any luck I'll be able to sort it out before the next release.

Did we get any further on this?

Not yet.

(In [4387]) small changes toward -fPIC, refs #857

Are there any unreleased patches for this? I'll start working on it soon :)

Not that I know, the patches here are mostly outdated, because the things are compiler dependent, and they were for gdc.

With ldc, a lot of time ago I had tried to compiled shared libraries on mac and it sort of worked, there were some issues, but it was not clear if they were ldc issues or shared libs issues.

ldc can generate shared libs without problems (you have to pass -relocation-model=pic to it), and you can also look at how (for example) qtd does it.

I would just use a new version name for it, if you do ./build.sh --version soopt

or something like that, then you can define the arch files with the correct flags, and I would check them in. I am not sure if you can use the ldc.inc, I think that you will have to define an ldcSo.rules (to link stuff differently, and for the different extensions), but it will mostly mirror ldc.rules.

The easiest thing (0 effort) would be to make just tango-user a dynamic lib, keeping tango-base static, but some effort in making tango-base mostly dynamic would be for sure welcome.

There the main issue then is to get the correct addresses for the constants in the gc (which should also work with ldc, as it should use the dynamic segment tables but should be checked if it is still true in practice.

The last thing is the startup sequence of an executable I haven't really looked at it, but I think that one might have to link a static object file (basically dmain2 or something like it, maybe it can be even more stripped down) that has still to be statically linked.

Well, after our discussion i've

goshawk@earth:~/workspace/test$ ldc test.d -I/home/goshawk/Documents/Projects/tango/trunk/user/ -L-L/home/goshawk/Documents/Projects/tango/trunk/build/libs/ -L-ltango-user-ldc-soopt
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `inflateEnd'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `CompareStringW'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `BZ2_bzCompressInit'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `deflate'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `deflateInit_'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `BZ2_bzCompress'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `inflateInit2_'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `inflate'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `deflateEnd'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `SetThreadLocale'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `BZ2_bzDecompress'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `inflateBackInit_'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `GetUserDefaultLCID'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `MultiByteToWideChar'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `BZ2_bzDecompressEnd'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `inflateInit_'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `deflateInit2_'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `BZ2_bzDecompressInit'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `BZ2_bzCompressEnd'
collect2: ld returned 1 exit status
Error: linking failed:
status: 1

I've to add -I/home/goshawk/Documents/Projects/tango/trunk/user/ due to bug #1733 and the ldc.conf is

goshawk@earth:~/workspace/test$ cat /etc/ldc/ldc.conf
// This configuration file uses libconfig.
// See http://www.hyperrealm.com/libconfig/ for syntax details.

// The default group is required
default:
{
    // 'switches' holds array of string that are appends to the command line
    // arguments before they are parsed.
    switches = [
        "-I/usr/include/d/ldc/libtango",
        "-I/usr/include/d/ldc/libtango/lib/common",
        "-L-L/usr/lib",
        //"-L-ltango-user-ldc",
        "-d-version=Tango",
        "-defaultlib=tango-base-ldc",
        "-debuglib=tango-base-ldc"
    ];
};

Well, finally i reached this point:

goshawk@earth:~/workspace/test$ ldc test.d -I/home/goshawk/Documents/Projects/tango/trunk/user/ -L-L/home/goshawk/Documents/Projects/tango/trunk/build/libs/ -L-ltango-user-ldc-soopt  -L-lz -L-lbz2
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `CompareStringW'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `SetThreadLocale'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `GetUserDefaultLCID'
/home/goshawk/Documents/Projects/tango/trunk/build/libs//libtango-user-ldc-soopt.so: undefined reference to `MultiByteToWideChar'
collect2: ld returned 1 exit status
Error: linking failed:
status: 1

I've googled for the 4 missing functions and they are Win32 functions.

I think there is a problem while compiling which includes these functions in the tango .so file in linux version.

Thanks to lastest trunk fixed by fawzi, i'm able to build the library, but there are problems when i try to compile a simple hello world program:

goshawk@earth:~/workspace/test$ ldc test.d -I/home/goshawk/Documents/-L-L/home/goshawk/Documents/Projects/tango/trunk/build/libs/ -L-ltango-user-ldc-soopt  -L-lz -L-lbz2
goshawk@earth:~/workspace/test$ ./test
./test: Symbol `_d_eh_personality' causes overflow in R_X86_64_PC32 relocation
Segmentation fault
goshawk@earth:~/workspace/test$ cat test.d
import tango.io.Stdout;

int main(){
    Stdout("Hello World shared library");
    return 0;
}

I was finally able to compile a hello world program but i've this error:

goshawk@earth:~/workspace/test$ ldc test.d -I/home/goshawk/Documents/-L-L/home/goshawk/Documents/Projects/tango/trunk/build/libs/ -L-ltango-user-ldc-soopt  -L-lz -L-lbz2
goshawk@earth:~/workspace/test$ ./test                                                                                                                                   
./test: Symbol `_d_eh_personality' causes overflow in R_X86_64_PC32 relocation                                                                                           
Segmentation fault                                                                                                                                                       
goshawk@earth:~/workspace/test$ gdb ./test
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
(gdb) run
Starting program: /home/goshawk/workspace/test/test
/home/goshawk/workspace/test/test: Symbol `_d_eh_personality' causes overflow in R_X86_64_PC32 relocation
[Thread debugging using libthread_db enabled]
[New Thread 0x7f79a4b296f0 (LWP 12764)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f79a4b296f0 (LWP 12764)]
0x00007f79a450c06b in _D5tango4math6random7Twister7Twister4seedMFkbZv () at tango.math.random.Twister.d:244
244                             mt[mti] = (1812433253U * (mt[mti-1] ^ (mt[mti-1] >> 30)) + mti);
Current language:  auto; currently minimal
(gdb)
            

If i link the executable with both user and base compiled with -relocation-mode=pic i get:

goshawk@earth:~/workspace/test$ ldc test.d -I/home/goshawk/Documents/-L-L/home/goshawk/Documents/Projects/tango/trunk/build/libs/ -L-ltango-user-ldc-soopt -L-ltango-base-ldc-soopt  -L-lz -L-lbz2                                                                                                                                                                          
goshawk@earth:~/workspace/test$ ./test                                                                                                                                                
Segmentation fault
goshawk@earth:~/workspace/test$ gdb ./test
GNU gdb 6.8-debian
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu"...
(gdb) run
Starting program: /home/goshawk/workspace/test/test
[Thread debugging using libthread_db enabled]
[New Thread 0x7fc24da896f0 (LWP 12008)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fc24da896f0 (LWP 12008)]
_D2rt7basicgc3gcx2GC12mallocNoSyncMFmkZPv () at rt.basicgc.gcx.d:572
572                 // Return next item from free list
Current language:  auto; currently minimal
(gdb) quit

If i compile using the tango user shared library and the tango base static library i get the previous error.

shared user library for x86_64 ubuntu linux on 5/09/09

(In [4934]) first attempts for dynamic lib, refs #857

the warning seems connected with http://hlvm.llvm.org/bugs/show_bug.cgi?id=2853 but there the binary seems to still work, but this is not the case with ldc

any thoughts on this, h3r3tic?

I'm afraid I know nothing about dynamic libs in Linux, Kris :(

I have successfully compile trunk-r5428 into a shared library and compiled with her "Hello, World" under linux32 and ldc

On the way, made a few changes. 1: tango.util.cipher.Cipher; All undefined methods in abstract classes should be declared as abstract. 2: tango.text.locale.Win32; undefined reference to MultiByteToWideChar? etc. put everything in the version (Windows) 3: tango.core.tools.Cpuid; segfault when performing the ctor module. As a temporary solution - made that version of Shared - ctor void.

Compile command: gcc -shared -Wl,-soname, libtango.so -o libtango.so *.o -lpthread -ldl -lm -lbz2 -lz after work bob

build with bob -o="-O -release -inline -relocation-model=pic -d-version=Shared"

Cpuid works after redisign.

Segfault in tango.net.util.c.OpenSSL while binding ERR_get_error_line_data

I comment it and all works.

I add all patches needed for building shard library with linux32 and ldc. Strange moment in OpenSSL: in in function bindFunc, when i binding ERR_get_error_line_data i have segfault. I rename ERR_get_error_line_data variable to ERR_get_error_line_data_ and static ctor works success. Compiler bug?

thanks Wazar

OpenSSL adjustment seems to be a non-compatible change? And CPUID seems to be a massive change ... can you provide some more info on those, please wazar?

I'm giving this a try and after successfully building the shared object and compiling a simple hello world, I get this error when the program is ran:

LANG=C LD_LIBRARY_PATH=~/tesis/ldc/lib ./test
./test: Symbol `_d_eh_personality' causes overflow in R_X86_64_PC32 relocation
Segmentation Fault

Linux, 64 bits, ldc 0.9.2, tango 0.99.9. To build tango I compiled the same files bob compile except tango/core/vendor/ldc/{bitmanip,vararg}.d (to avoid multiple definition errors) and tango/text/locale/Win32.d (because it uses the Win API, already fixed in svn). I added -relocation-model=pic to the ldc command line and -fPIC to the gcc command line. I linked all the objects together with -shared -lz -lbz2 -lpthread -ldl -lm -Wl,-soname,libtango.so.

Any ideas on what can be wrong?

There is a similar ticket in the LDC bug tracker: http://www.dsource.org/projects/ldc/ticket/360

BTW, it looks like that error is not causing the segmentation fault, this is the backtrace:

#0  0x00007f750850f7c0 in _D2rt2gc5basic3gcx3Gcx11__invariantMFZv (this=0x0)
    at tango/core/rt/gc/basic/gcx.d:1566
1566            if (inited)
(gdb) bt
#0  0x00007f750850f7c0 in _D2rt2gc5basic3gcx3Gcx11__invariantMFZv (this=0x0)
    at tango/core/rt/gc/basic/gcx.d:1566
#1  0x00007f750850f584 in _D2rt2gc5basic3gcx3Gcx16thread_InvariantMFZv (
    this=0x1903160) from /home/luca/tesis/ldc/lib/libtango.so
#2  0x00007f750850e38b in _D2rt2gc5basic3gcx2GC11__invariantMFZv (this=0x1903130)
    at tango/core/rt/gc/basic/gcx.d:273
#3  0x00007f7508501080 in _d_invariant (o=0x1903130)
    at tango/core/rt/compiler/ldc/rt/invariant_.d:24
#4  0x00007f750850d1cc in _D2rt2gc5basic3gcx2GC6mallocMFmkZPv (this=0x1903130, 
    size=72, bits=1) at tango/core/rt/gc/basic/gcx.d:422
#5  0x00007f7508513d39 in gc_malloc (sz=72, ba=1)
    at tango/core/rt/gc/basic/gc.d:149
#6  0x00007f750850325a in _d_allocclass (ci=0x7f7508aaf110)
    at tango/core/rt/compiler/ldc/rt/lifetime.d:128
#7  0x00007f75084db99e in onAssertErrorMsg (file=..., line=1497, msg=...)
    at tango/core/Exception.d:461
#8  0x00007f75085060b4 in _d_assert_msg (msg=..., file=..., line=1497)
    at tango/core/rt/compiler/ldc/rt/dmain2.d:65
#9  0x00007f750850f57b in _D2rt2gc5basic3gcx3Gcx16thread_InvariantMFZv (this=0x0)
   from /home/luca/tesis/ldc/lib/libtango.so
#10 0x00007f750850e38b in _D2rt2gc5basic3gcx2GC11__invariantMFZv (this=0x1903130)
    at tango/core/rt/gc/basic/gcx.d:273
#11 0x00007f7508501080 in _d_invariant (o=0x1903130)
    at tango/core/rt/compiler/ldc/rt/invariant_.d:24
#12 0x00007f750850d1cc in _D2rt2gc5basic3gcx2GC6mallocMFmkZPv (this=0x1903130, 
    size=72, bits=1) at tango/core/rt/gc/basic/gcx.d:422
#13 0x00007f7508513d39 in gc_malloc (sz=72, ba=1)
    at tango/core/rt/gc/basic/gc.d:149
#14 0x00007f750850325a in _d_allocclass (ci=0x7f7508aaf110)
    at tango/core/rt/compiler/ldc/rt/lifetime.d:128
#15 0x00007f75084db99e in onAssertErrorMsg (file=..., line=1527, msg=...)
    at tango/core/Exception.d:461
#16 0x00007f75085060b4 in _d_assert_msg (msg=..., file=..., line=1527)
    at tango/core/rt/compiler/ldc/rt/dmain2.d:65
#17 0x00007f750850f5dc in _D2rt2gc5basic3gcx3Gcx10initializeMFZv (this=0x0)
    at tango/core/rt/gc/basic/gcx.d:1527
#18 0x00007f750850cccf in _D2rt2gc5basic3gcx2GC10initializeMFZv (this=0x1903130)
    at tango/core/rt/gc/basic/gcx.d:247
#19 0x00007f7508513b85 in gc_init () at tango/core/rt/gc/basic/gc.d:77
#20 0x00007f7508506501 in _D2rt6dmain24mainUiPPaPPaZi6runAllMFZv ()
    at tango/core/rt/compiler/ldc/rt/dmain2.d:265
#21 0x00007f750850641b in _D2rt6dmain24mainUiPPaPPaZi7tryExecMFDFZvZv ()
    at tango/core/rt/compiler/ldc/rt/dmain2.d:227
#22 0x00007f75085063b6 in main (argc=1, argv=0x7fff8bc33278, env=0x7fff8bc33288)
    at tango/core/rt/compiler/ldc/rt/dmain2.d:275
#23 0x00007f7507537abd in __libc_start_main () from /lib/libc.so.6
#24 0x0000000000400959 in _start ()

Looking at the backtrace, it looks like some exception (AssertException?) is being thrown when the GC is initializing.

I have no idea if this is related but I'm pretty sure all module infos need to be collected from all the loaded dynamic libraries and also all the exception handling tables need to collected. Otherwise only the module constructors from the main application will be run and not the ones from the other dynamic libraries, i.e. tango.

Have a look at #1882 where I've added support for dynamic libraries on mac

I managed to *compile* tango as a shared object using the attached Makefile (based on what bob compiles) and applying the patch in bug #1928. I also discovered bug #1929 while trying to figure out why I get a segmentation fault in the main() (the C main, not _d_main()) when _STI_monitor_staticctor() is called.

BTW, this bug is getting big, and is hard to find out what's the current state of affairs. It would be nice if a small comment of the current state of this could be included in the bug description itself (if that's possible).

Makefile used to compile tango as a shared library using dmd

I tried compiling using the makefile on Ubuntu 10.04 with dmd 1.061 and got this error:

/usr/bin/ld: obj/libtango.so: No symbol version section for versioned symbol `_d_throw@4'
/usr/bin/ld: final link failed: Nonrepresentable section on output

how compile tango as a shared object on linux with LDC?