FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

segfault in passing arrays a lot

 
Post new topic   Reply to topic     Forum Index -> PyD
View previous topic :: View next topic  
Author Message
hoytak



Joined: 25 Oct 2007
Posts: 2

PostPosted: Thu Oct 25, 2007 11:22 am    Post subject: segfault in passing arrays a lot Reply with quote

Hello,

Not sure if this is the right place for a bug report, but best I could find.

I might be doing something wrong here, but it seems there's a bug in passing lists or arrays to D frequently. Essentially, I can pass arrays until the total number of list elements passed hits somewhere around 128000 and then it segfaults. It happens with lists, arrays, and numpy arrays, and can happen with just one call if the array is large enough.

This code is giving me problems:

For the d module:

////////////////////////////
Code:

module arraysegfault;

import pyd.pyd;

void test(float[] a)
{
}

extern(C) void PydMain() {
  def!(test);
  module_init();
}

/////////////////////////////////

Then running this:
####################
Code:

import arraysegfault as asf

i = 0
while True:
    i += 1
    print i
 
# Doubling the size of the array here halves the number of times I can
# pass it.

    asf.test(range(8))

###################
produces:

/////////////////

...
16440
16441
16442
16443
16444
16445
16446
16447
16448
Traceback (most recent call last):
File "arraysegfault.py", line 8, in <module>
asf.test([1,2,4,5,6,7,8,9])
SystemError: Objects/methodobject.c:112: bad argument to internal function
Segmentation fault (core dumped)

////////////////
With the following core dump:

#0 0xb7c4653e in _D3gcx3Gcx11fullcollectMFPvZk () from /home/hoytak/sysroot/src/pyd/trunk/examples/arraytest/build/lib.linux-i686-2.5/arraysegfault.so
#1 0xb7c46c34 in _D3gcx3Gcx16fullcollectshellMFZk () from /home/hoytak/sysroot/src/pyd/trunk/examples/arraytest/build/lib.linux-i686-2.5/arraysegfault.so
#2 0xb7c47538 in _D3gcx2GC11fullCollectMFZv () from /home/hoytak/sysroot/src/pyd/trunk/examples/arraytest/build/lib.linux-i686-2.5/arraysegfault.so
#3 0xb7c459fa in _D3gcx2GC18fullCollectNoStackMFZv () from /home/hoytak/sysroot/src/pyd/trunk/examples/arraytest/build/lib.linux-i686-2.5/arraysegfault.so
#4 0xb7c44653 in gc_term () from /home/hoytak/sysroot/src/pyd/trunk/examples/arraytest/build/lib.linux-i686-2.5/arraysegfault.so
#5 0xb7c33e5c in _fini () at /usr/local/lib/python2.5/site-packages/celerid/infrastructure/d/python_so_linux_boilerplate.d:15
#6 0xb7ff1a4f in _dl_fini () from /lib/ld-linux.so.2
#7 0xb7d4fd6d in exit () from /lib/libc.so.6
#8 0xb7d3a034 in __libc_start_main () from /lib/libc.so.6
#9 0x08048531 in _start ()

##############################################

Is there a way around this? I'm using D for the computationally expensive processing in a larger python program, so passing a list (or numpy array) to D code happens in the inner loop of the program, so this error is a problem.

Thanks!!!!
--Hoyt
Code:
Back to top
View user's profile Send private message
hoytak



Joined: 25 Oct 2007
Posts: 2

PostPosted: Sat Oct 27, 2007 4:55 pm    Post subject: Update Reply with quote

I played around with it a bit more, and perhaps have some more clues. It seems to be a memory management problem and may or may not be a pyd bug. For instance, it turns out that the following code also causes the segfault problem:

Code:

module arraysegfault;

import std.stdio;
import pyd.pyd;

void test()
{
  double[] ar;
  ar.length = 10;
}

extern(C) void PydMain() {
  def!(test);
  module_init();
}


The code crashes with a similar error after 8092 calls to test() from python. However, it works fine if no dynamic arrays are used.

I should also mention that I'm using pyd revision 131 checked out from the subversion repository, gdc 0.24 as the compiler, and I'm running on linux (Suse 10.2). If anyone wants more info I'd be happy to provide it.
Back to top
View user's profile Send private message
KirkMcDonald



Joined: 22 Jun 2006
Posts: 23

PostPosted: Fri Nov 23, 2007 3:10 pm    Post subject: Reply with quote

Can you duplicate this bug in regular (non-Pyd-using) D code?
Back to top
View user's profile Send private message
mariusmuja



Joined: 22 Nov 2007
Posts: 1

PostPosted: Mon Nov 26, 2007 6:23 pm    Post subject: Reply with quote

This problem seems to be Pyd related. Any time the GC is called (indirectly or directly using std.gc.fullCollect() ) from PyD-using code the program segfaults.

Backtrace:
-------------------
#0 0xb7c0c67e in _D3gcx3Gcx11fullcollectMFPvZk (this=0x808ca70, stackTop=0xbf997d48)
at ../.././libphobos/internal/gc/gcx.d:1643
#1 0xb7c0cd54 in _D3gcx3Gcx16fullcollectshellMFZk (this=0x808ca70) at ../.././libphobos/internal/gc/gcx.d:1582
#2 0xb7c0d658 in _D3gcx2GC11fullCollectMFZv (this=@0x8084dc0) at ../.././libphobos/internal/gc/gcx.d:766
#3 0xb7c0bb3a in _D3gcx2GC18fullCollectNoStackMFZv (this=@0x8084dc0) at ../.././libphobos/internal/gc/gcx.d:784
#4 0xb7c0a793 in gc_term () at ../.././libphobos/internal/gc/gc.d:153
#5 0xb7bfa108 in _fini () from /home/marius/ubc/nn/tmp/pyd/examples/hello/build/lib.linux-i686-2.5/hello.so
#6 0xb7f7a533 in ?? () from /lib/ld-linux.so.2
#7 0xb7f87240 in _rtld_global () from /lib/ld-linux.so.2
#8 0x00000000 in ?? ()
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> PyD All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group