FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Mango DLL

 
Post new topic   Reply to topic     Forum Index -> Mango
View previous topic :: View next topic  
Author Message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Tue Aug 31, 2004 7:31 pm    Post subject: Mango DLL Reply with quote

Kris, all,
I came across the idea, in working on DSP, to place my custom ServletProvider, ServletContext, and the rest, in a .dll so I may hook in the custom classes dynamically. My motivation behind this is to move toward a generic, configuration-driven server, which is at least a good direction for DSP (it'll make hooking Apache easier when I dare to try that again)... and maybe for future versions of Mango?

Kris, I believe I wrote you on this earlier... I hope you don't me posting it here. Smile

At any rate, I've stumbled across a weak spot in Mango when it comes to cramming half a server into a .dll. ServletContext likes to throw an exception in its static constructor on line 198.

Code:
logger = Logger.getLogger ("mango.servlet.Context");


It tears down the entire dll load process, and the error is tough to catch since the "standard" dll stub (on the digitalmars site) doesn't catch any errors. That's about all I have. I assume that this has something to do with not having a complete server inside the same link unit. Any ideas?

Also, are there any other caveats to mixing Mango with a .dll that anyone might know about?

Thanks in advance.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Tue Aug 31, 2004 8:40 pm    Post subject: Reply with quote

I'd certainly very much like to package Mango as a series of related DLLs. Unfortunately, the D compiler is too immature at this stage to contemplate doing that. Getting the GC and supporting cast into a DLL will be a start in that direction.

So, what is the exception? Is it a GPF? If so, it's very likely to be a static-initializer ordering issue (as has been noted a few times on the NG buglist).

The code snippet identified is setting up a singleton ... however, you could try moving it to the non-static constructor (immediately below) if you like. If it works at that point, then you've identified another initialization ordering issue.

If you do try that out, please post your results?

Thanks Eric ...
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Wed Sep 01, 2004 6:09 am    Post subject: Reply with quote

It's actually an access violation. I modified the dll stub to catch exceptions, which will at least tell me tha this is happening. Sadly, there's no context information on the exception itself, so I haven't a clue where its happening.

I agree that D's dll handling is immature for a whole host of reasons, but it is adequate for certain tasks. I haven't had too much trouble considering what I've put my compiler through.

Thanks for the info, I'll try that tonight.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Wed Sep 01, 2004 9:58 am    Post subject: Reply with quote

pragma wrote:
It's actually an access violation. I modified the dll stub to catch exceptions, which will at least tell me tha this is happening. Sadly, there's no context information on the exception itself, so I haven't a clue where its happening.

Yep; sounds like the static initialialization order problem. What generally happens is that the target (Logger, in this case) has not yet executed it's own static constructor before being asked to do something from another static constructor (ServletContext).

The expected initialization sequence is 'imports' first, followed by local lexical scope. However, both Andy Freisen and I have seen this break. Looks like you are seeing it also.

The only other option I can think of is that 'Logger' has not been resolved by the linker, and hence is null. I don't know how you'd manage to sneak that past the linker, but anything is possible.

If you have suggestions for how to resolve this in a more strategic manner, then fire away! For example, one might decouple ServletContext from Logger altogether by providing a method to set an ILogger. Unfortunately, that allows the user to set a null (which could be tested for, but it's not as clean as the static assignment), not set one at all (which would be bad), or set a logger with a "non standard" identity (which might actually be useful). The current setup tried to avoid those questions by simply setting the default for all contexts and moving on. It can easily change Wink
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Wed Sep 01, 2004 2:24 pm    Post subject: Reply with quote

Quote:
If you have suggestions for how to resolve this in a more strategic manner, then fire away!


Well, since you asked, my money's on the compilation/link order of the files. I really need to do some research on this to find out what is exactly wrong here, but Ihere's my $0.02 on the matter:

Arrow It wouldn't suprise me if that optlink is dutifully resolving the symbol dependencies between modules, but is clueless as to what order they are interdependent.

C manages to solve this static init problem completely by accident: #include is so braindead that you can't have cyclic includes anywhere without crushing your project. The preprocessor aggregates all these includes together to make (or simulate) one monster file for compilation. The result is that you can't reference static bits that aren't declared (and created?) ahead of where they're referenced.

Java sidesteps this in its classloader by running static inits when the .class is loaded into the VM. Reference any one class that's not already loaded and, *bang* its static code is ready to go simultaneously.

D's 'import' is a mixture of the two paradigms: it references code that is used like a roadmap (like #include) for a binary that is provided later (like Java only compiled in). This makes things work great during compilation, but falls apart once you go to link since all the linker does is glue code together and make sure nothing's missing. All the run-time semantics of what's needed first (like in Java) are totally absent in a compiled D program because of this. Furthermore, the only formal representation of these interdendencies is in the include statements that dissapear once compilation is complete.

Much like with the rest of D, this is a case where the tools "Trust the programmer". This is of little confort in this case: in reality, it feels more like I'm playing a runaway game of fizban with the compiler.

Idea IMO, a build tool that analyzes the import statements of .d files could easily fix all of this, provided that initalizers are run in link-order. Wink

Along that line, I'm thinking that changes in the makefile, and/or the order of imports in the various source files may be a good fix for now (but you'll have to abandon using response files... *retch*).

Also, we can sidestep the problem by rolling a static mangoInit() somewhere (staticinit.d?) and manually running all the module initalizers (in an explicit order) from that one static init routine. Ugly yes, but at least that's guaranteed to work.

I also have an idea for a SystemInitalizer class that would pay attention to any dependencies that are given it. The problem is: how to we make sure it runs *last*? Sad
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Wed Sep 01, 2004 3:38 pm    Post subject: Reply with quote

pragma wrote:
Quote:
If you have suggestions for how to resolve this in a more strategic manner, then fire away!

Well, since you asked, my money's on the compilation/link order of the files

Could be. As I understand it, the startup code has a list of modules to be initialized which, in turn, invoke their local static constructors. These are all invoked pre-main(), and the module + static destructors (compiled during the startup phase) are invoked post-main().

How are you getting the static constructor invoked for SevletContext within the DLL? Are you explicitly calling this?

Code:

 _moduleCtor();         // run module constructors

Would be handy to know exactly what is causing your GPF; whether Logger.getLogger() is actually called, or if Logger itself is actually null. Let me know when you've tried moving that call to the class contructor instead
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Thu Sep 02, 2004 6:14 am    Post subject: found it. Reply with quote

Looks like moving the call to the (non-static) constructor worked.

Manager.base is being referenced in a call to getRootLogger(), before that class' static constructor is being run. I've confirmed this by placing printf statements in static constructors in multiple files. ServletContext's static init runs first.

I did my best to jockey files around in the makefile, and sort the response file for making mango.lib in the order of creation (ls -Ts -r $(OBJ)*.obj > mango.rsp), to no avail. Seems like the order inwhich the files are presented doesn't have as much to do with things as I thought.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Thu Sep 02, 2004 10:26 am    Post subject: Re: found it. Reply with quote

pragma wrote:
Looks like moving the call to the (non-static) constructor worked.

Manager.base is being referenced in a call to getRootLogger(), before that class' static constructor is being run. I've confirmed this by placing printf statements in static constructors in multiple files. ServletContext's static init runs first.

I did my best to jockey files around in the makefile, and sort the response file for making mango.lib in the order of creation (ls -Ts -r $(OBJ)*.obj > mango.rsp), to no avail. Seems like the order inwhich the files are presented doesn't have as much to do with things as I thought.

Right. That's the static initialization bug that's been around for ages. Andy Freisen posted recently about it also:
Quote:
news:cg6p3m$1c7a$1@digitaldaemon.com
news:c9h3us$2qjl$1@digitaldaemon.com
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Thu Sep 02, 2004 12:46 pm    Post subject: Reply with quote

Idea Kris, I may have found out why this happens.

I did a little experiement to find out why this is happening. The problem isn't so much a bug but rather an unmentioned caveat to static code run at startup.

Exclamation In short: modules without module constructors cannot be prioritized, nor will their dependencies be prioritized appropriately during startup.

The following snippet will output all of the module dependencies that the module-init startup code uses. The constructors are run in rightmost tree-traversal-order and will not attempt to re-run anything that has already been completed (see std.moduleinit.d for details).

Code:
import std.stdio;
import std.moduleinit;

void main(){
   writefln("\n--MODULE INFO--\n");
   
   foreach(ModuleInfo info; _moduleinfo_array){
      writefln("?s", info.name);
      foreach(ModuleInfo imported; info.importedModules){
         writefln("\timport ?s",imported.name);
      }
   }
}


I used this and a collection of very simple files that simulated the relationship between ServletContext, Logger and Manager.

If my experiment is correct, running this against mango.lib will show that ServletContext does not show Logger as a dependency despite explicitly calling static code in that module. This is because Logger doesn't have a module constructor so it cannot create a dependency for Manager. This makes sense since dmd can only add module constructors to the init list, and Logger simply doesn't have one.

So the fix is somewhat involved for mango, but not impossible:
- Add an empty module constructor to Logger (since it is used during module construction elsewhere)
- Eliminate the mutual dependency between Manager and Logger (will get runtime errors due to the module-ctor cycle between these two)

And that should take care of it. Smile

Update:
'private import' also prevents the inclusion of a module dependency. You may have to relax select 'private import' statements to 'import' for now to satisfy the compiler. FYI I've posted all this to the bugs newsgroup. Cool
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Thu Sep 02, 2004 3:57 pm    Post subject: Reply with quote

pragma wrote:
Idea Kris, I may have found out why this happens.

I did a little experiement to find out why this is happening. The problem isn't so much a bug but rather an unmentioned caveat to static code run at startup.

Exclamation In short: modules without module constructors cannot be prioritized, nor will their dependencies be prioritized appropriately during startup.

Nice one.

I don't think that's quite correct though. First, there are three kinds of constructor (and hence destructor):

static module
static class
class

In Mango, usage of the first is limited to 1 or 2 modules (Christopher Miller's Socket.d for example). Mango does, however, have a number of the second variety; it is these static class constructors that are causing folks various problems.

As I understand it, you're saying that such constructors will not be run unless there's a static module constructor also? Well, the compiler will silently create one for you if you have static class constructors. The static module constructor then invokes each of the static class constructors within said module, in lexical order. That is, static class constructors will be invoked regardless of whether or not there's an explicit static module constructor (as one would certainly hope!).

The caveat appears to come in at a later point. Apparently the entries in _moduleinfo_array (built by the compiler) are optimized to include only those that require static construction in one or both forms noted above. That's cool; the issue lies with the importedModules[] of each entry: only imports sans attributes are noted ... if you declare the import as public, private or package, it is excluded from the importedModules[] and the dependency order is thus lost. A rather glaring bug, I'd say!

In fact, entire chains of dependencies are lost. It's astonishing that it ever worked where private imports are applied liberally (which Mango does).
Back to top
View user's profile Send private message
kris



Joined: 27 Mar 2004
Posts: 1494
Location: South Pacific

PostPosted: Fri Sep 03, 2004 6:34 pm    Post subject: Reply with quote

pragma wrote:
Update:
'private import' also prevents the inclusion of a module dependency. You may have to relax select 'private import' statements to 'import' for now to satisfy the compiler. FYI I've posted all this to the bugs newsgroup.

Looks like we both came to a similar conclusion at the around the same time Cool

Walter will most likely have this bug fixed within a week or two, and I'd rather not weaken the namespace isolation by removing the private import attribute. In the interim, can you temporarily work around this by moving Logger.getLogger() into the class constructor? If not, let me know so we can try something else instead.

- Kris
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> Mango All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group