FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Low level utils
Goto page Previous  1, 2
 
Post new topic   Reply to topic     Forum Index -> DDL - D Dynamic Libraries
View previous topic :: View next topic  
Author Message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Wed Jan 18, 2006 12:26 pm    Post subject: Reply with quote

pragma wrote:
larsivi wrote:
I created a ticket for ArgParser and attached my first shot at a generalized version.

http://trac.dsource.org/projects/ddl/ticket/39


Sounds cool. Could you post the modified code here or in the ticket so we can take a peek? Wink


Read that sentence of mine once more time Wink
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Wed Jan 18, 2006 1:17 pm    Post subject: Reply with quote

larsivi wrote:
pragma wrote:

Sounds cool. Could you post the modified code here or in the ticket so we can take a peek? Wink


Read that sentence of mine once more time Wink


Doh. Embarassed

Oh.. we have working attachments now! Awesome.

One critique: I think the current data structure is causing a lot more work than is needed.

Code:
    struct PrefixCallback {
    char[] id;
        char[] prefix;
    ArgParserCallback cb;
     }
    
     PrefixCallback[][uint] bindings;


How about using a cross-reference for the prefixes instead?

Code:
    struct Callback{
        char[] id;
    ArgParserCallback cb;
     }
    
     Callback [][char[]] bindings; // callbacks cross-referenced by prefix, inserted in bind order
     char[][] prefixSearchOrder; // stored prefixes, inserted in bind order


This way, the internal loops become a matter of iterating over the searchOrder then by the bindings for each prefix. This would also give us the effect of having the order of precedence for prefixes become a little more bind-order dependent, instead of purely length dependent.

(almost pseudo-code; feel free to hybridize this a little more)
Code:

foreach(char[] arg; arguments){
    foreach(char[] prefix; prefixSearchOrder){
        if(arg.length < prefix.length) continue;
        if(arg[0..prefix.length] != prefix) continue;

        char[] argData = arg[prefix.length..$];

        while(argData.length > 0){
            bit found = false;
            foreach (Callback cb; bindings[prefix]) { // this is where the x-ref comes in handy!
                // process all the callbacks in this prefix 'family'
            }
            if(!found) throw; /* yadda yadda */
        }
    }
}


This also side-steps the rather nasty side-effect of having same-length prefixes getting glommed together in the same run.

If a non-bind order search is preferred, then I guess the call to parse can be invoked with an option to sort the prefixSearchOrder, just as in your original code. The search order could even be independently set via a different method, say, something that takes a typed vararg or something.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Wed Jan 18, 2006 9:41 pm    Post subject: Reply with quote

I didn't expect my code to be optimal Wink

Anyway, I suppose your structure might be easier/less redundant.

I'm not really in a shape (the flu) to make a good audit on your solution, but I agree that we might just remove the sort on length option. Let's just document that prefixes like "--" must be inserted before "-".

I will try to implement your suggestions sometime during the day.
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Thu Jan 19, 2006 11:47 am    Post subject: Reply with quote

Ok, I've followed your lead and made a new version. In line count, I don't think there was any difference, but the new version should be more efficient as I took out a couple of more cornercases (thanks to your pseudo code).

Your placement of the while-loop was wrong btw, as it is used mostly to distinguish switches without needing whitespace between them.

The new version still works with my simple tests
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sat Jan 21, 2006 5:25 am    Post subject: Re: Low level utils Reply with quote

larsivi wrote:
I think that I will come upon a need for low level access to data in my ELFBinary class. I think the binary object should be exposed through a method in DynamicModule as shown below.

This would mean an abstract class Binary that the Binary classes inherit from. This will also help us with consistency in the different classes over time.

<code>
Binary getBinary(){
return binary;
}
</code>

I'll put it in my ELFModule for now, until we eventually have fleshed out some details. It might only be me who needs this for now.


Ok, I definately think this is necessary (as in the least intrusive way) to create a static elf linker. Will it be in DynamicModule, or should I keep it in ELFModule for now? Actually I think it should be in DynamicModule to at least make it theoretically possible to link OMF and COFF into an ELF executable.
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Sat Jan 21, 2006 10:04 am    Post subject: Re: Low level utils Reply with quote

larsivi wrote:

Ok, I definately think this is necessary (as in the least intrusive way) to create a static elf linker. Will it be in DynamicModule, or should I keep it in ELFModule for now? Actually I think it should be in DynamicModule to at least make it theoretically possible to link OMF and COFF into an ELF executable.


I gather that going strictly by the exposed methods for gathering exported symbols (getExport() and getModuleForExpor()) isn't enough to help resolve the needed fixups?

Its certainly an interesting approach, exposing the underlying binary, but there are a couple problems with that. Most notably is the fact that it makes it impossible to discard the binary file in favor of its in-memory representation. Assuming that we were to go with such an approach. Also, what kind of generic interface could the Binary class provide that isn't already serviced by DynamicLibary and/or DynamicModule?

More to the point, is there some kind of data other than symbol addresses that ELF needs to link? Maybe we can dish that kind of data up instead of the whole binary? Smile
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sat Jan 21, 2006 10:17 am    Post subject: Re: Low level utils Reply with quote

pragma wrote:
larsivi wrote:

Ok, I definately think this is necessary (as in the least intrusive way) to create a static elf linker. Will it be in DynamicModule, or should I keep it in ELFModule for now? Actually I think it should be in DynamicModule to at least make it theoretically possible to link OMF and COFF into an ELF executable.


I gather that going strictly by the exposed methods for gathering exported symbols (getExport() and getModuleForExpor()) isn't enough to help resolve the needed fixups?

Its certainly an interesting approach, exposing the underlying binary, but there are a couple problems with that. Most notably is the fact that it makes it impossible to discard the binary file in favor of its in-memory representation. Assuming that we were to go with such an approach. Also, what kind of generic interface could the Binary class provide that isn't already serviced by DynamicLibary and/or DynamicModule?

More to the point, is there some kind of data other than symbol addresses that ELF needs to link? Maybe we can dish that kind of data up instead of the whole binary? Smile


This isn't about resolving fixups, but creating process images which needs among other things to concatenate the text sections of it's constituent object files. I've not even started trying to resolve those relocation thingies.

Anyway, I'm realizing that I'm probablly better off with the DDLFile, rather than the binary object for this, there is no reason to parse it before it reach the concatenation stage. Unless I'm going to try linking in something else than ELF (sounds unlikely though).

Thus we/I neither need the getBinary nor the Binary abstract class for now. But I'd need a way to access the DDLFile. For single objects I can probably use DDLFile directly, but for archives I need to use the loader.

Hmm, I need some feedback to think more clearly about this Smile
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sat Jan 21, 2006 9:57 pm    Post subject: Reply with quote

Hmm, I'm still agreeing with my last post, I think, but my mind is currently fluctuating wildly. I'm currently not very worried about single object files, as I can work with DDLFiles directly, but archive files might be a tad more difficult. Maybe they should be able to return the raw data of each module in addition to the loaded modules? Or that DynamicModule has a getDDLFile instead of getBinary? Hmm, the possibilities are endless Confused

I'll probably just work around these problems for a while.
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Sat Jan 21, 2006 10:43 pm    Post subject: Reply with quote

Well, one thing that I would suggest is to think of DDLFile as a mango FileConduit instead, as this is what it is going to change to in the next few weeks.

Do what you can to work around the issues and get a working model going, even if it doesn't integrate with DDL 100?. Once its in SVN, we can all take a crack at refactoring the design so its a perfect fit.

Again Lars, thanks for your help.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Mon Jan 30, 2006 12:25 pm    Post subject: Reply with quote

I have committed the new ArgParser version and updated the tools correspondingly. And btw, the commit message hooks works great Smile
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sat Feb 04, 2006 2:19 pm    Post subject: Reply with quote

Hmm, not sure I post in the "correct" thread anymore.

When trying to hack on the linker tool, I've used a getBinary method in DynamicModule to get direct access to the binary, but If we have global attrs like I suggested earlier today (and use them extensively), then a better solution might be to have addModule() or addLibrary() in DynamicLibrary.

Thus, the library could have some logic, looking for the attribute library.addmodule.mode, which can for example be set to "add" which just adds the module, "resolve" to help unresolved symbols, "link" to link it into an already existing binary under production. If just for adding/resolving, it would still be a logic addition to DynamicLibrary. Methinks.

Thoughts?
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Sat Feb 04, 2006 5:29 pm    Post subject: Reply with quote

larsivi wrote:
Hmm, not sure I post in the "correct" thread anymore.


::shrug:: It's okay, just as long as you know where to look for my reply. Wink I can always move off-topic posts around later on.

Quote:

When trying to hack on the linker tool, I've used a getBinary method in DynamicModule to get direct access to the binary, but If we have global attrs like I suggested earlier today (and use them extensively)


Maybe we should take this into a separate thread, but what problem are you trying to solve exactly? Keeping the original binary file around is a bad move IMO, and should be avoided if possible.

Moreover: what is it about ELF that it can't be counted upon to behave just as the OMF implementation does? Is there something special about how to resolve symbols perhaps? Perhaps we should collaborate a bit more on this? Smile

Quote:
, then a better solution might be to have addModule() or addLibrary() in DynamicLibrary.


This *could* be done, but it would likely have to be done at the interface level only, and even then might not benefit all its inheritors. I've tried to refactor the DynamicLibrary base before, by providing a universal means of module storage, with little success mostly because each library type (omf, coff, zip, path, and elf) has its own peculiar storage and lookup schemes.

However, I'm not completely sold on my own opinion of this yet; it could certainly use a second look. Not that it can't be done, but it might screw with things and could ultimately hurt performance. Can ELF get by without this? We can always revisit it after things solidify a bit more?
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sun Feb 05, 2006 3:35 am    Post subject: Reply with quote

pragma wrote:
larsivi wrote:
Hmm, not sure I post in the "correct" thread anymore.


::shrug:: It's okay, just as long as you know where to look for my reply. Wink I can always move off-topic posts around later on.

Quote:

When trying to hack on the linker tool, I've used a getBinary method in DynamicModule to get direct access to the binary, but If we have global attrs like I suggested earlier today (and use them extensively)


Maybe we should take this into a separate thread, but what problem are you trying to solve exactly? Keeping the original binary file around is a bad move IMO, and should be avoided if possible.

Moreover: what is it about ELF that it can't be counted upon to behave just as the OMF implementation does? Is there something special about how to resolve symbols perhaps? Perhaps we should collaborate a bit more on this? Smile

Quote:
, then a better solution might be to have addModule() or addLibrary() in DynamicLibrary.


This *could* be done, but it would likely have to be done at the interface level only, and even then might not benefit all its inheritors. I've tried to refactor the DynamicLibrary base before, by providing a universal means of module storage, with little success mostly because each library type (omf, coff, zip, path, and elf) has its own peculiar storage and lookup schemes.

However, I'm not completely sold on my own opinion of this yet; it could certainly use a second look. Not that it can't be done, but it might screw with things and could ultimately hurt performance. Can ELF get by without this? We can always revisit it after things solidify a bit more?


As I said over there, I've moved to another branch. I'll hack there for a while, and see what shows up/what solutions are presented.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> DDL - D Dynamic Libraries All times are GMT - 6 Hours
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group