FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

DDL - D Dynamic Libraries
Goto page 1, 2  Next
 
Post new topic   Reply to topic     Forum Index -> DDL - D Dynamic Libraries
View previous topic :: View next topic  
Author Message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Sun Aug 21, 2005 7:17 pm    Post subject: DDL - D Dynamic Libraries Reply with quote

Brad, it looks like I'm going to have to branch into a new project again.

Code:

import ddl.ddl;
import std.stdio;

uint add2(uint a,uint b){
   return a+b;
}

uint add(uint a,uint b){
   return add2(a,b);
}

void main(){
   DynamicLibrary lib = loadDDL("ddltest.obj");

   writefln("\nDDL Exports");
   foreach(ExportSymbol sym; lib.getExports){
      writefln("?s ?0.8X",sym.name,sym.address);
   }

   typedef uint function(uint,uint) AddHandle;
   
   // get the add method from the library
   AddHandle handle = cast(AddHandle)(lib.getExport("_D7ddltest3addFkkZk").address);
   
   // take care
   assert(handle != null);
   
   // behold!
   writefln("\nadd(?d,?d) = ?d",42,69,handle(42,69));
}


Output:
Code:

DDL Exports
__init_11TypeInfo_Pv 007E52F0
__arguments_Aa 007E52E8
_D7ddltest3addFkkZk 007E5411
__arguments_Aaiik 007E5304
_assert_7ddltest 007E551A
__Dmain 007E541D
_D7ddltest4add2FkkZk 007E5400
__arguments_AaAaPv 007E52FC

add(42,69) = 111


The above works with OMF binaries. I plan to write at least an ELF loader, and possibly a COFF loader if time/resources/demand permit.

So what say ye?
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Mon Aug 22, 2005 7:01 am    Post subject: Description Reply with quote

DDL- D Dynamic Libraries

Runtime loading and linking for object files in D. Supports OMF, with planned support for ELF and COFF binary formats.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
AgentOrange



Joined: 21 Jul 2005
Posts: 61

PostPosted: Tue Aug 23, 2005 4:25 pm    Post subject: Reply with quote

This project greatly interests me as I am planning on using D for comercial projects but I very much still desire dynamic libraries. But I wanted to ask you if you have discussed any of this with Walter. Im just curious what his plan is for this. Because I have been meaning to ask him on the newsgroup but Ive noticed most questions he ignores (probably because hes already answered them). So Im just curious if you know if he has said anything on this topic. This is going to have to happen either way, im just wondering if he has any intentions at all.
Back to top
View user's profile Send private message
JJR



Joined: 22 Feb 2004
Posts: 1104

PostPosted: Tue Aug 23, 2005 8:28 pm    Post subject: Reply with quote

One can't wait for Walter to answer. He's trying to manage so many things already but still appears to feel a deep responsibility for single handedly directing D's future.

People eventually realize that if they want to see something important happen for d (from their perspecitve; Walter doesn't always share the myriad of peoples different feelings about what is important, one of his biggest faults Wink ), they'll have to take the initiative on implementing those features themselves (as best they can). Walter appears to be struggling just to keep up with the bug fixes for D, let alone add such features to the dmd tool chain.

Furthermore, Walter usually concentrates most on implementing features that his programming experience marks as critical. Thus, Eric is doing us a huge favour by working on the intricacies of a ddl, something I haven't seen Walter get exceedingly excited about (I think his perspective was "too much work").

I imagine, if all goes well and he succeeds in his task, Eric will be able to present the results to Walter. Walter will either accept it and integrate the ddl protocols with d or not say anything at all. Either way, it doesn't matter. ddl will be so useful, it's likely everybody will want to use the system whether Walter supports it or not.

Yeah, go Eric! Smile

-JJR
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Wed Aug 24, 2005 8:06 am    Post subject: Reply with quote

Thanks for the support guys!

JJR hit the nail right on the head: Walter is simply too embroiled with getting DMD straightened out, that projects like this are simply out of scope for him.

However, there are other constraints that Walter may be under that we're not aware of.

For example, if you were he, and wanted this kind of functionality, wouldn't you just take chunks of OPTLINK and shove them into the D runtime? Now, if you look at how important OPTLINK is to DMC/DMD's success and digitalmars' commerical model, you'll see that it would be a very bad idea . Even rewriting it for D specifically, would simply add another point of maintenence, something that his experience probably wouldn't allow; not to mention it would be invariably derivitave of the C version anyway. Also, there may be issues with OpenWatcomm's not-so-open-source licensing scheme (or a much older agreement), as OPTLINK and OW share some of the same (ancestral) code.

By this I firmly believe that Walter would be violating his own IP by attempting this exercise, one way or another. If if I'm wrong, and he's simply deadlocked due to other responsibilities, the end result holds true. Someone other than Walter had to do something. Smile

Quote:
Thus, Eric is doing us a huge favour by working on the intricacies of a ddl, something I haven't seen Walter get exceedingly excited about (I think his perspective was "too much work").


For someone like Walter, who is doing a ton of work already, its too much to handle. I myself had to basically drop everything to get this prototype out the door in a short timeframe, and its nowhere near done. But if you compare the OMF specification to the ELF one, I think we'd all agree that one of the harder parts is nearly over.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
Derek Parnell



Joined: 22 Apr 2004
Posts: 408
Location: Melbourne, Australia

PostPosted: Wed Aug 24, 2005 6:10 pm    Post subject: Reply with quote

pragma wrote:
For example, if you were he, and wanted this kind of functionality, wouldn't you just take chunks of OPTLINK and shove them into the D runtime? Now, if you look at how important OPTLINK is to DMC/DMD's success and digitalmars' commerical model, you'll see that it would be a very bad idea . Even rewriting it for D specifically, would simply add another point of maintenence, something that his experience probably wouldn't allow; not to mention it would be invariably derivitave of the C version anyway. Also, there may be issues with OpenWatcomm's not-so-open-source licensing scheme (or a much older agreement), as OPTLINK and OW share some of the same (ancestral) code.


OPTLINK is written in optimized assembler. Walter has given me every indication that he is never going to touch the OPTLINK source code again. I've highlighted a number of bugs in it and I don't think he is going to fix those. Its too much trouble. Walter actually suggested that I write a replacement for OPTLINK after I suggested that it should be too hard - stupid me! Wink
_________________
--
Derek
skype name: derek.j.parnell
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Wed Aug 24, 2005 6:53 pm    Post subject: Reply with quote

Derek Parnell wrote:
OPTLINK is written in optimized assembler. Walter has given me every indication that he is never going to touch the OPTLINK source code again. I've highlighted a number of bugs in it and I don't think he is going to fix those. Its too much trouble.


Ahh, that makes sense then. Given the number of options that OPTLINK supports (not to mention full legacy OMF support... yuck!), I guess it really is monster to maintain.

Quote:
Walter actually suggested that I write a replacement for OPTLINK after I suggested that it should be too hard - stupid me! Wink


Laughing Touche'. Having almost written a linker myself, I can attest to how hard that would be.

Although it wouldn't be that bad if you didn't mind not having a life afterwards. Sad
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
AgentOrange



Joined: 21 Jul 2005
Posts: 61

PostPosted: Sun Sep 04, 2005 12:14 am    Post subject: Reply with quote

do you need any help with this project?
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Sun Sep 04, 2005 6:00 pm    Post subject: Reply with quote

AgentOrange wrote:
do you need any help with this project?


As a matter of fact, yes. Out of curiosity, what is your forte' programming wise?

Right now, I am refining the architecture of the project, so that contributions will not disrupt the overall design of things. I should have that published some time this week. Wink

Outside of testing, here is what's needed (in no particular order):

- ELF file loader for PIC-style ELF
- (Microsoft) COFF file loader
- In-situ symbol loader
- Test suite

All of these pieces, except for the COFF loader (DLLs and implib can be used as workarounds), are quite important. They give rise to fully os-independent runtime linking, which should be fairly straightforward once these low-level bits are in place.

The in-situ loader may stay purely theoretical as the project moves forward since there's no clean way to divulge what symbols exist within an EXE (I think I've talked about this elsewhere in the forum). Basically what is needed is a way to resolve a module's dependencies against what is available in the currently running program.

It can be solved by parsing a .map file to determine where all the symbols exist. Another possibility is to require a preprocessor (like Ben Hinkle's dflect) to generate a symbol table for use at runtime. Either way, it's not pretty, but certainly workable.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
AgentOrange



Joined: 21 Jul 2005
Posts: 61

PostPosted: Mon Sep 05, 2005 1:29 am    Post subject: Reply with quote

Well, Ive been programming all sorts of things for a long time mostly in c++, but Ive recently become a very large fan of D. I normally work in game development these days and am quite comfortable working close to the metal. I find this project quite interesting, although I havent fully gotten my head around it yet. I grabbed the code from svn a couple of days ago and I couldnt get the example you posted to work. It didnt see any exports in my modules.

also I really like the idea of having reflective capabilities in the language. I came to D after spending about a year researching languages and compilers in a general plan to roll my own language to take care of about 70? of the crap game developers do in C++ just to get around the language. In my mind the major thing still missing from D is reflection. For my own purposes I intend to use a modified D similar to Dflect if walter never actually implements this, but I believe he has mentioned wanting to have it if possible. I think in the long term reflection is definately a great piece that you need for this project. I think something like Dflect might give a big win to a project like this. just some thoughts.

Ill take a look at the COFF format, that should be pretty straightforward.
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Mon Sep 05, 2005 7:14 am    Post subject: Reply with quote

AgentOrange wrote:
Ill take a look at the COFF format, that should be pretty straightforward.


Excellent, thanks for lending a hand. Please feel free to write here or PM me if you have any questions regarding the library layout. FYI, I'll be breaking down ddl.all.d into additional files, uploading stub code for the loader/linker class tree and adding a /test directory for test code.

Also, I'm sorry you had trouble with the code, last night's code commit was broken. I plan to have this ironed out today. Smile

As for reflection, I couldn't agree more. Ideally, I'd like to see a reflect/emit library that supports (if not defines) the D ABI... it would be a huge boon to the language.
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
AgentOrange



Joined: 21 Jul 2005
Posts: 61

PostPosted: Tue Sep 06, 2005 5:14 am    Post subject: Reply with quote

ok I grabbed the latest but im getting these errors building test

Code:
>build ddltest.d
ddl\omf\OMFLoader.d(31): identifier 'DynamicLibraryLoader' is not defined
ddl\omf\OMFLoader.d(31): DynamicLibraryLoader is used as a type
ddl\omf\OMFLoader.d(31): class ddl.omf.OMFLoader.OMFLoader base type must be class or interface, not void
test\all.d(29): module test3 cannot read file 'test\test3.d'


i looked at it very briefly and couldnt figure out why it doesnt see DynamicLibraryLoader, but i think its a problem in the test because i can build a library out of the ddl source. so its no biggie i just wanted to let you know
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Tue Sep 06, 2005 8:22 am    Post subject: Reply with quote

I ran into this yesterday. Are you using build.exe to compile the source?

Try running it through with -cleanup once, and then try a second time. It should build on the second pass.

I haven't a clue why it screws this up. Any ideas?
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
Derek Parnell



Joined: 22 Apr 2004
Posts: 408
Location: Melbourne, Australia

PostPosted: Tue Sep 06, 2005 6:06 pm    Post subject: Reply with quote

pragma wrote:
I ran into this yesterday. Are you using build.exe to compile the source?

Try running it through with -cleanup once, and then try a second time. It should build on the second pass.

I haven't a clue why it screws this up. Any ideas?


You might also like to use the -full switch to ensure all the object files are up to date.
_________________
--
Derek
skype name: derek.j.parnell
Back to top
View user's profile Send private message
pragma



Joined: 28 May 2004
Posts: 607
Location: Washington, DC

PostPosted: Tue Sep 06, 2005 7:20 pm    Post subject: Reply with quote

Derek Parnell wrote:
pragma wrote:
I ran into this yesterday. Are you using build.exe to compile the source?

Try running it through with -cleanup once, and then try a second time. It should build on the second pass.

I haven't a clue why it screws this up. Any ideas?


You might also like to use the -full switch to ensure all the object files are up to date.


Mr. Parnell, once again I am in your debt. This solved my problem handily. Smile
_________________
-- !Eric.t.Anderton at gmail
Back to top
View user's profile Send private message Yahoo Messenger
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> DDL - D Dynamic Libraries All times are GMT - 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group