FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Library naming convention implementation
Goto page 1, 2  Next
 
Post new topic   Reply to topic     Forum Index -> Build
View previous topic :: View next topic  
Author Message
jcc7



Joined: 22 Feb 2004
Posts: 657
Location: Muskogee, OK, USA

PostPosted: Wed Sep 20, 2006 7:11 am    Post subject: Reply with quote

Gregor wrote:
There is currently no convention or even a semblance of a convention, but there are also few-to-no libraries that generate linkable .a or .so files. That makes this the perfect time to at least loosely enforce one. Not even attempting to enforce any sort of convention is a very effective way to shoot yourself in the foot.
I think you've missed what makes build so popular. Build lets the user easily do what he wants to do. Build is highly customizable to do whatever the user wants. Source files can be stored wherever the user wants to store them. It doesn't force people to set up their source tree in a certain way. Flexible is a key to its success.

Some other tool can be in charge of enforcing conventions, but that's not what the role of build has been.

Has dsource.org's clock been reset? Why is my reply above the message I'm trying to reply to? (Have I discovered time travel?)
Back to top
View user's profile Send private message AIM Address
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 9:45 am    Post subject: Library naming convention implementation Reply with quote

I'd like to start implementing my library naming convention, as documented at http://www.prowiki.org/wiki4d/wiki.cgi?LibraryNamingConvention , and would like to know if:

1) You'd be at all willing to accept (hopefully fairly well-written) patches implementing this in build.
2) You have any interest in, after my patches are working on GDC on GNU/Linux, porting it to DMD and/or GDC on Windows (I have no knowledge of DLL's, no desire to learn about their inner workings, and no system to do so anyway)
Back to top
View user's profile Send private message AIM Address
Derek Parnell



Joined: 22 Apr 2004
Posts: 408
Location: Melbourne, Australia

PostPosted: Wed Sep 20, 2006 10:35 am    Post subject: Re: Library naming convention implementation Reply with quote

Gregor wrote:
I'd like to start implementing my library naming convention, as documented at http://www.prowiki.org/wiki4d/wiki.cgi?LibraryNamingConvention , and would like to know if:

1) You'd be at all willing to accept (hopefully fairly well-written) patches implementing this in build.
2) You have any interest in, after my patches are working on GDC on GNU/Linux, porting it to DMD and/or GDC on Windows (I have no knowledge of DLL's, no desire to learn about their inner workings, and no system to do so anyway)


I've read through it once so far, and it did not seem to be a useful idea. I'll read it again and try to see why its a good idea. So give me some time to digest its ramifications.

Anyhow, if implemented, it would be optional. Some people don't like other people's naming conventions and I'm not about to start dictating.

I have no knowledge about how Linux does libraries and why should it matter?
_________________
--
Derek
skype name: derek.j.parnell
Back to top
View user's profile Send private message
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 10:59 am    Post subject: Re: Library naming convention implementation Reply with quote

Derek Parnell wrote:
I have no knowledge about how Linux does libraries and why should it matter?


This is absolutely not a UNIX standard. I've layered my library naming convention on top of UNIX's standard library naming convention, but that's sheerly a matter of convenience, the actual naming is non-platform-specific.

Derek Parnell wrote:
I've read through it once so far, and it did not seem to be a useful idea. I'll read it again and try to see why its a good idea. So give me some time to digest its ramifications.


Currently, there's basically no support whatsoever for libraries. I do a lot of software building and software design, and so I have a somewhat unique view on the building of software. There's nothing more obnoxious than having no library naming convention for a number of reasons that I won't delve deeply into, not the least of which is generating proper library file names on different platforms with slightly different ideas of how everything should work.

build at present, as far as I know, is generally used to compile almost exclusively source files into one binary, with no shared library support (or even libraries at all). That is, to build against mango for example, one traditionally copies mango into some include path, then -I's it in with build, which causes build to build the pieces from source. This works fine, but is reminiscent of DOS or UNIX from twenty years ago: we have shared libraries now.

I see no compelling reason to go into depth on why shared libraries are good, anyone ought to be able to see this, and there are about a billion links on the internet that will explain it in full.

The problem with shared libraries is that there is in general no implied association between source file names and library file names. This is part of why we have autoconf et cetera. In the Windows world, this is maintained via Microsoft's iron fist, so the problem may be less pronounced there.

By implementing a convention which links package names in D to library names, one can use shared libraries with no knowledge or need for knowledge of who actually implemented it, and it will just work. Because build currently usually uses the source, this isn't a problem, but when D shared libraries become more common (and if they don't, this language is dead in the water), it's very important that either method can work with no change to the source.

Derek Parnell wrote:
Anyhow, if implemented, it would be optional. Some people don't like other people's naming conventions and I'm not about to start dictating.


There is currently no convention or even a semblance of a convention, but there are also few-to-no libraries that generate linkable .a or .so files. That makes this the perfect time to at least loosely enforce one. Not even attempting to enforce any sort of convention is a very effective way to shoot yourself in the foot.
Back to top
View user's profile Send private message AIM Address
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 11:00 am    Post subject: Reply with quote

jcc7 wrote:
Gregor wrote:
There is currently no convention or even a semblance of a convention, but there are also few-to-no libraries that generate linkable .a or .so files. That makes this the perfect time to at least loosely enforce one. Not even attempting to enforce any sort of convention is a very effective way to shoot yourself in the foot.
I think you've missed what makes build so popular. Build lets the user easily do what he wants to do. Build is highly customizable to do whatever the user wants. Source files can be stored wherever the user wants to store them. It doesn't force people to set up their source tree in a certain way. Flexible is a key to its success.


Using libraries is an alternative to the build-everything-from-source style. It's not intended to replace what build does, but to add to it.

And actually, by not supporting libraries or requiring that users do a nasty manual build/find of libraries, build is being significantly less flexible.

jcc7 wrote:
Some other tool can be in charge of enforcing conventions, but that's not what the role of build has been.


Thereby making them not standards and ruining the entire purpose.

jcc7 wrote:
Has dsource.org's clock been reset? Why is my reply above the message I'm trying to reply to? (Have I discovered time travel?)


Yeah, that's mighty weird ... wonder where this one will land.
Back to top
View user's profile Send private message AIM Address
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 12:02 pm    Post subject: Reply with quote

(Second response, necessary ^^)

jcc7 wrote:
Build is highly customizable to do whatever the user wants. Source files can be stored wherever the user wants to store them.


The library naming convention has no bearing whatsoever on source files.

jcc7 wrote:
It doesn't force people to set up their source tree in a certain way.


The library naming convention has no bearing whatsoever on the arrangement of the source tree.

jcc7 wrote:
Flexible is a key to its success.


Not when rigid is a huge boon to usability with absolutely no disadvantage.

I should also point out that D itself is fairly unflexible in the arrangement of the source tree, but you don't seem to be complaining about that. And furthermore, all I'm proposing is a standard for mapping package names to library names, which the source maintainer has clearly chosen already (seeing as that they're the source maintainer's package names). Does this limit the possible names for library files created without hackery? Yes. Is this a bad thing? NO, NO, NO

jcc7 wrote:
Some other tool can be in charge of enforcing conventions


This makes no sense. Having a tool in charge of enforcing the convention but not implementing the convention elsewhere is ridiculous. It defeats the purpose of a convention.

jcc7 wrote:
, but that's not what the role of build has been.


No, it hasn't. And therefore, we should refuse to change for the better, right?
Back to top
View user's profile Send private message AIM Address
Derek Parnell



Joined: 22 Apr 2004
Posts: 408
Location: Melbourne, Australia

PostPosted: Wed Sep 20, 2006 12:08 pm    Post subject: Reply with quote

Ok, I've read it a few more times now.

I do not understand what problem this is a solution for.

Can you explain the problem you are experiencing? And not in terms of how it should be solved; just the problem please.

Gregor wrote:

Using libraries is an alternative to the build-everything-from-source style. It's not intended to replace what build does, but to add to it.


If one is not building from source code, what is one building from? The D compiler only uses source code. Shared libraries are not used in the compilation or the linkage process. Or are you talking about creating shared libraries from source code? I'm confused.

Gregor wrote:
And actually, by not supporting libraries or requiring that users do a nasty manual build/find of libraries, build is being significantly less flexible.


I don't understand what you mean by 'support'? Are you referring to using shared libraries during Build's processing?, or during compilation, or during linkage? or what? Or are you refering to creating libraries, shared and/or static?

Gregor wrote:

jcc7 wrote:
Some other tool can be in charge of enforcing conventions, but that's not what the role of build has been.


Thereby making them not standards and ruining the entire purpose.

Ruining what purpose? The Build utility's job is to read SOURCE CODE to work out what needs to be COMPILED and LINKED. And then to invoke the tools to do that compilation and linkage. How do shared libraries fit in to that purpose?

In you wiki article, you say that given the source line
Code:

import a.b.c.d.e.f.g;

that Build needs to look for '.a' and 'so' files? Why would it do that? It can't read those files, as it can only read SOURCE files. If it get such an import statement, it needs to find where a/b/c/d/e/f/g.d is located and read it. That is why -I is required so it knows what relative directories to scan through.
_________________
--
Derek
skype name: derek.j.parnell
Back to top
View user's profile Send private message
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 12:17 pm    Post subject: Reply with quote

My purpose is to make library files an alternative to building everything from source. The two components of that are:

1) Building library files from a source directory.
2) Linking against said library files while building another program.

I don't care how people name source files, so long as they conform to D's import convention (that is, directories = packages, subdirs = subpackages, .d files = modules). I don't care where people put .di files (which would be necessary for library linking) so long as they are on some include path (though this could be automated in the future).

My ONLY PURPOSE at present is to make linking against libraries a transparent alternative to building entirely from source.

This involves slight modifications in the compiling and heavy modifications in the linking phases you describe.

As per what "problem" I'm solving, I'm trying to prevent a problem from happening, not solve one. Eventually, people will start making library files with D. I've had enough experience with bizarre conventions, so I want to make sure that there's a simple, sane convention in place before (shared) libraries become prevalent.

It seems to me like the most logical place for these changes to be made is the tool used to build software - and the de facto standard for that is build.


In short: build will still do what it does. But it will also support building libraries and building with libraries.


EDIT: Tpyo.
Back to top
View user's profile Send private message AIM Address
Derek Parnell



Joined: 22 Apr 2004
Posts: 408
Location: Melbourne, Australia

PostPosted: Wed Sep 20, 2006 1:48 pm    Post subject: Reply with quote

I think I can see the issues now. So bear with me a bit longer... Smile

Ok, so we have a library has already been created. It contains some modules that we wish to use in an application we are building. We don't want to recompile those modules - either because there is no need to or we don't have the source code for them. But we do have the .di files for them.

This means that the compiler needs to know where those .di files are, and the linker needs to know where the library file(s) are. And all we have are import statements.

Ok, so far so good. The -I switch tells the compiler the root(s) to scan for the .di files. Here is where your suggestion comes into it (I think). If the Build program can also use those roots (and sub-roots) to look for library files, it can add those libraries to the linker command line.

In summary, given
Code:

import a.b.c.d.e.f.g;

Build can scan through a/, a/b/, a/b/c, a/b/c/d/, a/b/c/d/e/, and a/b/c/d/e/f/ looking for any library files residing in those directories, and if found add the path and library name to the linker command line.

I hope this is what you are talking about. Smile

I don't see what naming convention has got to do with it though. Unless you expect Build to take into consideration version numbers embedded in the library file name, so that it selects the most recent version? Not sure how this would work.
_________________
--
Derek
skype name: derek.j.parnell
Back to top
View user's profile Send private message
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 2:08 pm    Post subject: Reply with quote

That's mostly what I'm saying, with one exception: The libraries don't form a directory hierarchy like you're suggesting. This can make linking against shared object files at runtime a nightmare.

The naming convention is that the package name is encoded into the library file name. So, with
Code:
import a.b.c.d.e.f.g;


it will look through every library file name, from most specific to most general, for one that exists. If it exists, and is following the naming convention, it contains all the symbols being imported.

The encoding of the package name into the library name is simple, the library is named (under UNIX):
libD.<package name>.<extension>

So, for a.b.c.d.e.f.g, it would look at:
libD.a.b.c.d.e.f.<valid extensions>
libD.a.b.c.d.e.<valid extensions>
etc.

The other side of this is that build ought to be able to produce these libraries as well, and this production should be painless. Just add some flag, and it'll turn every package into a library (or further subdivide if subdivisions are specified by the upstream author).
Back to top
View user's profile Send private message AIM Address
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 2:09 pm    Post subject: Reply with quote

Erm, lemme be a bit more specific: Every top-level package is (by default) turned into a library.
Back to top
View user's profile Send private message AIM Address
Derek Parnell



Joined: 22 Apr 2004
Posts: 408
Location: Melbourne, Australia

PostPosted: Wed Sep 20, 2006 2:25 pm    Post subject: Reply with quote

Gregor wrote:
That's mostly what I'm saying, with one exception: The libraries don't form a directory hierarchy like you're suggesting. This can make linking against shared object files at runtime a nightmare.


Huh? Hang on. Linking at runtime is NOT the responsibility of Build. The application controls this and Build doesn't know anything about it. Build only deals with static linkage. It cannot deal with runtime linkage because the location and library names are determined at run time and not compile time.

Gregor wrote:
The naming convention is that the package name is encoded into the library file name. So, with
Code:
import a.b.c.d.e.f.g;


it will look through every library file name, from most specific to most general, for one that exists. If it exists, and is following the naming convention, it contains all the symbols being imported.

But where will it look for these library files? I mean, what is the directory, or list of directories, that it should scan through looking for libraries?

Gregor wrote:
The encoding of the package name into the library name is simple, the library is named (under UNIX):
libD.<package name>.<extension>

But why do they have to be named so? Is it so the humans can look at the name and make a reasonable guess as to what modules it should contain? The linker doesn't give shit about the name so any library found in the expected places could be used.

Gregor wrote:

So, for a.b.c.d.e.f.g, it would look at:
libD.a.b.c.d.e.f.<valid extensions>
libD.a.b.c.d.e.<valid extensions>
etc.


But in which directory is "libD.a.b.c.d.e.f.<valid extensions>" expected to be found in?

Gregor wrote:
The other side of this is that build ought to be able to produce these libraries as well, and this production should be painless. Just add some flag, and it'll turn every package into a library (or further subdivide if subdivisions are specified by the upstream author).

Hmmmm... got to think that idea through a bit more, I suspect.
_________________
--
Derek
skype name: derek.j.parnell
Back to top
View user's profile Send private message
Gregor



Joined: 05 May 2006
Posts: 72
Location: Portland, OR

PostPosted: Wed Sep 20, 2006 2:34 pm    Post subject: Reply with quote

Through this entire discussion, you've given no thought to shared object files. This is not good, nor is it promising. D should be capable of using .so and .dll files.

Derek Parnell wrote:
Gregor wrote:
That's mostly what I'm saying, with one exception: The libraries don't form a directory hierarchy like you're suggesting. This can make linking against shared object files at runtime a nightmare.


Huh? Hang on. Linking at runtime is NOT the responsibility of Build. The application controls this and Build doesn't know anything about it. Build only deals with static linkage. It cannot deal with runtime linkage because the location and library names are determined at run time and not compile time.


This assertion is valid for linking against static libraries, but falls flat with shared libraries. The runtime linker expects shared libraries to be in a standard location, so the compile-time linker also needs to expect them to be in a standard location.

Derek Parnell wrote:
Gregor wrote:
The naming convention is that the package name is encoded into the library file name. So, with
Code:
import a.b.c.d.e.f.g;


it will look through every library file name, from most specific to most general, for one that exists. If it exists, and is following the naming convention, it contains all the symbols being imported.

But where will it look for these library files? I mean, what is the directory, or list of directories, that it should scan through looking for libraries?


The standard system library path.

Derek Parnell wrote:
Gregor wrote:
The encoding of the package name into the library name is simple, the library is named (under UNIX):
libD.<package name>.<extension>

But why do they have to be named so? Is it so the humans can look at the name and make a reasonable guess as to what modules it should contain? The linker doesn't give shit about the name so any library found in the expected places could be used.


The runtime linker does care when they're shared object files, that is, .so and .dll files.

Derek Parnell wrote:
Gregor wrote:

So, for a.b.c.d.e.f.g, it would look at:
libD.a.b.c.d.e.f.<valid extensions>
libD.a.b.c.d.e.<valid extensions>
etc.


But in which directory is "libD.a.b.c.d.e.f.<valid extensions>" expected to be found in?


The standard system library path.

Derek Parnell wrote:
Gregor wrote:
The other side of this is that build ought to be able to produce these libraries as well, and this production should be painless. Just add some flag, and it'll turn every package into a library (or further subdivide if subdivisions are specified by the upstream author).

Hmmmm... got to think that idea through a bit more, I suspect.


You have to or I have to? It's a workable, logical plan as far as I'm concerned.
Back to top
View user's profile Send private message AIM Address
aldacron



Joined: 05 May 2004
Posts: 1322
Location: Seoul, South Korea

PostPosted: Wed Sep 20, 2006 5:46 pm    Post subject: Reply with quote

Gregor wrote:
Through this entire discussion, you've given no thought to shared object files. This is not good, nor is it promising. D should be capable of using .so and .dll files.

...

This assertion is valid for linking against static libraries, but falls flat with shared libraries. The runtime linker expects shared libraries to be in a standard location, so the compile-time linker also needs to expect them to be in a standard location.

...

The standard system library path.



It seems to me you are suggesting that DMD should be able to accept shared libraries on the command line at compile time. If that's what you are getting at, I can see where the confusion is coming from.

First, that's a definite Unixism. You are talking about standard library names, runtime linkers and standard library paths. That's all in the Linux world. Derek comes from the Windows world (so do I, which is why this conversation is causing my head to spin). I think your intent is getting lost in translation. The only compiler I know of on Windows that can accept a DLL on the command line is MingW -- and most people don't even know that since I often find myself bringing it up in different forums.

On the Unices, you can pass a shared object file to GCC at compile time and the runtime linker will pull it in at runtime. Windows doesn't function that way. On Windows, you link to a static library which contains the DLL exports and the DLL is loaded at runtime. The term "runtime linker" doesn't exist in the Windows world AFAIK.

Second, there's no naming convention on Windows for library files or DLLs, nor is there a standard library path. You can set up a LIBRARY environment variable that some compilers will use, but there is certainly no standard path. Windows has a standard search pattern for DLLs at runtime, starting with the working directory, so most apps just drop any non-system DLLs there. But there certainly isn't any naming convention, nor does there need to be.

Finally, I don't think it is clear at all what you are proposing that Build should do. Are you suggesting that Build should pass any shared library files the app requires on to the compiler as it does already for static libraries?

This is a confusing conversation. If you could be a little more clear about exactly what you want Build to do with shared object files it would be helpful.
_________________
The One With D | The One With Aldacron | D Bits
Back to top
View user's profile Send private message Send e-mail
aldacron



Joined: 05 May 2004
Posts: 1322
Location: Seoul, South Korea

PostPosted: Wed Sep 20, 2006 5:53 pm    Post subject: Reply with quote

Gregor wrote:
My purpose is to make library files an alternative to building everything from source. The two components of that are:

1) Building library files from a source directory.
2) Linking against said library files while building another program.


Build can already build libraries from a source directory and link against them when building another program. It only builds an executable when it encounters a main or WinMain method.

Quote:
My ONLY PURPOSE at present is to make linking against libraries a transparent alternative to building entirely from source.


What does this mean?

Quote:

This involves slight modifications in the compiling and heavy modifications in the linking phases you describe.

As per what "problem" I'm solving, I'm trying to prevent a problem from happening, not solve one. Eventually, people will start making library files with D. I've had enough experience with bizarre conventions, so I want to make sure that there's a simple, sane convention in place before (shared) libraries become prevalent.

It seems to me like the most logical place for these changes to be made is the tool used to build software - and the de facto standard for that is build.


Wouldn't the better place for any changes needed be in the compiler and the linker? What does patching Build have to do with anything?


Quote:
In short: build will still do what it does. But it will also support building libraries and building with libraries.


Build already does support that. And if the compiler supports building shared libraries, build supports that, too. If the compiler accepts shared libraries on the command line, Build supports that, too.
_________________
The One With D | The One With Aldacron | D Bits
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> Build All times are GMT - 6 Hours
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group