View previous topic :: View next topic |
Author |
Message |
phoenix
Joined: 06 Mar 2005 Posts: 68
|
Posted: Thu Aug 16, 2007 9:15 am Post subject: Developer tasks: Descent / Mmrnmhrm integration |
|
|
Having finished a basic code completion feature in Mmrnmhrm (available at SVN), and nearing a 0.1 release, I'm now thinking about what next tasks to take on.
If we want to integrate Mmrnmrhm features into Descent, then, like I said in the other thread, the key to that is the dtool component. dtool is where the parser, ASTs, reference resolver (both for find ref and code completion) are. DTool code is both Mmrnhmrm and Eclipse independent, altough it doesn't work by itself.
But before we even think about integrating semantic features, I would sugest first integrating the parser, as that's pretty much the first line of functionality. I've just finished updating dtool's parser to the latest descent version, and it would be quite nice if you could integrate them. That would mean a few changes (which have already implemented in my branch of the parser):
Adding some visitor classes, and the accept0 methods for DMD Nodes, and a minor hierarchy change (should not bring conflicts).
And if we are to share the same code base, we need to decide how to do it. One way is to split the parser component of descent.core into a separate plugin. Another way is to simply have dtool have descent.core has a dependency. I'm still not sure what would be preferable, so do comment if you have any preference, altough that last option is likely the the best, for starting. In any case, another requirement of integration is that now we would have to be mindful of changes not breaking each other's plugin code (not likely, but could happen).
I'll detail this more on MSN when I have the chance. _________________ Bruno Medeiros |
|
Back to top |
|
|
doob
Joined: 06 Jan 2007 Posts: 367
|
Posted: Thu Aug 16, 2007 1:19 pm Post subject: |
|
|
I'm really looking forward to this |
|
Back to top |
|
|
fraserofthenight
Joined: 08 Apr 2007 Posts: 33 Location: Seattle, WA
|
Posted: Fri Aug 17, 2007 4:08 pm Post subject: |
|
|
That sounds like a really good idea. Ary has the final say, but if it doesn't break any existing functionality, I'm cool with your making any changes you want. I doubt accept0 will break much of anything, so go for it. What hierarchy change were you thinking?
Right now, Ary is working on resolving a couple OutOfMemory issues, so if there's no objections, I can work on dtool integration. I think making dtool dependent on descent.core would be easier for us and would maintain the overall structure of the JDT, however if Mmrnhmrm will continue to live in independence, it may be better to split off the parser.
I think we should also work on some sort of indexing of references. The auto-import feature of the JDT is my favorite part of it by far (I don't think I've written a Java import statement in years...) so I'd like to see that feature implemented as soon as possible. It'd be nice to have markings for unresolved symbols in general, which can be done without complete semantic analysis as long as every reference is bound, though I haven't looked through dtool, so I'm not sure if this is possible with the current implementation (template/mixin expansion especially might make this difficult).
Write me an e-mail - fraserofthenight@gmail.com - and we'll see what we can work out. |
|
Back to top |
|
|
asterite
Joined: 01 Jun 2006 Posts: 235 Location: Buenos Aires, Argentina
|
Posted: Fri Aug 17, 2007 8:38 pm Post subject: |
|
|
In fact, today phoenix integrated he's part of the parser with Descent... since I started to change some things to use char[] instead of String, I still need to fix some problems to commit the final, integrated version.
After that, you could try to get some of dtool functionality, that would be awsome. (we can still continue porting the semantic, since maybe that will get at to the same point as dtool, but with more advantages... I don't know).
I know I've started the project, but I'm realy clueless about some stuff, like how people generaly works with "compiles to native" languages, since I almost always work with Java and C# (that's why integration with a compiler is taking so long). So you guys can make lots of decisions and suggestions, and I'll surely take them if I'm sure I know less than you (which should happen more than 90% of the time ). |
|
Back to top |
|
|
phoenix
Joined: 06 Mar 2005 Posts: 68
|
Posted: Sat Aug 18, 2007 5:09 am Post subject: |
|
|
fraserofthenight wrote: | That sounds like a really good idea. Ary has the final say, but if it doesn't break any existing functionality, I'm cool with your making any changes you want. I doubt accept0 will break much of anything, so go for it. What hierarchy change were you thinking?
Right now, Ary is working on resolving a couple OutOfMemory issues, so if there's no objections, I can work on dtool integration. I think making dtool dependent on descent.core would be easier for us and would maintain the overall structure of the JDT, however if Mmrnhmrm will continue to live in independence, it may be better to split off the parser.
|
Like Ary said, I've already committed my changes to descent.core. The hierarchy change is having:
ASTDmdNode -> ASTNode -> TreeNode & others
The TreeNode class is just to reuse some generic tree behavior, but there shouldn't be any issues nor any differences in using that.
The ASTDmdNode -> ASTNode split is because the nodes from the dtool AST (ASTNeoNode) also inherit from that ASTNode. This is made that way as to allow to convert only some nodes and not all (to improve performance), thus having a "hybrid" AST, but having the visitor and all other mechanism still work correctly. This is not currently done by dtool (all nodes are converted), but there are some nodes (like Expression nodes) which are pratically the same as their dmd conterparts, and with some minor non-structural changes to the original dmd nodes, they wouldn't need to be converted at all. (Unlike the nodes related do definitions and references, whose structure is heavily modified and sanitized) Stuff like this should very important to performance, because having two entire ASTs created is a great performance waste.
I've also converted dtool to be dependent on descent.core, altough I havent commited those changes yet. The plugin binary dependency is not a problem for Mmrnmrhm, only source issues matter. For example, its not critical but it would be nice, from an organization and software engineering perspective, that the parser component of descent.core would be contained in it's own plugin (er even just a source folder) and not have any dependencies on the rest of descent.core. (Currently it depends on DDocComent, AST's api level, PublicScanner and one or other minor classes)
fraserofthenight wrote: |
I think we should also work on some sort of indexing of references. The auto-import feature of the JDT is my favorite part of it by far (I don't think I've written a Java import statement in years...) so I'd like to see that feature implemented as soon as possible. It'd be nice to have markings for unresolved symbols in general, which can be done without complete semantic analysis as long as every reference is bound, though I haven't looked through dtool, so I'm not sure if this is possible with the current implementation (template/mixin expansion especially might make this difficult).
Write me an e-mail - fraserofthenight@gmail.com - and we'll see what we can work out. |
Indexing would be quite nice, altough I was thinking of using for other feature: Open Type. I find very, *very* useful, especially for large projects (like by workspace which has Descent+Mmrnmrm+JDT+Eclipse Platform:P). But it would take some time to implement: unlike other features of JDT which I have examined a bit (like model LRU caching) I haven't looked on how this one works. I only know they search for definitions and then store than info in an index (both memory and disk based I think), but don't know any details.
Also the next feature I'd like to go into, is adding support for definition lookup to expressions nodes, and have a simple builder (invoking bud or something). I also have to start working on my thesis doc.
But you could take a stab at it of course.
BTW, I've just noticed I had the wrong MSN address in my forum profile, I've corrected it, but anyway I've also added you to MSN. _________________ Bruno Medeiros |
|
Back to top |
|
|
stack_overflow
Joined: 18 Aug 2007 Posts: 3 Location: Brasil
|
Posted: Sat Aug 18, 2007 10:41 pm Post subject: |
|
|
Did you have some clue where code completion (intellisense) will be functional in descent?
The project is great! with intellisense it will be the best D IDE ever.
Thanks. |
|
Back to top |
|
|
phoenix
Joined: 06 Mar 2005 Posts: 68
|
Posted: Sun Aug 19, 2007 5:04 pm Post subject: DTLK |
|
|
OMG! I'm so excited with what I've found thid weekend that I'm nearing an orgasm! Did anyone of you knew about the DLTK project:
http://www.eclipse.org/dltk/ ? I just looked into it in more detail this weekend and found it was just what I was looking for! What is DLTK? Basicly, if you check the creation slides, it is, among other things, a framework designed to structure and refactor JDT's code to allow it to be used by other IDEs! It creates an API that client IDEs can extend so as to have:
Quote: | 1 Structured source model, with delta management.
2 Indexed source code. (By default all declarations are indexed). Fast search engine.
3 Many free UI, like ScriptExplorer, Outline, Search and so on. |
This is fraking awesome! And although the name seems to indicate DLTK is made for dynamic languages, for what I've seen, it should be equally useful for a static, compiled language like D. Check this video which demos the Ruby IDE implementation based on DLTK:
http://us.xored.com/~haiodo/EclipseCon07/dltk-ruby.mov
It's pretty awesome for such a recent language as Ruby, and furthermore, I think we could have that same set of features with D, in not too long from now! I've already started porting Mmrnmhrm to DLTK! Check it out, DLTK changes our whole development situation!
A little background on how I came to DLTK:
I was personally never too fond of the ideia of blindly porting JDT over, without external help and without fully understanding it's internal workings (begin the control freak I am ). That was part of the reason I developed Mmrnmhrm seperatly, from scratch, except for the parser(altough also copying some of the few code of JDT I took the time to fully understand, which was mostly UI stuff).
However I gradually (actually recently) realized that it would take a lot of work to do from scratch all that boilerplate UI and core code that JDT (and consequently Descent as well) already had. Especially the core is major feature of Descent, that some users may not be aware. I mean, Descent has many strong points, such as the debbuger integration, the formatter, all the UI boilerplate code supporting the model, the editor, preferences etc., but the core's language element features, such as resource delta processing, model delta generation (for IDE clients), and especially the automatic handling and caching of language elements are all powerful features, and very hard to implement. The last aspect in particular (the caching) should be very important for IDE performance in large projects.
Because of this I favored having dtool's features integrated into Descent, but I did wish there was a more structured (and evolution friendly) way to support all of those JDT common language features. In Mmrnmhrm itself I did already write the code separating D specific parts from language-generic parts (not that I had enough code worth reusing though).
While integrating the parser I dabbled on this, and then I recalled the DLTK project, which I already had heard of several weeks ago, but didn't give it much tought since I thought it was for dynamic languages and did not even see what their framework provided. But this weekend I did stumble upon it, and having looked into it in more detail, I was amazed. The words "Generalizes JDT code and follows its architecture" blew me away. That project makes all the sense for Eclipse. After checking that it could be used for static languages as well (there isn't really any reason why it shouldn't), I was more than sold. I only wish I had found it sooner, as it makes several things in Mmrnmhrm obsolote (and many more in Descent) _________________ Bruno Medeiros |
|
Back to top |
|
|
phoenix
Joined: 06 Mar 2005 Posts: 68
|
Posted: Sat Aug 25, 2007 6:12 am Post subject: |
|
|
A summary of my last talk with Ary:
I've converted Mmrnmhrm to DLTK (available at SVN), preserving all functionality it had before convertion, and also gaining:
* A java-like language model with caching, several kinds of buildpath entries, full UI boilerplate code, (Mmrnmhrm's previous native implementation was very rudimentary in all these aspects.)
* Miscelleanous UI boilerplate code, such a better editor (code folding, quick outline, bracket matching), preference pages for coloring, code templates & compilers.
* Indexing of model type elements (class, structs, etc.) and a working Open Type feature based on it.
This is enough to show that DLTK is viable and worthwhile. Plus the fact that DLTK has just been accepted as an Eclipse Foundation project means it should have reliable support in the future, should any issues arise.
I'm looking next to integrating Type Hierarchy and Search features, (but right now I'm going to have to suspend that and work on my thesis document which is late )
Note: one does not have to use DLTK AST hierarchy to integrate with DLTK (except using ASTNode as a base class), it just means one will have to do more work to integrate the other semantic features of DLTK if you don't use their hierarchy. _________________ Bruno Medeiros |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|