FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Statement...the pain.
Goto page Previous  1, 2, 3
 
Post new topic   Reply to topic     Forum Index -> DDBI
View previous topic :: View next topic  
Author Message
aaronc542



Joined: 22 May 2007
Posts: 35
Location: NJ, USA

PostPosted: Sat Jan 31, 2009 11:56 am    Post subject: Reply with quote

larsivi wrote:
aaronc542 wrote:

It seems that decoupling:
1) introduces a cyclic import dependency - this is how Mysql trunk is - unless you get around it with an interface


Please point me to the cyclic dependencies.



Just look at DDBI trunk. MysqlDatabase imports MysqlResult which imports MysqlResult.
Back to top
View user's profile Send private message
aaronc542



Joined: 22 May 2007
Posts: 35
Location: NJ, USA

PostPosted: Sat Jan 31, 2009 2:06 pm    Post subject: Reply with quote

larsivi wrote:
In any case, going away from decoupled interfaces needs good reasons, not the opposite.


I think the burden is on us both. I think you have good points and I think I have good points. And I hope we can both come to a better understanding of how to do this. Yeah, it's not easy to get this right. I think this is my third implementation of this stuff - so I'm kind of coming from the point of not running into some of the problems I encountered the first few times I did this. Still not perfect...
Back to top
View user's profile Send private message
aaronc542



Joined: 22 May 2007
Posts: 35
Location: NJ, USA

PostPosted: Sat Jan 31, 2009 2:39 pm    Post subject: Reply with quote

larsivi wrote:
detachResult is not a good compromise because it means you need to have the decoupled version there in any case, but only invoking in on request.


Well I think the idea with detachResult is to hide database-dependent, more advanced functionality from the average user. While still allowing those who no what a complex client side join is to do it.

Look with the two class solution, you either need to be tracking all of the Result classes you're allocating from Database or tell the user to do it explicitly by writing:

Code:

auto res = db.query(......);
debug assert(res !is null);
scope(exit) res.close;


What I'm saying is that I think it's less complicated if your db can automatically close the last result when the user starts the next query. With detachResult, the user can bypass this behavior and clean up his own resources. I liken it to the GC in D - yes you still want to manage your own memory in a lot of cases - but for the casual user, you should try to get this out of the way and put the burden on the language. (Granted sometimes I wish I could just shut off the D-runtime altogether in some cases... but anyway that's not most of the time.)

Ok, so, to be honest, I was kind of annoyed to be writing "scope(exit)" all over the place then having to go back and make sure I did it. Even with this in Sqlite, it seemed necessary to keep a linked list of all of the Statements you allocated to make sure that at the end you could clean them all up. And that was with Statements - which are supposed to stick around. Having to do something similar with Results, which are supposed to be temporaries, just makes me cringe....

I'm trying to be practical and have good design. Look at it like this. Imagine you didn't know anything about what the user was supposed to expect from a DB API. (I admit that I agree with you - I automatically expect there to be a result class - the way I am proposing in some ways seems counterintuitive to me.) You don't know anything about what the user's supposed to expect, but you know a few things about OO design. And then you looked at the user's code which had scope(exit) written all over the place and that it was all fairly predictable. And in most user code, you'd probably never find a complex client side join - theyed usually just want to close that result set and get on with it. And you also realized that in the cases they didn't - it was usually incorrect. Ok... Then you looked at the implementation code and found either cyclic dependencies which didn't seem to have any solution or some other complicated way to get around this. I think it'd be natural that to ask if this was the best way to do things. Even if everyone told you "well this is just what the user expects".

Anyway, I'm trying to point out that I think both design decisions need some justification.

Ok, yes Lars, I agree that complex client side joins is a use case. We should allow for it. I also agree that I too intuitively expect something like "Result query(char[] sql)". But the question is, are these good enough reasons to justify that we should tell all our users that they have to write "scope(exit) res.close;" every time they run a query or else their code is not exception safe and could have memory leaks? I'm presenting a way around this (although for Statement we'll still need to tell everyone to write "scope(exit"). Ok, look, please tell me honestly - what will a user say if we present them with two API's - one that requires the "scope(exit)" stuff and one that doesn't? And I think even for a purist it would be hard to say definitively that the "scope(exit)" way is correct and good design whereas my proposal is incorrect and bad design.

In any event, I should say I'm glad we're discussing these issues and I'm glad we're all giving DDBI some attention because it was dead for a while. Hopefully we can come to some agreement which makes us happy...
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sun Feb 01, 2009 5:52 am    Post subject: Reply with quote

aaronc542 wrote:
larsivi wrote:
aaronc542 wrote:

It seems that decoupling:
1) introduces a cyclic import dependency - this is how Mysql trunk is - unless you get around it with an interface


Please point me to the cyclic dependencies.



Just look at DDBI trunk. MysqlDatabase imports MysqlResult which imports MysqlResult.


Ouch, I really thought I hadn't done that (MysqlResult importing MysqlDatabase). That particular issue is easily enough solved by passing the MYSQL* instead, it is the only thing used from the database instance.

I think I have a strategy for how to solve this in a better fashion though, but I would like to hear your thoughts on what all the issues are;


    * Result/Statement needs the MYSQL* to do some of the tasks.
    * If the database connection is closed, Result/Statement should know since any instances no longer will be valid.
    * If Result/Statement wasn't explicitly closed by user, then DDBI should be able to do it when they've become invalid.


Other thoughts?
Back to top
View user's profile Send private message
aaronc542



Joined: 22 May 2007
Posts: 35
Location: NJ, USA

PostPosted: Sun Feb 01, 2009 12:45 pm    Post subject: Reply with quote

larsivi wrote:

I think I have a strategy for how to solve this in a better fashion though, but I would like to hear your thoughts on what all the issues are;


    * Result/Statement needs the MYSQL* to do some of the tasks.
    * If the database connection is closed, Result/Statement should know since any instances no longer will be valid.
    * If Result/Statement wasn't explicitly closed by user, then DDBI should be able to do it when they've become invalid.


Other thoughts?


Yeah, these are a lot of the issues that have made me think a one class solution is better. Ok, in addition to this, with Sqlite you can't leave those dangling results open when you try to close - it will give you an error - you have to clean it up. Same thing with multi-result queries in Mysql - if you don't free all the results you can't run another query nor can you close your connection.

Look, I think if we had struct destructors and could overload opDot like in D 2, maybe we could solve some of this. Although with this solution there'd be no support for multiple extended interfaces based on IResult - there would just be IResult end of story. But, still it may be better. Maybe someone could convince Walter to release a D 1.1 that just had struct destructors, copy constructors, and opDot so we could do better RAII.

I realize that even with my solution you still might need to write "scope(exit)" a lot because I don't think you can run a prepared statement for instance before you've closed a multi-result query from the standard mysql C API. One solution, of course, would be to call closeResult from within prepare before returning a prepared statement so that nobody can run into that problem. Grrr...

Ok, I think we all can agree that the Mysql C API is a serious pain in the ass.

Unfortunately, that pain in the ass is probably our most important backend.
Back to top
View user's profile Send private message
aaronc542



Joined: 22 May 2007
Posts: 35
Location: NJ, USA

PostPosted: Sun Feb 01, 2009 12:46 pm    Post subject: Reply with quote

larsivi wrote:

I think I have a strategy for how to solve this in a better fashion though, but I would like to hear your thoughts on what all the issues are;


    * Result/Statement needs the MYSQL* to do some of the tasks.
    * If the database connection is closed, Result/Statement should know since any instances no longer will be valid.
    * If Result/Statement wasn't explicitly closed by user, then DDBI should be able to do it when they've become invalid.


Other thoughts?


Yeah, these are a lot of the issues that have made me think a one class solution is better. Ok, in addition to this, with Sqlite you can't leave those dangling results open when you try to close - it will give you an error - you have to clean it up. Same thing with multi-result queries in Mysql - if you don't free all the results you can't run another query nor can you close your connection.

Look, I think if we had struct destructors and could overload opDot like in D 2, maybe we could solve some of this. Although with this solution there'd be no support for multiple extended interfaces based on IResult - there would just be IResult end of story. But, still it may be better. Maybe someone could convince Walter to release a D 1.1 that just had struct destructors, copy constructors, and opDot so we could do better RAII.

I realize that even with my solution you still might need to write "scope(exit)" a lot because I don't think you can run a prepared statement for instance before you've closed a multi-result query from the standard mysql C API. One solution, of course, would be to call closeResult from within prepare before returning a prepared statement so that nobody can run into that problem. Grrr...

Ok, I think we all can agree that the Mysql C API is a serious pain in the ass.

Unfortunately, that pain in the ass is probably what most of our users will end up using.
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sun Feb 01, 2009 1:07 pm    Post subject: Reply with quote

I haven't actually tested yet, but none of the restrictions you mention related to mysql and multiresults are documented in the C API (that I could see).
Back to top
View user's profile Send private message
aaronc542



Joined: 22 May 2007
Posts: 35
Location: NJ, USA

PostPosted: Sun Feb 01, 2009 1:13 pm    Post subject: Reply with quote

http://dev.mysql.com/doc/refman/5.0/en/c-api-multiple-queries.html

See the part that says "Failure to process the result this way may result in a dropped connection to the server"
Back to top
View user's profile Send private message
larsivi
Site Admin


Joined: 27 Mar 2004
Posts: 453
Location: Trondheim, Norway

PostPosted: Sun Feb 01, 2009 1:18 pm    Post subject: Reply with quote

Yes, that is bad enough, but it doesn't say that you can't disconnect before clearing out all the results. If that isn't possible, _then_ things are much more difficult (no matter how things are done).
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> DDBI All times are GMT - 6 Hours
Goto page Previous  1, 2, 3
Page 3 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group