FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

SQLite3 4invalid UTF-8 sequence

 
Post new topic   Reply to topic     Forum Index -> DDBI
View previous topic :: View next topic  
Author Message
jicman



Joined: 22 Dec 2004
Posts: 298
Location: Rochester, NY

PostPosted: Mon Jan 07, 2008 10:22 pm    Post subject: SQLite3 4invalid UTF-8 sequence Reply with quote

Greetings.

I have been working with sqlite3 for a while, now. But, in the last few days, I have been getting lots of "4invalid UTF-8 sequence" errors. Any idea what causes these errors?

I have done a few google searches and these errors appear to be D related, since there are no other ones for SQLite3.

Any ideas?

thanks,

jic
Back to top
View user's profile Send private message
jicman



Joined: 22 Dec 2004
Posts: 298
Location: Rochester, NY

PostPosted: Tue Jan 08, 2008 6:35 pm    Post subject: Reply with quote

Ok, a little more history...I have developed an application that uses a local and a shared folder SQLite3 DBs and the D SQLite3 wrapper is failing on execute commands. For example, this UPDATE command,

UPDATE LSOpenJobs SET

ProjID = '871',
subProjID = '865',
parent = '209',
children = '',
login = 'id999999',
cust = 'CoCo',
proj = 'Tape Binder OpMan',
PClass = 'Quote',
PSubClass = 'PM',
bdate = '2007-12-05',
ddate = '2007-12-05',
edate = '',
pm = 'jose',
pmuk = 'First, Name',
lang = 'de-DE',
vendor = '',
vEmail = '',
invoice = '0.0',
ProjFund = '177',
PMTime = '1.50',
A_No = 'B9OPMSL0',
wDir = 'D:\Projects\CoCo\000871-CoCo-Tape-Binder-OpMan\Quote',
BiliDir = '',
TMDir = 'T:\Projects\_TMs\CoCo-TMs\actual',
DeliveryDir = '',
paid = 'n',
paidDate = '',
notes = 'Operator Manual',
status = 'o',
pages = '',
ta = '',
fromLang = 'en-US',
techPM = '',
termPM = '',
Xtra0 = '',
Xtra1 = '',
Xtra2 = 'y' WHERE id = 1601;

threw an exception,

DBIException: near "bs": syntax error

as you can see, there is no syntax error above. What I would like to do is
to turn on some debugging utility inside the DBs to see what it is receiving
and why the DB is refusing the execute call, if it is, at all.

Any help or ideas are appreciated.

thanks,

jic
Back to top
View user's profile Send private message
stonecobra



Joined: 25 May 2004
Posts: 48
Location: Rough and Ready, CA

PostPosted: Fri Jan 11, 2008 3:41 pm    Post subject: Reply with quote

Does anyone know how to get more/better debug information from the sqlite API? Unfortunately I am a noob when it comes the the sqlite C API.
Back to top
View user's profile Send private message
Alan Knowles



Joined: 23 May 2006
Posts: 11

PostPosted: Fri Feb 29, 2008 5:41 pm    Post subject: Reply with quote

I've change execute/query to include logging/debugging code.
basically set
db.debugmode = 1
and/or
db.logfile = "/tmp/myfile.log";

Unfortunaly I dont think it helps your SQL error - I'm getting this as well, so I'm going to be looking closer at it.


override void execute (char[] sql) {
char** errorMessage;
if (this.debugmode) std.stdio.writefln("EXECUTE: %s",sql);
if (this.logfile.length) std.file.append(this.logfile , "EXECUTE: " ~ sql ~ "\n");
if ((errorCode = sqlite3_exec(database, sql.dup.ptr, null, null, errorMessage)) != SQLITE_OK) {
if (logfile.length) std.file.append(logfile , "ERROR: " ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
throw new DBIException(
toDString(sqlite3_errmsg(database)) ~ "\nSQL: " ~ sql
, sql, errorCode, specificToGeneral(errorCode));
}
}
Back to top
View user's profile Send private message Send e-mail
jicman



Joined: 22 Dec 2004
Posts: 298
Location: Rochester, NY

PostPosted: Fri Feb 29, 2008 10:42 pm    Post subject: Reply with quote

If you find anything, please let me know. I have left it alone because of lack of time. It happens 3 and 4 times a day, now. And depending on high usage, a bit more. There is nothing wrong with the data in the db, it is just a annoying bogus error message.

thanks,

jose
Back to top
View user's profile Send private message
Alan Knowles



Joined: 23 May 2006
Posts: 11

PostPosted: Sun Mar 02, 2008 6:53 am    Post subject: Reply with quote

These are changes for native bind (it's not particulary clean or designed... but works - it would be interesting to see if this works better than the emulated bind code re: Utf8 issues..

you need to fix a few interface issues etc... -but this is the core code.


Code:

// from SQLiteDatabase.d


override SqliteResult query (char[] sql) {
      char** errorMessage;
      
      if (debugmode) std.stdio.writefln("QUERY: %s",sql);
      if (logfile.length)   std.file.append(logfile , "QUERY: " ~ sql ~ "\n");
      sqlite3_stmt* stmt;
      if ((errorCode = sqlite3_prepare(database, toCString(sql), sql.length, &stmt, errorMessage)) != SQLITE_OK) {
         if (errorfile.length)   std.file.append(errorfile , "QUERY: " ~ sql ~ "\n");
         if (errorfile.length)   std.file.append(errorfile , "ERROR(" ~ std.string.toString(errorCode) ~ "):" ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
         if (logfile.length)   std.file.append(logfile , "ERROR(" ~ std.string.toString(errorCode) ~ "):" ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
         throw new DBIException(toDString(sqlite3_errmsg(database)), sql, errorCode, specificToGeneral(errorCode));
      }
      return new SqliteResult(stmt);
   }
   
   
   override void* nativeStatement(char[] sql) {
      char** errorMessage;
      sqlite3_stmt* stmt;
      if (debugmode) std.stdio.writefln("NEW STATEMENT: %s",sql);      
      if (logfile.length)   std.file.append(logfile , "NEW STATEMENT: " ~ sql ~ "\n");
      
      if ((errorCode = sqlite3_prepare(database, toCString(sql), sql.length, &stmt, errorMessage)) != SQLITE_OK) {
         if (errorfile.length)   std.file.append(errorfile , "QUERY: " ~ sql ~ "\n");
         if (errorfile.length)   std.file.append(errorfile , "ERROR(" ~ std.string.toString(errorCode) ~ "):" ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
         if (logfile.length)   std.file.append(logfile , "ERROR(" ~ std.string.toString(errorCode) ~ "):" ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
         throw new DBIException(toDString(sqlite3_errmsg(database)), sql, errorCode, specificToGeneral(errorCode));
      }
      return cast(void*)stmt;
   }
   
   
   
   
   override void bind(void * stmtv, int n, char[] str)
   {
      sqlite3_stmt* stmt = cast(sqlite3_stmt*) stmtv;      
      if (debugmode) std.stdio.writefln("BIND: @%d:%s",n,str);      
      if (logfile.length)   std.file.append(logfile , "BIND: @"~std.string.toString(n) ~ ":" ~ str ~ "\n");
      

      if ((errorCode = sqlite3_bind_text(
         stmt,
         n+1,  // Index of wildcard
         std.string.toStringz(str),
         str.length,  // length of text
         SQLITE_STATIC
      )) != SQLITE_OK) {
         
         if (errorfile.length)   std.file.append(errorfile , "BIND: @"~std.string.toString(n) ~ ":" ~ str ~ "\n");
         if (errorfile.length)   std.file.append(errorfile , "ERROR(" ~ std.string.toString(errorCode) ~ "):" ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
         
         
         throw new DBIException(toDString(sqlite3_errmsg(database)), "", errorCode, specificToGeneral(errorCode));
      }
   
   
   }
   override void bind(void * stmtv, int n, int i)
   {
      
      sqlite3_stmt* stmt = cast(sqlite3_stmt*) stmtv;
      if (debugmode) std.stdio.writefln("BIND: @%d:%d",n,i);      
      if (logfile.length)   std.file.append(logfile , "BIND: @"~std.string.toString(n) ~ ":" ~ std.string.toString(i)  ~ "\n");
      if ((errorCode = sqlite3_bind_int(
         stmt,
         n+1,  // Index of wildcard
         i
      )) != SQLITE_OK) {
         if (errorfile.length)   std.file.append(errorfile , "BIND: @"~std.string.toString(n) ~ ":" ~ std.string.toString(i)  ~ "\n");
         if (errorfile.length)   std.file.append(errorfile , "ERROR(" ~ std.string.toString(errorCode) ~ "):" ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
         
         throw new DBIException(toDString(sqlite3_errmsg(database)), "", errorCode, specificToGeneral(errorCode));
      }
   
   }
   
   override  void executeStatement(void * stmtv)
   {
      sqlite3_stmt* stmt = cast(sqlite3_stmt*) stmtv;
      if ((errorCode = sqlite3_step(stmt)) == SQLITE_OK) {
         if (errorfile.length)   std.file.append(errorfile , "EXECUTE STATEMENT:\n");
         if (errorfile.length)   std.file.append(errorfile , "ERROR(" ~ std.string.toString(errorCode) ~ "):" ~ std.string.toString(sqlite3_errmsg(database)) ~ "\n");
         
         sqlite3_finalize(stmt);
         stmt = null;
         throw new DBIException(toDString(sqlite3_errmsg(database)), "", errorCode, specificToGeneral(errorCode));
      }
      sqlite3_finalize(stmt);
      stmt = null;
   }
    override SqliteResult queryStatement(void * stmtv)
   {
      return new SqliteResult(cast(sqlite3_stmt*) stmtv);   
   }
      
//------------------------------------------------
Changes to Statement..

   void * stmt;
   /**
    * Make a new instance of Statement.
    *
    * Params:
    *   database = The database connection to use.
    *   sql = The SQL code to prepare.
    */
   this (Database database, char[] sql) {
      this.database = database;
      this.sql = sql;
      this.stmt = database.nativeStatement(sql);
   }



void bind (size_t index, char[] value) {
      if (this.stmt) {
         database.bind(this.stmt, index, value);
         return;
      }
      binds ~= escape(value);
   }
   
   
   void bind (size_t index, long value) {
      if (this.stmt) {
         database.bind(this.stmt, index, value);
         return;
      }
      binds ~= std.string.toString(value);
   }

   void execute () {
      if (this.stmt) {
         database.executeStatement(this.stmt);
         return;
      }
      
      
      database.execute(getSql());
   }

   /**
    * Query the database.
    *
    * Returns:
    *   A Result object with the queried information.
    */
   Result query () {
      if (this.stmt) {
         return database.queryStatement(this.stmt);
       }
      
      return database.query(getSql());
   }


Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic   Reply to topic     Forum Index -> DDBI All times are GMT - 6 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group