SQLite 3 and Java

The other day, I went off about how great SQLite was and everything, as well as how to use it with Ruby. Today I'll tell you how to use SQLite with Java. Just download SQLite JDBC drivers from here. Figure out where sqlite.jar and sqlite_jni.dll (native file) are, allow your java program to find them, and then access your sqlite database using JDBC. If you aren't familiar with jni and you're using windows, you can throw that DLL file into C:/WINDOWS. Support is cake for anyone who has used Java before. The documentation for the SQLite JDBC project can be found over here. As I said before, SQLite is a good database, especially for personal projects (where running your own database server is overkill) and quick testing. Also, the SQLite SQL implementation supports more than the in-memory database HSQLdb does, which I feel is the other good alernative. I'm liking SQLite more than HSQLdb. But I'm probably especially whiny about HSQLdb right now, since it didn't support the SQL case-when statement that I wanted to use for my last project.

As for examples... well, it's JDBC: the beauty of it is I don't need to give examples. But since I have examples, here's some very brief code to create a connection. If you don't know JDBC, read the JDBC documentation from Sun, it's probably worth knowing regardless of what you program for. With that, following this brief example (for which requires importing java.sql.Connection and having sqlite.jar and sqlite_jni.dll on your paths), I'm all done.

// Load the Database Engine JDBC driver
Class.forName( "SQLite.JDBCDriver");

// connect to the database
Connection conn = DriverManager.getConnection( "jdbc:sqlite:/filename", "", "" );

//close the connection. So pointless.
conn.close();

SQLite 3 and Ruby

As per my wife's request, I built a financial management spreadsheet application, because "we" needed to keep better track of our finances.. I'll probably go into that program some other day as it's actually proven useful. But enough about that, today I want to talk about the database I used for it. This database was SQLite 3. If you haven't heard of it, SQLite is an ACID-compliant relational database management system contained in a relatively small C library where the SQLite library is linked in and thus becomes an integral part of the program with the database accessed through simple function calls. Yes, that long run-on sentence was courtesy of wikipedia, copy, and paste. In my words: SQLite is a database that exists as an easy to access text file (via c function calls) instead of requiring an external database server to be running somewhere waiting for requests. A free and tiny embedded database with great SQL support.

My program was written in ruby. For database access, I used the sqlite3-ruby gem, which wraps up those c function calls with ruby. If you're unfamiliar with gems, they're simply packaged ruby applications that allow easy installs. Simply go to your command prompt and type in "gem install sqlite3." This will work as long as you've installed ruby with gems and have an internet connection.

Once you've installed the gem, you're all set. To show how easy it is to use, I'll go through a short example. This example creates a database with a table describing teams and a table describing how a certain team scored on a certain game. It then queries the database in order to print out team rankings. Yeah, it's an odd way to get that information, but it was part of a homework assignment I had. I'm unsure whether the query is very good, but it's going to serve as my example.

First, create your connection and (possibly) insert information thusly.

require 'sqlite3'

if( !File.exists?( 'test.db' ) )
  db = SQLite3::Database.new( 'test.db' ) #The database is automatically created
  db.execute( "create table team ( id INTEGER PRIMARY KEY, name VARCHAR(255) )" )
  db.execute( "create table team_gm ( id INTEGER PRIMARY KEY, game_id INTEGER, team_id INTEGER, score INTEGER )" )
  db.execute( "insert into team_gm values( 1, 1, 1, 5 )" )
  db.execute( "insert into team_gm values( 2, 1, 2, 5 )" )
  db.execute( "insert into team_gm values( 3, 2, 1, 3 )" )
  db.execute( "insert into team_gm values( 4, 2, 2, 6 )" )
else
  db = SQLite3::Database.new( 'test.db' )
end

Then query and manipulate data thusly. You've got lots of freedom here. I suggest to at least read the FAQ if you don't want to read all the documentation.

query = "select z.name, sum(case when x.score > y.score then 1 else 0 end) as wins, " +
"sum(case when x.score < y.score then 1 else 0 end) as losses, sum(case when x.score = y.score then 1 else 0 end) as ties from team_gm x " +
"join team z on x.team_id = z.id " +
"join team_gm y on x.game_id = y.game_id AND x.team_id <> y.team_id group by x.team_id " +
"order by wins-losses-ties desc"

db.execute( query ) do |row|
  print row[0],": ",row[2]," Wins, ",row[3]," Losses, ",row[4]," Ties\n"
end

And finally close your connection thusly.

db.close

This database was exactly what I wanted and the sqlite ruby access is very simple. Granted, I'm probably way behind for not having used this before (I've been using HSQLDB for my "easy" databases), but at least I know now. I bet I'll be using it a lot now.

Regular Expressions and Dates

Regular expressions are great and I think programmers should always keep in practice with them because you really never know when it'll be useful to parse through text. When I was first learning Regular Expressions, I practised with dates. Eventually, I made myself a short program in ruby. It takes a string as input and if the string suggests a date, the program is supposed to figure out which date. Lots of stuff can be done with that date, such as a report of the day of the week, how far in days the date is from today, or when the end of the month is. All the calculations are trivial if a date is given. The tricky part was determining if a string specified a date, and if so, extracting the date.

As I said, I used regular expressions for figuring out what the date meant. I thought of all the formats a user might use and made a case for each of them. This has actually been a useful exercise and I've used the results of it a lot. The following are all the cases that I could think of... I very much doubt that I got all of them. Remember, this is for ruby, so the reg-exp formatting may be different for you. The i means ignore case and \A and \Z can be used to specify the beginning and end of the string respectively. The only way I can think of improving the recognition for now is to allow for spelling errors. Can you think of any other ways a person might reasonably suggest a date as a string? Here goes.

1. Of the format MM/DD/YY, MM-DD-YY, or MM\DD\YY. YY may be YYYY in any of the cases. This is the trickiest pattern, since the user may mean either Year/Month/Day or Month/Day/Year. Two digits for the year makes it difficult to assume what the user meant. I chose to use the Month/Day/Year form, unless the user uses four digits to specify the year, in which point it is easy to figure out what they meant. Therefore, YYYY/MM/DD is also a valid format . As for the year, if four digits are not specified, then I assume that they are specifying the current millineum (2000).
/\A(\d+)\s*(-|\/|\\)\s*(\d+)\s*((-|\/|\\)\s*(\d+))?/

2. Of the format "June 27, 1983", "Jun 27, 1983", or "June 27" (in which the current year is implied). Granted Junileropwf 27, 1983 would also be valid here, but I didn't think such cases were important enough to detect and were instead a waste the readibility of the expression. In specifying the month, a minimum of 3 characters are required. The comma is optional, but at least one space must separate the tokens. If a day and year are not specified, then day one of the specified month in the current year should be used.
/(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)([^\s]* (\d+)(.*))?/i

3. Of the format YYYYMMDD. It has to be in that order, I didn't make allowances in this case.
/(\d\d\d\d)(\d\d)(\d\d)/

4. Of the format YYYY. The date is interpreted as Jan 1 of the year YYYY.
/(\d\d\d\d)/

5. Last or Next or Current Day of the Week (ie last Thu, next Thursday, or Thursday). I used a minimum of three letters for the weekday name to avoid matching cases I didn't want it to match. Once again, I didn't test that the letters following the first three were correct because I didn't think it improved the match any more.
/(last |next |this |)(sun|mon|tue|wed|thu|fri|sat).*/i

6. Yesterday, today, tomorrow. Also, my wife and I made up words Yupterday (the day before yesterday) and Threemorrow (the day after tomorrow.) [--We wanted to make up for the lack of an English word for these common references. Yes, we're pretty goofy people. Please feel free to adopt the words.] The test for these is really simple and reg-exp isn't even really needed. It's actually less effecient if you want single matches, O(nm) instead of O(n). Here's the case for the heck of it:
/(today|tomorrow|yesterday|yupterday|threemorrow)/i

If you think the program I described might be useful, here's the whole current version as html. Do whatever you want with it. I've found it useful and added a bunch of date converting methods for flavor (like get the end of the month, return day statistics, or get number of days between two dates) , but the heart of the whole thing is still the regular expressions.

Some Funny Links

Today, I (probably stupidly) gave away the link to my blog to more people. To make up for the low quality of my previous posts, I offer some of the funnier links I've come across. Of course, they are probably the kind of humor that only I'd enjoy. But at least I tried. Yeah, these are all pretty old.

General
An old He-Man storybook with commentary
English Papers
How stupid Superfriends is looking back. Specifically, the Riddler.
Retitled Romance Novels

++ Images
Did I get the job?
Find X
My favorite web comic strip.
Bunny

# Some Programming Humor
1. Stupid Hacker
2. Evolution of a Software Engineer
3. Clock in Perl

Quotation of the day: "I like to throw sandwiches at the homeless. It satisfies both my altruistic and sadistic urges."

Good Free Reads

Stories are an important part of life. Not only are they entertaining, but they can effect your life if you let them. A good story will not only spark your imagination, but will give you new pespectives about the world. Of course, there are a lot of times where stories just take too long to read. I had a friend who brought books to work to read which he never ended up liking. It took months to finish each book and along the way my friend would lose interest. Good short stories would be way better for short bursts of reading, so I suggested some to him and he seemed to enjoy them a lot more. It worked well. And so I'll share with the internet in general.

The following is a list of online stories (that probably don't justify this really long introduction.) Although a bit on the computer geeky side, these are all stories that I think are especially good reads (which is why I wanted to share them.) They are all free, excellent quick reads that focus on ideas and characters rather than visualization and are located on fairly unsuspicious urls. In other words, they're ideal for fun short break or lunch time reading. If you know of anything else you think I should add to this list, please please please e-mail me. ^_^


Stories

(1) Orson Scott Card
Without doubt, Orson Scott Card is my favorite author. At his website, many short stories are provided. Here are my favorites.
   (a) Atlantis

It's very long and takes a bit before it's interesting, but it's really good once it does. If you've read Pastwatch, you might think you already read this... but you haven't, keep reading.

   (b) Ender's Game

If you like the book and haven't read this, I think it's still worthwhile. If you haven't heard of Ender's Game... after reading this, I'm sure you'll want to pick up the full novel.

   (c) Homeless in Hell

It's a Christmas story about an interesting interpretation of Hell and Santa Clause.

   (d) Missed

It's kinda on the sad side about a guy coping with death, please keep that in mind. Quite short though. One short break's worth.

   (e) Prior Restraint

This is a pretty interesting story, but I can't say anything about it without kind of ruining it.

(2) Neal Stephenson
   (a) Jipi and the Paranoid Chip

Lots of very cool ideas, flow of thoughts, and character interaction. As with Neal Stephenson's stuff, it has an abrupt ending.

   (b) The Great Simolean Caper

Interesting, it touches upon some of his normal themes. Smart characters, cool capers. Ending isn't as abrupt as normal, I think.

(3) Wizardry
Rick Cook wrote a series that blends fantasy with programming. Sound strange? It really isn't that strange and it's actually really quite cool. If you don't like fantasy or programming... then... maybe you should keep away. The first two novels (of five) were made free. They're the best ones anyways, in my opinion. These novels are pretty short, but it should still take a few hours to get through them. Maybe a week of lunches alone.

(4) Suzumiya Haruhi no Yutsu
This is a translation of a series of Japanese light novels. Some chapters are long, but it's not difficult at all to find constant good stopping points. This story is about a normal high school guy trying to deal with a very eccentric classmate who seems to unknowingly control the world and a bunch of supernatural people drawn to her. It's a comedy, mostly. Wiki for more information, but if you like Japanese comics/animation then this is an excellent find. There's also an anime of this series.

(5) Densha Otoko original posts
Umm... this is the black sheep of the list. It's not really a story, although it does tell a story. It's an organized, edited version of a forum thread that chronicles the relationship between an otaku (japanese super geek) at the thread asking for help and a non-otaku woman he had helped. It's pretty fun to read though... a very geeky romance story. Also, a Japanese drama came out of this (also called Densha Otoko), and it's pretty funny.

(6) The Book and the Sword
A great Chinese story to complement the Japanese stories. While the book is very long, the story arcs in the book are actually very episodic and pretty short. The Book and the Sword is a kung fu novel by Jin Yong and it's a really fun read.

(7) The Last Question
I can't really say I like Asimov's writing. I mean, I like his ideas and I do understand why he's important to sci-fi, but I don't like his writing style and so I rarely have fun reading his stuff. The ideas carry this story pretty well though. Since this was a pretty well thought out story, very famous, and a very quick read I decided to list it here. Passes the time on a very slow day, at the very least.


Essays

The following is a list of my favorite online essays. Any programmer and many non-programmers should find many excellent reads in this list. What's here deals primarily with introductions to ideas, examples of thought flows, productivity boosters, and springboards to interesting topics. While some are technical, they don't delve very deeply into the actual technology, instead giving you enough information to decide if you want to devote the time to really learn the topic. With all that said, here's the list. All good reads. If you know of anything else you think I should add to this list, please please please e-mail me. ^_^

(1) Orson Scott Card
Yep, my favorite author again. In addition to novels, he also does a lot of reviews and world commentaries on his web site. I find them interesting, even if I don't agree with everything that he says. These essays were carefully chosen by me as the ones that were the most interesting and thought-provoking to me. Read more of them if you'd like, as most of them are quite interesting.
   (a) Why Making Choices is So Hard

This is the first of OSC's essays that resonated with me. This talks about making choices.

   (b) Freakonomics

A review of Freakonomics, which is an interesting book. It also takes an example from the book and expands upon it, reaching another theory. I like seeing people's thoughts flow and this essay has a lot of very nice examples of thinking about cause and effect, as well as morality.

   (c) Weapons of Mass Destruction

This one provides interesting thoughts about what makes a weapon a WMD as well as some commentary about how OSC thinks the balance in the world works.

   (d) Taking Animals Seriously

This one is very interesting. It contains a lot of ideas about how people think, as well as how different people see the world. It does this by focusing on an autistic author who has devoted her life to making animal slaughterhouses more humane.

(2) Steve Yegge's Essay Collection
Link to his current blog
Steve Yegge writes a lot. His writings made me begin to really improve my programming and they constantly cause me to evaluate the way I think about things. That's why I believe these are very important. I think every programmer should read these because after reading, whether or not you agree, you will have thought a lot. Especially the fresher programmers, like me. Plus, they're pretty entertaining and he says funny stuff. I chose to list my favorite essays. Narrowing them down is difficult since I found it very worthwhile reading them all.
   (a) Tour De Babel
   (b) You Should Write Blogs
   (c) Five Essential Phone Screen Questions
   (d) Practicing Programming
   (e) Saving Time
   (f) Practical Magic

(3) In the Beginning, there was the Command Line by Neal Stephenson
In the End, I find this horribly long, boring in places, and a little out of date. But since I didn't grow up during the era, I ended up learning a huge amount from this. Some of the ideas are interesting, especially if you're pretty new computers. If you really enjoyed this, Mother Earth Mother Board is freely available too. This is another monstrously long essay... it might as well be a novel. It's about Neal Stephenson following FLAG, a global undersea fiber optic cable. He comments a lot about the history and politics around FLAG, as well as about his surroundings. But some spots are way too slow for me.

(4) Joel on Software: Best Software Writing
Essay Links for Best Software Writing
Archive Links
Joel on Software is one of those famous blogs with lots of entries worth reading. I can't think of anything in particular, except maybe this post. He released a compilation of what he felt were the best programming related essays of the year 2005. They really are some of the best software essays. Plus he talked about what makes writing good. The second link I've provided gives links to all the essays he chose. The last link is to his archives for his major blog entries. You may as well just start with the first essay there and work your way down.

(5) Random Tech Essays
   (a) 10 Question, 8 Programmers

This was a very interesting interview. 10 questions which 8 well-known programmers answer. Very, very interesting and a very cool idea. Everyone aspiring to be a good programmer should give this a read because it's good to know how those among "the best" view the field.

   (b) Lisp for Imperative Programmers

This is an introduction to functional programming where the reader is assumed to know only about procedural programming. Anyways, if you're a purely procedural programmer (C, Java, Ruby, etc..) and wanted to know what functional programming was all about, I can't think of any better read than this.

   (c) The Best 46 Free Utilities

This is a list of what the author considers the best windows free utilities for various tasks. I like his suggestions and this is the first place I will go if I want to find a new tool. Hey, it's kind of like an essay.

   (d) Programming Rock Solid Code

This is an interesting piece about the work environment of a place that must write the code to launch and run a multi-billion dollar shuttle.

   (e) No Silver Bullet

A classic CS essay about programming tools and languages.

Jin Yong's Wuxia

Quick Summary of Links:
Romance of the Book and Sword (Buy the translation)
Sword of the Yueh Maiden
Return of the Condor Heroes (Divine Eagle, Gallant Hero)
Wuxia English Forum
Wuxia Fan Translations

Actual Entry:
I've been reading Jin Yong's (aka Louis Cha) Wuxia novels recently. Jin Yong is known as one of the best Wuxia writers. He created 15 Wuxia stories that are famous throughout a lot of the world. I've read three of the stories (soon to be four), but that's only because it's all the translated-into-english text I can find. If I could read Chinese, I could just read it all from here (but it's all simplified.)

Maybe I'm getting ahead of myself. Wuxia is a genre that comes out of China and literally means "martial arts heroes." They take place in China's past, usually among real events, but everyone's abilities are hyper exaggerated. It's a blend of philosophy, conflicting honor codes, and fantasy martial arts. The best I can compare it to in western culture would be modern takes on Arthurian or Robin Hood stories. I guess Westerns are kind of similar also. If you've seen "Crouching Tiger, Hidden Dragon", that's an example of a wuxia story. Except Jin Yong's stories are much better than these examples, and a lot more stuff happens involving much cooler characters.

If you like shonen japanese stories, there is no doubt you'll love the stories Jin Yong has written. In fact, you'll very likely like the novels more than those comics. I'm pretty sure almost all the really good anime fighting ideas (with the chi or martial arts moves) were stolen from China. The fighting used is really interesting and very well imagined. The novels have lots of romance elements, political plots, and technique-laden action. The characters are all really cool and interesting.

So far, I've read the Book and the Sword, Return of Condor Heroes, and Sword of the Yueh Maiden. I've ordered Deer and the Cauldron... but it's pretty pricey at $35 for each of the three books that make the story up. Each book I've read has been very good. Read Sword of the Yueh Maiden if you're doubtful. It's the only short story of Jin Yong's novels and it's a great taste of what the genre is like. For a full listing of his stories, please consult his wikipedia entry, from which you can also get summaries. Beware of spoiling yourself though. I'm going to go over the works that are completely available in English. There are a lot of translations of his, but most of them are in progress works.

Sword of the Yueh Maiden is a short story about two nations at war with each other. You can read it in English here. The weaker nation isn't weaker by much: its only problem is that the swordsmen of the other nation are far stronger. The story is about one of the court advisors looking for a way to overcome this weakness.

The Book and the Sword can be read here and bought here. It's about the actions of a very heroic secret society. It starts with several rescue plots until the "big secret" is uncovered. It has a large cast of characters, such as Chen who fights using go stones and a whip to hit accupoints. There's also the lightning fast one-armed brother and the Mastermind strategist brother. These characters are naturally all ranked in their organization. There are great and powerful female characters as well. It's all great fun.

Return of Condor Heroes follows the growth of a boy named Yang Guo and his relationship with his teacher Xiaolongnu. Yang Guo is the brilliant son of a defeated villian and no one wants him to grow up like his father. However, all his masters treat him poorly so he runs away to a cave where he becomes the student of Xiaolongnu. Lots and lots of stuff happen from there and Yang Guo grows stronger and stronger even despite bad things happening all the time. It's been fan translated and that translation can be found here.

The Deer and the Cauldron was translated into English and released as three installments. Book one can be purchased here, book two can be purchased here, and book three can be purchased here. I haven't read it yet, but it's supposed to be funny. I will update this when I get to read it.

The Fox Volant of the Snowy Mountain was also translated into english and can be purchased here. However, I've heard that the translation was really bad.

Unfortunately for me, these books have been adapted into TV series, which I've been spending a lot of energy recently obtaining and then a lot of time watching. These series all last about 40 episodes and it's still not really enough time to show all the events from the novels. Furthermore, these things are pretty much corny and horrible as far as I've seen. They try to rely on cg effects that would be embarrassing in a old playstation game and really has no business in a live action film. I also don't like all the unnatural sounding voice dubs. Despite all my problems with them, I keep watching and am slowly beginning to enjoy it. The stories are good and every once in a while, things are done right (read: not ridiculously poorly.) The only one I've seen like that so far is Tian Long Ba Bu 2002. Return of Condor Heroes 2006 was ridiculous, but the actors looked too cool to complain too much. Okay, mostly Liu Yifei.

Tian Long Ba Bu follows three fairly different heroes whose paths constantly intertwine. There are a lot of characters here and a lot of twists. It's a fun series to watch and no segment really drags for very long. Some story arcs are especially gripping. The best thing is that there are so many characters that are very likable and all the different storylines are equally interesting. The thing that makes this a good tv series is the (relatively) low use of CG, the good casting, and the actually fun to watch fight scenes. I hope I can find another adaptation like this.

I want to mention that, so far, Jin Yong's stories are the only wuxia I've read that I'd really recommend at this point. I've gone through The Sword and the Exquisiteness and Sentimental Swordsman, Ruthless Sword by another very famous wuxia author, Gu Long. While I did read through them both, I didn't like them very much. I couldn't relate to his characters and I felt that they were very one dimensional - a few very good, many very bad, or some inconsequential. I do not feel that he built a very compelling or deep world for his stories to take place. I read his wiki entry, and I think that the differences I saw between Gu Long and Jin Yong are reflected in the ways that they led their lives.

And that's all I've got to say. I just wanted you to give this genre a try if you haven't already, y'know, bring it to your attention. I had no idea about the quality of this stuff until I happened by it after reading the wiki entry on the movie Kung Fu hustle. I mean, you want to open your horizons and see what popular Chinese fiction is like, right? I tried to explain it, but there was no way I could do the stories justice. So please just follow any of the links I gave earlier and give it a try. If you'd like more information, read Jin Yong's wikipedia entry, the fan translation base, and this forum. I'll update this entry as I get more information and as more translations are completed.

Oh, and one last thing slightly related thing. I recently found a freely available translation of the Romance of the Three Kingdoms. If that interests you, take a look over here.