Rusty is back from Javapolis and had this to say about regular expressions in place of StringTokenizer:
I had one "A-Ha" moment during the conference when Ian Darwin pointed out in his session of regular expressions in Java 1.4 that a regular expression can do anything StringTokenizer and StreamTokenizer can do, and do it better. I've been telling developers for a couple of years not to use StringTokenizer and StreamTokenizer (They never do what anybody expects or wants them to do anyway. They're designed for parsing Java source code, not for reading tab and comma delimited files. ) However, I've been telling developers that if they need tokenizing functionality they should write their own class. Now I realize you don't even have to do that. Regular expressions - they're not just for search and replace anymore.
Since I've almost exclusively used Apache's Jakarta-ORO library to do my Regular Expressions in Java, I figured I better catch up with the JDK 1.4 way of doing them since I'm forcing myself to switch to the new JDK (thanks Diego, ugh.). There's probably lots of links out there, but here's the Regular Expressions Lesson from Sun's Java Tutorials. Seems to have most of the info I was looking for.
Like the Log stuff, it seems that the OSS library is much more powerful, but if it's included in the API, it'll be good to know that way of doing it and utilize it when possible.