Apps
Contact
Seminars
RSS FeedSeptember 20th, 2010
Hadoop
Just received from Amazon – “Hadoop: The Definitive Guide“. Now on to the difficult part of actually reading it. 
September 13th, 2010
Mindmap Notes from Hadoop Meetup
Hadoop meetup of September 7th was awesome. Here is an image export of my (incomplete) mindmap notes. If you need the original mind map file, please drop me an email.
September 11th, 2010
Hadoop Meetup (Washington DC, Sep 7th)
Excellent gathering at the Hadoop Meetup this past Tuesday. Two good speakers: Tom White and Aaron Cordova. Tom presented some interesting possible additions to Hadoop (8 things missing he would like to see in Hadoop). Aaron presented ideas on how to use HBase for distributed single NameNode. I am new to Hadoop myself, but got a lot out of this meeting.
Tom is the author of “Hadoop: The Definitive Guide“. I have ordered the book, have not received it yet, but as I hear, it is one of a couple of good books on Hadoop.
My mind map notes from the meeting (obviously incomplete and inaccurate) are here. If there is another Hadoop meetup in DC, I am very likely to attend.
July 31st, 2009
Venn Diagram Queries
Venn Diagram queries in Oracle have saved the day for me many many times. Consider this:
You have a table “analyzethis”, and say mv (modelvalue) is in 0/1 and tv (TruthValue) is in 0/1. Then, the following query does something nice.
select sum (val), match from
(select count(*) val, ‘M’ as match from analyzethis where mv=1 and tv=1 union
select count(*), ‘M’ from analyzethis where mv=0 and tv=0 union
select count(*), ‘N’ from analyzethis where mv=1 and tv=0 union
select count(*), ‘N’ from analyzethis where mv=0 and tv=1)
group by match
This query returns:
Sum(val), Match
1450, M
2300,N
Of course a similar result can also be obtained by decode, sign and square functions.
July 15th, 2009
Can I have your telephone string, err, I mean number?
Some things go without saying. Such as, a telephone number is a number. Like 703 123 4567. But if I go back to Dave Roberts, who taught me CSCI 242 many years ago, he would say, can you give me your telephone number, and add it to Peter’s and subtract Anna’s? Well, that really doesn’t make much sense.
In database modeling terms, telephone number is a String, a varchar2, a text. Not a number. This pretty much goes without saying as well, for anyone who has been data modeling for more than a few days. Here, are a couple of reasons telephone number is considered a String, not a number:
- It can have symbols. People enter their telephone numbers themselves, using hyphens, parenthesis, plus sign, alphabetical (1-800-CALL ATT), etc. The database mostly needs to save the data that the user entered, and show it later. It would be unacceptable to modify the user’s data and show it in a different format. Would be cute and dumb at the same time.
- You don’t do any Math on the telephone number. You don’t take average, or sum of all telephone numbers, like you do with invoice_total and avg _time_in_queue.
Now, there really are systems that store a telephone number as a number, not a varchar2. That is primarily done as a short cut for validation. These systems have those annoying (but perhaps highly functional) websites where you get those messages (“Please use numbers from 1-10 only.”) when trying to enter a white space or parenthesis in a telephone number. The good aspect of that is, that they are very very good at catching the 10 digit US/Canada telephone numbers. Works absolutely great for utility companies and gas electric cooperatives with US as their only market, but if international market space is your thing, then not so much.
I take this as my gentle remember, that all rules are meant to be learnt, and then we can break some. The good news is that you get to pick which ones you break! The bad news of course is that you need to learn them first, and then break them.