The signal to noise ratio

Posted by chetan on February 01, 2008

The traffic here is out of control, and, with the introduction of the 1 lakh car (which I think is fantastic, btw) it’s bound to get a lot worse, at least in the near term. For a while I thought a big part of the problem was the lack of traffic signals at most intersections and a complete lack of order on the road. The heterogeneous traffic mixed with the all out war mentality creates an extremely hostile environment.

It wasn’t until they put a signal at the [fairly minor] intersection near the office that I realized just how well “organized” this particular brand of chaos can work. Right away, about five minutes in the morning and ten minutes in the evening were added to my commute, just sitting in traffic. The very next day, the light changed to a flashing yellow (go slow!) with a traffic cop directing traffic. In the evening it seemed to be working normally but traffic was much more fluid. Ten minutes sitting still became roughly 5 minutes. Either people took a different route tonight or they just got used to the fact that, sometimes, you need to stop at an intersection.

SimpleDB: MapReduce for the masses?

Posted by chetan on December 16, 2007

On Thursday, Amazon announced SimpleDB, “a web service for running queries on structured data in real time.” As many others have noted this more or less completes the cloud computing stack that Amazon has been steadily building, ever since they launched the Simple Storage Service (S3), early last year.

Where their earlier releases (S3, Elastic Compute Cloud [EC2], Flexible Payments, Mechanical Turk) commoditized much of the infrastructure required for building scalable applications, SimpleDB (SDB) and the earlier Simple Queue Service (SQS) are bringing cutting edge technologies and design patterns to the masses. First they made it cheap and easy to have a cluster; now they’ve made it cheap and easy to use a cluster! Amazing.

What’s even more startling is just how much Amazon gets it, and just how far off base Salesforce was earlier this year when they announced, as a “platform as a service”.

Then again, maybe they’re not even competing services at all.

Amazon is clearly providing services targeted towards developers and entrepreneurs with the goal of enabling them to explore new and innovative ideas by lowering the cost of entry. They’re providing the basic building blocks for developers to do exactly what they have done (and spent the last 10 years building) and they’re providing it at a very competitive price.

The proposition is different: they market to the same target audience but the selling point is not their tools — they offer a run of the mill JAVA-based platform — their selling point is the market that they can deliver. A built-in customer base of [enterprise] users and a marketplace for connecting those users with the applications they want.

But I digress. Amazon’s SimpleDB is an important, if small, step towards moving the web to column-oriented databases like Google’s BigTable, or the relatively unknown, opensource Hadoop project, now largely sponsored by Yahoo!.

What sticks out to me most, however, is the choice of name. It’s called “SimpleDB” and yet it’s neither a “database”, as most would understand it, nor is the concept of a column-oriented database “simple”; it requires an orthogonal way of thinking. What is clear, however, is that the choice of name was a very deliberate move by Amazon to market this technology to the masses. It will take some time for developers to come around and see the light, but when they do, we’re in for another huge advance in dynamic web applications.

And you may find yourself in another part of the world

Posted by chetan on December 16, 2007

Ever have a moment where you suddenly stop and ask yourself: how did I get here, to this exact moment in time?

For me it’s almost an out of body experience where I look [down] at myself in a particular instant and I try to work backwards to the events that led up to it. It seems to be happening a lot, of late, and it always comes back to the same question. Two weeks ago, I was sitting in the domestic terminal at the Bombay airport, reading a book of no consequence and listening to my iPod while waiting for my flight to Pune when I suddenly looked up and thought to myself, “what the fuck am I doing here? How the hell did I get here?”

It happened again yesterday. I was sitting in my hotel room, eating a fabulous lunch (brunch, really, but it was 3pm; scrambled eggs, french toast, hash browns, chicken sausage, and hot chocolate) and looking out the window, out over the gloomy sprawl of Chennai before me, when it came up again: “what the fuck am I doing here? How the hell did I get here?”

There’s an obvious answer (I’m here for Operative as their resident software ninja, blah, blah) but that’s not good enough. It doesn’t really answer the question as to why I’m here, in the here and now. Even more, I can think of lots of reasons that I shouldn’t be here; why I should be somewhere else, doing something else. And only a few reasons (e.g., conscious decisions I’ve made) for being here.

And yet here I am.

Nice shoes, but I still think you’re stupid

Posted by chetan on December 16, 2007

My boss asked me a question recently, which really got me thinking: would it make any difference if I [my boss] wore slacks and a shirt every day? That is, do outward appearances at the office really matter that much? Could dressing a little better mean being taken more seriously?

It took me a really long time to answer the question, because to me it doesn’t affect how I judge a person’s performance, and it was difficult for me to look at the question objectively and realize that not everyone thinks the way I do. Everyone has their own criteria for judging performance or value. Which got me thinking about meritocracies and why they don’t work in business.

Many large, successful Open Source projects are, or appear to be (I’ll get to this later), meritocracies. The Apache Software Foundation (ASF) is a popular example. So what’s so different about open source software projects that a meritocracy not only works but seems to come so naturally?

Clearly much of it has to do with the voluntary nature of open source; when faced with an adverse situation in the management of a project you have two clear options when all else fails: stop contributing or fork. Both options carry an equally low risk, from an individual standpoint. The same two options are available in the business world, however the risks are much greater when one’s career and livelihood lie in the balance but the rewards are potentially much greater for forking — e.g., quit and start a competing product or service.

Even in the latter case, can a meritocracy really work in the long run? The incubation phase of almost any startup is entirely merit-based — from acquiring funding to competing in the marketplace, both will be judged on the merits of the ideas and the products or services the business is able to produce. But how long can this be sustained? Once a company grows out of it’s core group of people into a full-fledged business, it becomes increasingly difficult to keep the same level of excellence and far more costly to replace people at the upper levels. The stakes are also much higher for everyone involved, as noted by the ASF:

“What is interesting to note is that the process scaled very well without creating friction, because unlike in other situations where power is a scarce and conservative resource, in the apache group newcomers were seen as volunteers that wanted to help, rather than people that wanted to steal a position.”

So it becomes an issue of power and the ability to wield it. Bill Gates commented on a recent survey, saying: “Communication skills and the ability to work well with different types of people are very important too,” he said. “Software innovation, like almost every other kind of innovation, requires the ability to collaborate and share ideas with other people, and to sit down and talk with customers and get their feedback and understand their needs.” In other words, to innovate you need more than just technical ability and great ideas. You need the ability to communicate those ideas and work with people who may not always be as strong technically.

And that’s where strong leadership comes in. The problem with any large organization is that it’s almost impossible for everyone to be in complete agreement all the time. Even in a merit based organization like the ASF, consensus is not a given and people have found ways to game the system. Without strong leadership at the top you’ll have a constant power struggle to fill that void at every level in the organization, as different groups vie for control. (See also: IRAQ)

The best ideas can win out, but sometimes it takes a dictator.

I found a pebble in my hair

Posted by chetan on December 15, 2007

I woke up this morning and there it was. Now I know why I kept thinking something was falling on my head last night! It wasn’t just my imagination…

The Da Vinci Code

Posted by chetan on December 15, 2007

As far as film adaptations go, I thought it was pretty excellent. It pretty much played out exactly as I pictured it when I read it in January. Of course, the movie had already been out for some time when I read it so I knew who was playing the various characters but I pictured Robert Langdon to be an older, bearded version of Tom Hanks, not the young, long-haired version in the movie. A little more like Indiana Jones than Nic Cage in National Treasure, I suppose.

The one bit in the movie that I couldn’t remember was the part about him falling down a well as a kid and having claustrophobia ever since. Not sure if they just threw that in or what, but apparently I’m not the only one.

international indeed

Posted by chetan on December 15, 2007

11:31:48 PM chuan: funny
11:32:02 PM chuan: eating thai curry and watching tokyo drift in india