iOS5 is here

[Programming Language] is great because non technical people can understand it

The VBScript from hell: typed vs. typeless languages

iPad: the sad state of the art

SELECT * is bad for performance

XSLT Transformation to analyze MSSQL Server 2005 traces

During the final user validation of a new packaged application, we were getting complaints of bad performance, and the usual suspect, the database, was apparently perfectly well. This is where the SQL Server 2005 trace facilities shine. You can trace what is going on inside the database engine with quite a high level of accuracy, including filtering the "noise" generated by other users, other application components, or yourself.

After running a trace while users were exercising the application, I became completely convinced that the database was not at fault. However, I wanted to put my conclusions on a paper, and for that there is no replacement for issuing a formal report. Problem was, to create that document you need to take the trace data and use it to generate charts, rankings and whatever you need to support the conclusions.

This was going to be easy, just import the trace data into a spreadsheet and play around with it. Let's start with the trace data itself. The first thing you need is to save it on a file. The SQL Server trace tool only gave me the option of saving it on its own proprietary format, intended to be used by the tool itself, or use XML.

I choose XML, after all this is something close to a universal standard, isn't it. Yes, it is. I took my XML file and tried to import it in Excel. Only to discover that Excel was expecting basically a "flat" XML file, where the rows of your dataset are children nodes. SQL Server 2005 trace format is not like that. SQL Server 2005 uses a generic "Column" node with an identifier and a name.

No problem, I said to myself. This is XML, so a simple XSLT transform should get me the data in the columnar layout that Excel expects. Simple, right? So simple that I tried a couple of Google searches before convincing myself that either SQL Trace tool is not that popular or simply nobody had wanted to do this before.

And in the end, simple it was. Many ages ago I wrote quite a few XSLT transformations, so I was familiar with what I wanted to do and how to do it. However, memory fades quickly when you don't touch a subject in a while. And there are always new things to learn, which is to say that there are surprises around the corner, even when you use something as standard and plain vanilla as XLM. In the end, it took me the best part of four hours to get this right.

So much time over what I expected that I'm documenting it here for future generations and my lousy memory as well. And because to be honest, I expected that someone had already done this.

First, the XML output of the SQL Server trace tool contains an innocent looking line at the beginning:

<tracedata xmlns="http://tempuri.org/TracePersistence.xsd">

It took me a while to discover that my XSLT processor was trying to fetch that XSD and failing to walk beyond the root node due to silently failing at that. So the first thing you need to do is to edit the trace file and leave it as:

<tracedata>

Almost there. Then you create an XLST file with this (sorry for the lack of indentation, I don't want to spend too much time fighting with the Blogger way of rendering):

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<!-- Extracts the SQL Server trace data to columnar format -->
<!-- Author: consultuning@gmail.com -->
<!-- Last updated: 4/6/2011 -->
<xsl:output method="xml"/>
<xsl:strip-space elements="*"/>

<xsl:template match="/">
<Events>
<xsl:apply-templates select="TraceData/Events" />
</Events>
</xsl:template>

<xsl:template match="Event">
<Event>
<Name><xsl:value-of select="@name" /></Name>
<xsl:for-each select="Column">
<xsl:variable name="ColumnId" select="@name" />
<xsl:element name="{$ColumnId}">
<xsl:value-of select="." />
</xsl:element>
</xsl:for-each>
</Event>
</xsl:template>

</xsl:stylesheet>

And use your favorite XSLT processor for generating the transformed XML:

xsltproc [xslt file] [xml trace file] >[output XML file]

And that's all. Well, you'll get some data truncation warnings from Excel if the traced SQL statements are too long, but overall you'll be able to import this into Excel without problems.

The worst part of all of this is, after looking at the trace I discovered that there were some users having seriously bad response times. But that is the subject of another post.

Advice for spammers

I must confess: I don't know how most of the internet based business works. But from what I read, it seems that "pagerank" plus "traffic" is all you need to earn tons of money without actually doing much of value. There seems to be whole operations, called search engine optimizers^(*) (or SEOs) devoted to changing a site or a set of sites so that they appear in the top Google search results.

How it works, I'm not sure. And it has not interested me too much in the past. From what I understand so far, these SEOs trick Google into displaying in its top results links to sites that are merely a collection of ads, plus some content literally ripped of from somewhere else. The idea is to leverage the enormous scale of the internet so that even if only a small percentage of users click on your ads, you get a respectable amount of cash in reward of your hard labour. If you're thinking of how spam mail works you're right, this is exactly the same logic. One gullible user in a million is likely enough to pay for all this in ad revenue.

If that's not fraud, it is pretty close. At the very least, they are polluting Google search results and giving its customers less useful results only to get a slice of that valuable traffic and ad revenue and probably violating a few Google terms and conditions.

The problem is, the Internet is so huge that even a company with the resources of Google cannot completely stop this. More so, Google is always striving to eliminate manual intervention in anything in as much as possible. Which is something that creates a never ending arms race between fraudsters and Google. Entire sites and organizations are devoted to second guess what the latests updates to Google ranking algorithms means for their customers so that they can keep appearing at the top search results.

I wish someone with the actual resources to quantify it could estimate the cost of this kind of activities. Probably the sheer infrastructure costs of those operations, plus the time wasted by Google users on useless search results would be more than enough to end poverty in a couple of countries. Each year.

Sorry, going off topic here. And flawed logic: while poverty could be eliminated from a country, how SEOs would pay the rent? The idea will shift poverty from one place to another, with the only hope of getting a more valuable return from one group than the other. Which opens another completely different debate that this blog is not really prepared to enter.

What is really interesting to watch is the arms race. Each time Google refines its ranking algorithms trying to defeat the cheaters SEOs trying to return links that are actually useful for the user, cheatersSEOs change their tactics to remain undetected.

One of the ways of increasing your relevance has been, since the original PageRank publication, the number of sites that contain links to another site. Over time, Google has changed the original algorithm, sure, but there are cheatersSEOs out there that are convinced that external links still adds value.

As blogger allows commenters to enter a URL for their site, there is really no other way than cheatersSEOs of explaining some of the comments I'm getting in this blog. So far, blogger has been pretty good flagging them as spam.

But I keep thinking that some of them could actually be relevant and useful. So let me clearly explain the guidelines for comments being accepted on this blog.

• Comments should only absolutely agree with the points made in the post. The purpose of the comment is only to praise on the following areas:
1. How clever, clear and structured the blog post is.
2. How the post is magically synchronized with the hot topics in the industry.
3. How appropriate and relevant to the problem the reader has at hand. In its infinite wisdom, the blogger always chooses the topic that is most interesting to the reader in this precise moment of its existence. Both of course for personal and professional dimensions of life.
• Any dissenting comments will, shortly after posting, realize their enormous mistake and post immediately an apology that summarizes the qualities emphasized in the first rule.
• Logic arguments are allowed as long as don't conflict with the first rule.
• Passionate discussion is allowed as long as it supports what is stated in the first rule.

Above all, there is a meta rule that has to be always met. Even when it contradicts the first rule. Mind you, you still need to maintain certain level of internal consistency in your comment. As a recent example, take  Delhi Sexy E***rts (sorry, no link provided). I no doubt that its employees are really improving their lives by reading this blog. Even find the topics discussed interesting and engaging -perhaps there is, after all, a level of affinity among the professions, again a topic for another discussion. But please, do not mention how good you found the different tabs in the site. This blog does not use tabs. Please.

Remember, anything not following these rules is automatically flagged as spam.

Before the flames start yes, I know, there are people out there that call themselves SEOs that are not interested in making you jump to a site at any cost. Yes, there are SEOs that just make sure that your site is correctly structured so that search engines can index it really well. But compared to the other kind of greasy SEOs, they are in minority. Proof? Quick, what is the average signal/noise ratio of your latest Google search? How many times have you given up on jumping link after link ending up in different pages that contain exactly the same text? My advice for these "white hat" SEOs: find another name for your profession. SEO is becoming quickly an undesirable term for a resume.

^(*)See, I know so little about how internet business works that perhaps SEO does not actually mean that. Maybe its short for "Search Engine Organizers". Or "Search Engineer Operations"

Anything not scripted is a risk

Is your privacy worth 100$?

Suppose you come across some nice guy on the street that makes you the following offer:

Hey, man, you really look like a nice person. Can you give me your list of friends, tell me where you are during the day, tell me what your opinions are in related to topics of my choice and generally tell me what/when/if you like or not things I ask about? I’ll get this information from you during your entire life, and perhaps I may want to share it with a few selected partners. You’ll have no control of who I partner with and which pieces of information I will share.

Being usually one kind of interrogative person, you’ll naturally ask back, of course, how much will you pay me?

This is basically what consumer panels have done for years. An old tool in the market analytics since a lot of time, consumer panels have been used by marketers to try to understand how people behave, react and interpret things. The biggest problem with consumer panels, besides costs, is I think what is called the "documentary effect": people usually act differently than they say. This term comes from media panels, because you're more likely to say that you watched an interesting film about quantum mechanics than admit that you were engaged with the latest "Got Talent" edition.

Now, put down in a piece of paper what you consider a sensible amount to charge for this information. Do it before you read the rest of the post.

Then suppose you came across some nice guy on the street that makes you the following offer:

Listen, I have an irresistible proposal for you: see, I will give you the ability to stay in touch with your friends, send them mail messages and chat with them about your holidays, show them your photographs and allow them to write comments about them.

Being usually one kind of interrogative person, you cannot resist and ask: how much all this will cost me?

Now, put down in another piece of paper what you consider a sensible amount to pay for this. Forget for a moment the plethora of services that provide this for free and force yourself to write a number.

Then take the first and second papers and put one besides the other. Now take the first paper and write below your number: 100 USD.

I'm not sure if the number is fair or not. Perhaps it's not too much, after all. That’s what this information is worth for Facebook(*). If I was explicitly asked to sell my personal information, I'd consider 100 dollars a very low price.

I haven't seen any Facebook business plans, but I think it's not crazy to assume Facebook business model revolving around giving a way for marketers to advertise with scalpel-like precision. The cost difference between what a consumer panel, with all its lack of precision costs, and the maximum 100 USD Facebook per user value is a dream come true for the marketer, especially since you're avoiding documentary effects.

And for Facebook, after operational and capital expenses, the difference is their profit.

And what's wrong with this? After all, Google is giving away excellent services for free in exchange for the privilege of being able to snoop your data. On paper, Facebook just moves this idea a bit further.

My only concern with all this is, what do you get in exchange for all those profits FB will collect? The ability to send messages and upload photos? Give me any day a Facebook like where I could, with a one off payment of 100 USD, do exactly the same that can be done with Facebook.

I'd pay them. Assuming of course that I was interested in using FB-like services. Which I'm not. But that's another story.

(TL;DR and advice for prospective clients: no, I have not a FB page, profile or anything like that. Nor I feel the urge to do so. And I'm sure that FB owner is a nice guy bad and this post was not meant to insult people that enjoy FB)

(*) Estimation based on 50 billion USD valuation divided by the estimated 500 million active users.

The Japanese Daimyo and the Ninja School

About three hundred years ago, Japan was a country fragmented into many small kingdoms or clans, whose leaders were constantly battling among themselves.

Wars were constantly going on between different clans, with each one having to depend on an increasingly sophisticated system that provided the manpower and resources necessary to keep their territories safe and at the same time take advantage of the weaknesses on the other side and expand their area of influence.

War lords were looking for creative ways of getting the most out of all the money they spend. And one of their pet peeves was the cost of their armies. Not only it was a huge piece of their budgets being used to pay and feed the army and its suppliers, but also keeping a permanent army of trained people just in case you need them was a drain on resources for other activities on their territories. A solider that is idle is not harvesting, building, or otherwise producing anything else. Add to that the cost of training them and war was an expensive proposition.

To contain the cost drain of having to sustain an army during peace periods, they tried a few solutions. The simpler one was to treat all the population as soldiers, but only ask them to go to war when necessary. This soldier on demand approach did not looked like a bad idea. Most of the time, during peace periods, farmers could farm, craftsmen could work on their craft and artists could create art. If the need arose, those folks could fight for the kingdom as well, and during the usually short time when they were at war was not a significant interruption of their production cycles.

But soon a problem became apparent. Ordinary people, even trained, did not put too much passion when fighting. They had other interests in life beyond becoming skilled in using weapons or killing enemies. On demand armies were routinely defeated by much smaller, professional armies. Those armies were composed of skilled individuals, specifically trained for that purpose, and more important, that wanted to be soldiers in the first place.

But for the war lords, keeping a permanent army had also a few problems beside the cost. For one, professional soldiers want to permanently improve their weapons, which has a cost. Also, they want from time to time to practice and hone their skills, so they tended to propose war as the solution to every problem the lord had. The lords did not wanted to have their decisions become overridden by the military, and the military wanted to have proper careers where they could climb up the command hierarchy based on their merits in the battlefield.

As the armies got bigger and powerful, the lords also felt that the military were limiting their choices, sometimes even dictating his decisions, based on some arcane military strategy concepts impossible to understand. Oh, and the pain of technology upgrades: each time a new weapon was created they had to pay for expensive retraining. Those tried to save on training costs found too late and in the worst place, the battlefield, that savings in that were offset by much higher costs later on. And don't forget the pockets of resistance, of soldiers rejecting new weapons just because they were used to their classic ones. These group was the worst, for the war lord knew they were going to be killed in the field.

So neither the dedicated army nor the on demand one were a good solution. The lords, clever as they were, still wanted to improve the cost efficiency of their armies, loose their dependence from them and at the same time keep intact their capability of fighting over their neighbourhoods.

It all started with the training. Retired soldiers started to set up small warrior schools that trained the future soldiers. Based on their reputation as former warriors, those Ninja masters established themselves as the reference on combat training excellence. Lords started to send their future soldiers to those schools, as a way to save on training.

Soon, they realized that two things were happening. First, every other lord was sending their soldiers to be trained at the Ninja schools, so they armies were on par in terms of skills with their enemies. But they told to themselves that it was their military command that made the difference, after all, fighting skilla were already a commodity. There's little to learn once you've mastered the martial arts, the sword and the firearms. Second, the training costs were a small part of the overall cost of having an army. They still had to keep their soldiers feed and happy during peace periods.

Until one of them, famous for his forward thinking and proactive attitude, had what seemed to be a good idea. Instead of keeping an army just in case we need it, let's hire one on demand when we need it. That had many advantages. Their soldiers will be well trained by the masters, so the army will be as powerful as it could. And the best of all: no war, no costs. It was a win-win situation.

At first, he was a bit worried because he thought, what if other lords have the same idea? Will that end in a ridiculous situation where a Ninja school will end up engaging in war against himself, and get paid by two different lords to do it? Nah, Ninja masters were honourable enough not to do that, and the conflict of interest would destroy their reputation. Could he lose complete control over war operations? Nah, he had a couple of generals that will kept under his servitude that could override the Ninja master decisions at any time.

It was all well and good at the beginning. The kingdom enjoyed immediate benefits, as more resources could be allocated to grow and prosper instead of fighting in conflicts with other lords. Ninja masters charged a reasonable price and even took in their payroll most of what was the army.

But then, one day, the lord had an idea to expand its frontiers. Of course, it would require applying some force over the neighborhood kingdoms. No problem, he thought. He went to the Ninja master and shared his great idea. But the Ninja master nodded and said, "sorry sir, we don't have enough soldiers to do that" To which the lord said "well, just recruit some more" And the Ninja master answered, "I cannot, good soldiers are difficult to hire, take a lot of time to train and I've a good portion of them already engaged in a conflict in the other side of the country. Nothing that affects your security, of course, but I don't have that many resources"

The lord had to back out from his grandiose plans, but he was still confident that his decision had been the right one. At least he could count on having an army as good and as big as the one that he transferred to the Ninja school.

Until one day, he was attacked from the lord in the southern border. At the beginning of the conflnict, he lost a lot of territory because his on demand army took a lot of time to appear in his defence. The ninja master kindly pointed out that his time to react was agreed in advance and he was under no obligation to mobilize its forces sooner than that.

What should have been a short conflict that should have been settled down fairly quickly transformed into a long agonizing war that took a lot of time to come to an end. The lord realized that the Ninja master was not putting all its resources in his war. Instead, he was playing a delicate cost benefit balance, keeping enough forces assigned to defend the lord so that he would not lose the war. But was not willing to go the extra mile to resolve the conflict sooner. In fact, the longer the conflict, the higher the profit for the Ninja master. Worse yet, most of the soldiers that he transferred to the Ninja school were no longer there. The Ninja master had sent them to other conflicts or fired them according to their own interest. There was no loyalty or passion in the fighting. What the lord believed was saving by externalizing his army was more than lost the very same moment that he needed the army.

Worse yet, he was at the mercy of the Ninja master. He had now in fact the power to surrender the kingdom to the rival neighbourhood if he wanted to. The lord had no choice but keep paying increasingly higher sums for a decreasingly empowered army, with soldiers whose skills were not up to date. The icing on the cake was when he asked the Ninja master to improve their soldier's ability, to which the Ninja master answered "well lord, you know, training is expensive and your fees are not enough for that"

At this point, the Ninja schools were the ones that actually decided when and where conflicts were going to be started. And who will win them. And how much would the lords paid for that. The lords had lost any control over their ability to use force to attain their objectives. The Ninja schools ruled the island, and there was no way for them to regain control.

The lesson? They forgot what the purpose of the army was. Blindly lowering its costs without knowing what compromises are being made is not good. Making your war cheaper is useful as long as you don't sacrifice its purpose.

And, if you've read this far, you are probably wondering, what's your point? Replace in the history the fictional war lord with a modern corporation and a Ninja school with a modern outsourcing software services company.

(N.B: any history references not completely false are due to sheer luck and not attributable to any documentation or fact finding work)

CAB and TAB: a waste of time for everyone

I've been trying for weeks to write a piece on ITIL that did not sounded vindictive or bitter. I really wanted to be fair on ITIL, or to be more precise, I'm all in favor of having some kind of change control and IT management in place.

But having to go through the ITIL recommended change management processes a couple of times has revealed me the true costs and overhead of ITIL. I'm currently immersed in an organization that drunk the ITIL kool aid a few years ago. And so far, my experience with the ITIL implementation is very, very negative.

A couple of stellar examples of this are the Change Approval Board (CAB) and the Technical Approval board (TAB). According to ITIL, these boards should review and approve every change made on production environments. In theory they are composed of the people with the most extensive technology and business knowledge. This is because they are the last line of defence before anyone can touch live systems and wreak havoc.

So far, theory is good. This small but selected team of individuals can detect and sort out incompatible changes, and block them if they step into critical business periods, even deny them completely if they don't fit with the technology standards. To make things more convenient for everybody, and unless there is an emergency, the team meets on a weekly basis. Otherwise, they would be unable to do any other work than review and approve changes.

Let's try to determine the composition of this CAB and TAB teams. For a big organization, you'll have attending CAB and TAB meetings someone with knowledge of the ERP system, right? Oh, yes, and put in some networking guys. And some storage people. With luck, your organization has also a CRM application, so add one to the mix. And is far too common, especially in big multinationals, to have their own localized versions of ERP and CRM packages. Well, also you probably have some customer facing web sites, do you? Another one to add. Done? Not yet, then there is the desktop support people and the security and incident response teams probably have something to say about changes. And don't forget the server maintenance team. Ooops, sorry, if your CRM and ERP have a database component, let's add one DBA to the mix, possibly one for each database flavor.

Done? Wait, are there systems related to production facilities? Add one more. And so on. The point is: in complex and big organizations, the technology environment is complex and the knowledge is fragmented across many different teams. Each and everyone of them needs to be represented in CAB and TAB meetings.

I see a few problems emerging from all this:

One, while the size of the team is by itself something to worry, that's nothing compared with the different degrees of connection between them. The networking team usually do not talk very much with the ERP team, while the DBA team is very close to the storage team but almost completely ignorant of what the networking team does. And so on. For something so focused on productivity as ITIL, it is surprising that they dictate allocating a lot of valuable time from valuable people just to approve changes. More so when a fair amount of those changes are tangentially, if at all, interesting for them. Seriously, if the networking guy wants to add a firewall rule, do you really think the ERP guy is going to stop him and ask if he's using the right subnet mask?

Second, these approvals are holding off changes to a weekly cycle. Rare is the project or change that can be executed ahead of schedule, but if that happens, forget about applying your changes earlier than planned unless you've managed to save a whole week of time and can catch up the prior approval meeting.

Third, the cost of validating the change could be very well higher than the cost impact of the change going wrong. ITIL is full of "everything has to be managed" in the name of being able to find efficiencies, but little is said of the cost of this management.

Clearly ITIL does not see those things as problems. And reading around ITIL, I think I've found why. Picture this, when you last saw a small group of highly skilled individuals that could have visibility and knowledge of a big production environment, whose cost was already sunk in other activities and working in organizations with long, long, change cycles?

The mainframe world, of course. ITIL was created for the mainframe shops, and it is likely to be a much better fit for those environments.

For the rest of today's world, CAB and TAB are a waste of time. Any half baked web based workflow management system can do better, and provide the same level of change control. Plus, it allows the DBA guy to skim over the network changes, and the network guy to just surface scan the ERP module changes. The problem? It would not be ITIL. The consecuences of that are the subject of another post, probably.

But I don't have time to write that post, I need to attend a CAB meeting now....

Unicode explained to the younger developers

I had recently a question from one developer about the different ways that Oracle has to declare a string column. He was specifically confused between VARCHAR2(xx), VARCHAR(xx) and finally VARCHAR2(CHAR xx).

Short answer: forget about VARCHAR, use always VARCHAR2. The difference is related to how it handles Unicode characters. VARCHAR2(xx) will have room for xx bytes, where VARCHAR2(CHAR xx) will have room for xx characters. If your dealing with new code, use VARCHAR2(CHAR xx), if you're maintaining legacy code use VARCHAR2(xx). The legacy code will have issues dealing with Unicode, but you'r going to fool it if you think that using VARCHAR2(CHAR xx) will improve the situation, because other places in the code are likely assuming VARCHAR2(xx) has some way of dealing with Unicode, or none at all. Either way when those parts use your tables, they will likely not properly understand what's there.

While the question was interesting in itself, he was puzzled by the answer. His answer was, "that's all well and good, but why there are so many different ways of declaring what is simply a string?"

Here's one of those moments where experience weights in and you can indulge in a bit of history, and your interlocutor, who feels that at least your answer deserves some gratitude, listens to your explanation anyway. So here it goes.

In the dawn of time, character sets had 7 or 8 bits. ASCII, ECBDIC or some ASCII variant. In those ancient times, 8 bits were enough to store a character. Maybe you had some bits to spare, especially if you were using English characters, but 8 was enough. The only problem for applications was to know which character set they were using, but was usually easy to solve. Did you say the only problem? No, there was another, much bigger problem. 8 bits were enough to represent most of the western world characters, but 8 bits were not enough to represent all the characters at the same time.

That meant that if your application had to deal with one character set, you were fine. If your application had to deal with many different languages at once, then you were in trouble. What you do, store for each string the character set it was using?

There was no good solution for this problem. But using 8 bits for everything had a lot of advantages. Each character was a byte. Millions of lines of code were written assuming sizeof(char) == 1. Copying, comparing and storing strings assumed that each one took one byte. The world was a stable place for almost everyone, except for the poor souls who had to maintain applications that worked with languages (Chinese?) that needed more than one byte to represent a character.

Then came Unicode to save the world. The only problem is, depending on the way you choose to represent Unicode, you may need more than one byte for each character. In some of the most popular, backward compatible Unicode encodings, you actually need a variable number of bytes to represent a character. Time to review all your string handling code. You can no longer assume that increasing by one a pointer will get you the next character. You can no longer assume that a string needs in memory as many bytes as characters it has. You can no longer compare the byte values of each character to determine their sort order.

Of course, if you're young enough, or have never the curiosity to use C/C++ or FORTRAN, you've never seen this problem. Your handy string class provides everything you need to handle Unicode wrapped in a nice package. The memory size of char[] and byte[] are different, but you essentially don't care about that.

Oracle, being in existence before the Unicode days, is of course greatly affected by the change. Not because Oracle cannot adapt itself to Unicode, but because of the huge codebase that needs to maintain backwards compatibility with. That's why they have invented VARCHAR2(CHAR xx). It is for them the best way to support modern Unicode encodings and at the same time remain backwards compatible.

Since VARCHAR2 is not standard SQL to begin with, extending its syntax further is not a big loss anyway. So next time you have to interface with legacy string data, think about how it is encoded.

This code is crap

Helping customers get the most out of their systems is good and interesting job. One gets to know a lot of industry sectors, processes and ways of working, as well as some good people.

Invariably, the job involves reading code. And writing code, usually a tiny fraction of the whole body, because you focus on the parts that provide most benefit to your customer. Whenever I wrote code, I always try to make it stand out for its clarity, performance and readability. Experience has taught me that having your code reused or called from a lot of places is a sign of a happy customer, so it made sense to do my best for the customer.

As you can imagine, one develops a good sight for reading code. And I've seen my fair share of bad code over the years. Very very bad code. It is difficult sometimes to keep yourself neutral and resist the temptation to think of yourself as some kind of elite coder that can see what other people can't. With enough time and experience you learn that there are a number of factors, all of them human related, that can affect significantly the outcome of your work. Even if you try to deliver your best effort, sometimes you just were under pressure to meet a deadline and had to rush something out. Or perhaps you have a child with a cold waiting for you at home and that worries you much more than the quality or performance of what you're writing.

But that perception changed when I recently had to look at some code written five years ago.

At first, it was not that bad: at least the formatting was consistent. But it was pretentious and sophisticated beyond necessity. There were missing oportunities for simplification everywhere. Performance could be easily doubled with a number of simple, apparently obvious changes. There were comments, but essentially useless because they were centered in irrelevant parts of the code. Design decisions were not explained properly, there was no summary explaining why on earth certain algorithm was chosen or modified.

There were obvious refactoring spots all over the place. The code could have been much sorter, cleaner and efficient.

I was becoming impatient with that code, and finding progressively more difficult to sympathize with the original coder.

The only problem was, I was the one that wrote that horrible code.

As it turns out, that was I considered good code five years ago. I was so shocked that I started to dig around other, older fragments, of my old code. Ten years ago it was even worse. What I wrote fifteen years ago was practically unreadable.

Suddenly, I felt the urge to fix all that. I then realized that these lines are still happily executing many times every day, processing millions of transactions. And nobody yet has replaced it with something better, probably because they don't need to touch it.

It took me a while to recover from, but then I think I learned something. I’m not as a good coder as I think I am. You’re probably not a good coder too. Yes, there are good coders out there. Probably the club of good coders has members whose last names are Knuth, Kernighan, Aho, Torvalds, Duff or Catmull. But I'm not there. Not now at least.

I'm now a good enough coder to recognize good code. Perhaps over time I'll get better, enough to be considered a good coder. In the meantime, what I deliver seems to be good enough.

The huge gap between geeks and business types

I've never been on the buying side of some work for hire sites, but I'm sure that they offer the potential buyer the option of  targeting to a single person a concrete job offer, because sometimes I get job offers that are not visible to the rest of the pool. Forgive me for the self promotion, I only can say in my defence that (a) I'm not making this up and (b) customer loyalty is one of the primary measurements of customer satisfaction and is something I'm quite proud of.

I suspect that they also offer additional options to the buying side to better target their offer to the individuals that look more likely to be a good fit for the job. It really makes sense, since those sites are crowded with hundred of thousands of potential candidates, and the prospective buyer knows in advance that most of those offers, while dead cheap, will not provide the services that they are looking for. One of those options has to be "send this job offer to the top 1%"

Being in the top ranks of some of those sites, I frequently receive job offers that seem to be targeted at that top 1% but unfortunately (or fortunately, see later) fall outside of my main knowledge domain.

The last one I received exemplary illustrates a lot of what is currently wrong with the perception that business types have of software development in general. The job description reads:

"h.264 codec implementation is not following standard specification including for AVCDecoderConfiguration record (you can find it in the specifications for H.264)...please complete that compliance.

if you are comfortable with x.264/ ffmpeg or you are a c++/vc++ expert, this should be a very easy task for you."

My first reading did not trigger any alarms. After all, if you're familiar with the H.264 specs, it should be easy for you to find the reference spec. A bit harder would be to find the place in the codec source code where this record is incorrectly generated, and fixing it with all the testing necessary to make sure you are not breaking anything else would probably be an order of magnitude more complex.

Still, if you're familiar with H.264 this should not be too hard. Not "very easy" as the job offer reads, but not too hard. Much harder is of course, to reach the level of mastery necessary to understand the H.264 standard and be able to program a codec for it. Let's see some of the barriers that someone that has reached that skill level has had to overcome.

• H.264 is proprietary, to have experience with it means having experience in a media related technical business. Unlike other fields where you can write the next Apache killer at home, usually involvement with H.264 means a paid job.
  
• H.264 is large and complex. I doubt that there is a single individual that understands everything in the standard down to the necessary details. Most likely, there are field experts in each area of the codec pipeline that understand completely each associated section in the standard.
  
• The skills necessary to write an optimal video codec in C/C++ go far beyond the intermediate level. It takes years to write a good video codec implementation. Nobody is going to come out with one out of a weekend hobby project.
  

The resulting profile is someone who has mastered the H.264 standard, which means experience in a commercial codec vendor, together with C/C++ skills well above average. Maybe someone that has devoted the canonical ten years to perfect and improve its skills in video compression.

Would you trust someone with that profile to fix the codec? I surely would. The next question is, how much this is worth?

This is the point where I usually read the offered amount. According to the job offer, the maximum amount that the buyer was willing to pay for this was $120. According to some sources, this amounts to less than two days flipping burgers in a fast food restaurant.

Remember: you're not paying for the time, you're paying for the experience.

Customer constraints may affect your ability to tune

Blog anniversary

The great NoSQL debate - what NoSQL is NOT good for

The great NoSQL debate - what NoSQL is good for

The great NoSQL debate - History

Security errors and overmitigation

2009 Database Tuning Predictions

Vista patching, or how to miscalculate the costs of owning an OS

Indexes cost money

All this industry talk about cloud computing is, more than ever, placing under the spotlight the true cost of IT infrastructure. Combined with the current economic climate, this is sparkling an interesting and refreshing debate on what is the actual cost of supporting and growing an IT infrastructure. And more and more, a realistic cost breakdown of how much each piece of the infrastructure is emerging.

And one of the surprises of this exercise is the exploding costs of mass storage.  Data centre storage is under an increasing pressure. Data Warehouses keep storing more and more data points and history. ERPs continue to take on more business processes. Web front ends generate incredible amounts of detailed user activity information. Graphics and multimedia file formats are taxing file servers and document repositories at the far end of their capacity. Mail is now an accepted medium for conducting business, and mailbox size is increasing without an apparent limit.

On top of that, legal regulations demand that most everything has to be kept during five, seven, or in extreme cases I've seen, fourteen years.

Technology is also evolving to catch up with all those demands. From SAN/NAS storage, tiered data management, storage outsourcing, you name it. But, except for some notable examples (Amazon EBS, Google) the business IT environment at large is failing to meet that demand.

This is even more difficult to explain to the non-IT executive levels, since at the consumer level storage, there is an expanding and ever increasing amount of USB and firewire devices offering amazing capacities at prices that are getting lower and lower. Not to mention that the average desktop PC now comes with a 1GB hard disk as the standard size.

Usually, the point that non-IT business folks fail to realize is that it is cheap and easy to increase the raw amount of storage available.  Not so cheap is to back it up daily, keep that archive during five years and make sure that is recoverable.

My usual response in the typical elevator conversation about storage cost with business types (yes, those conversations actually happen) is to answer, "yes, you can purchase a couple of TB for a few bucks. Now, tell me what you do if those disks fail. Will you be making back ups? Will you be able to recover them in five years? Will you be able to provide a replacement in 4 hours if they fail?" It's not that they are trying to outsmart you, it's simply that all those dimensions of the support infrastructure costs are simply invisible for them.

Oh, I hear you cry, is this one of these posts of late that do not have anything to do with database tuning? No, this time my point is actually related to DB tuning. In fact, all of the above is essential to introduce the topic. The topic is "just create an index and this will be faster"

This is so wrong, and I think that I've blogged it before, that deserves to be refuted in all its possible dimensions. First, yes, there are situations where it actually makes sense to create an index.

But an index is not free. First, an extra index in your database will penalize inserts and updates on the affected table. Second, the index takes storage space, that expensive and scarce resource. Third, sometimes there are better alternatives to adding an index. Two of the typical are:

- Caching values read from the database, instead of querying for them each time they are needed. This has to do with a tendency during maintenance to prefer always adding new code rather than change existing code. The more difficult to maintain the code is, the higher the chance that maintainers simply do not want to touch it. This is safer for them, but also imposes a performance overhead as it cannot take advantage of work already done by the existing code.

- Converting procedural code to SQL data set code. Query execution plans favor working with whole tables and result sets over slowly iterating over them.

So, next time someone proposes an index on that pesky 4TB table, ask, are there any other alternatives? Tuning exercises should not provide a single answer to a problem, but a balanced cost/benefit analysis of different alternatives. It's not enough to answer "just create an index", you have to say "creating an index is the best option as opposed to...."  That puts the conversation with the paying customer in the right level.

The tragedy of tactical vs. strategic solutions

I've noticed lately an increased tendency for the so-called "Tactical Solutions" in big organizations. Those are usually focusing their IS efforts around a few key systems, termed the "Strategic" ones. Strategic is synonym of big, complex and ambitious.

I've always heard the term "tactical solution" used to denote something that was put together quickly, without paying too much attention to the strategic aspects of an organization. Its data model does not try to encompass every possible process in the world, and its functionality is a fit for purpose for the existing business process, without taking into account the expected changes introduced by the business in the future. It's something built  knowing that it is going to be replaced in the near future by the all encompassing, strategic application, but solves now today's problem.

Fact is, the all encompassing, strategic application does not always arrive. Well, in fact, it's quite likely that it will never arrive. Two are the strongest contributing factors:

Plans are never completely executed: forget about five year long plans, face it. Nobody in any organization even remotely knows what they will be doing in three years time. Plans change as the economic environment changes, as competition changes, as legislation changes , as technology changes and as boards of directors change. When I see one of those five year long plans, I always do as if I accept them and cannot but admire the lack of grounding in reality that the people that make them and those that believe them have.

Complex solutions are likely to fail: the more complex and ambitious the strategic solution is, the more it is likely to fail to deliver. This happens also outside IS and is a inevitable side effect of having to deal with a complex problem. It's a fact. Complex development fails, at a much higher rate than simpler development. So even if people sticks to the plan and do their best to execute them, there is a built-in high probability of failure in those three to five year plans.

If you think I'm exagerating, I've seen tactical solutions that were intended to last 12 months being used for five years. And did the strategic solution arrived after five years? No, the tactical solution was replaced by another tactical solution, yes, you guessed right, intended to last only 12 months.

Of course, this is an extreme example, and I'm ready to admit that there are organizations (very few) that are actually able to stick to the plan and execute it during three to five years. But rarely the tactical solution is effectively replaced. Similar to the demise of dinosaurs, the tactical solution lower size and complexity allows it to change and adapt much faster and better to the environment than the monolithic and big strategic solution. And the point is, the "tactical solution" ends up playing a key role in the application landscape, and thus becomes strategic. As with any strategic system, it ends having to comply with the same requirements as its bigger brothers. It has to be fault tolerant, it has to scale up, it has to have change control, it has to be changed to adapt to business requirements and it has to be auditable.

What? You're not able to build those attributes into your tactical solution? Didn't you think about those requirements while building it and it is now heniuosly expensive to do it? Well done, you built actually what is a tactical solution. Instead, what if your tactical solution is fault tolerant, can scale up, is under change control, is easily changed to fit new requirements and is auditable? Congratulations, you built an strategic solution much cheaper and quicker than the strategic solution is going to be built.

Whatever the result, it's time to stop using the term "tactical solution" as something quick and dirty developed for the short term and instead start describing it as what they really are. Like the Ubuntu joke (Ubuntu is an ancient African word that means "I don't know how to install Debian") "Tactical Solution" means "I don't have time for the big guys to tackle my problem and I'm solving it myself"

Just take care of making it strategic while building it.

GAE: the good, the bad and the irrelevant

From what I read, the general feeling is that there are two irreconcilable factions in the cloud/no cloud debate: those that go for it and those that do not.

For such a highly technical topic, I'm surprised to see GAE being debated at almost emotional levels. In one side, there are folks that look like Google fanboys and see nothing but advantages for your business to host things in GAE. The other side seems to be in total denial of even consideration of using GAE for anything even remotely related to their line of work.

My view is that as with any question complex enough in life, the answer is "it depends". That's why I put together the following points that I think you should consider before betting for GAE.

Note that Amazon EC2 offers you a much lower level (system instance) hosting, these considerations still partially apply. And note that they are based more on experience with application hosting and development in general than my experience in GAE, which still is below what I wish it were. Bombs away.

The irrelevant

This may sound provocative, and in fact this is exactly the feeling I want to evoke. There a couple of things that I hear over and over against GAE but I'm still trying to understand the logic behind them.

Data Protection and security

GAE detractors say that they will lose control of their data by using GAE. Your data will be in Google's hands and that is bad. Let's face it, and please be realistic. Depending on how many degrees of outsourcing you have in place, your data is already in many hands, and is as safest as the weakest link on the chain. Very few business take even elementary precautions with their business data when dealing with it internally. Simple things like backup encryption are not being done for business sensitive data.

If anything, Google's massive scale and focus on technology will give the average business a much higher degree of data protection than they have today, if only because they can spread the huge capital cost of setting up all that infrastructure across thousands of customers. In theory, Google could potentially do whatever they wanted with your valuable business data, but it would be probably against their own interest to do so, if only for the lost business due to lack of confidence. And who do you think it has the greater incentive to use your data for nefarious purposes, the guy that takes the daily backup tapes away from your (shared? hosted?) data center or Google? Have you already taken any measures to encrypt your laptop hard disk? Do you allow for USB disks to be plugged on your workstations? Both thinks pose a greater risk to your data, at least judging by public reports and security experts, than anything that can happen inside Google's cloud.

I'm not saying that the folks at Google are perfect. They will, at some point, have a problem with the data they keep for third parties. Nobody knows when, but it will happen, if only because in any security scheme the weakest link in the chain is the human at the keyboard, and no matter how well Google recruits staff, they are not perfect.

I've yet to read any reports of security breaches at Google, and I'm sure that any hacker wishing to raise its market value would be quick to announce any kind of success with Google infrastructure. Security is a multiple layered onion and that the two topmost layers, the users and the application, are still being nursed by you. So look again at all those expensive firewalls and IDSs in your data center (that is, if you have any of those), guess how many of those Google has and start worrying instead about the most likely risks as of today: an application programmer leaving any SQL injection holes in your application or a user saving a spreadsheet with a list of all your customers in his laptop and the forgetting it in the back seat of a cab or in a train.

Uptime

This comes down to two reasons. The first is that they don't have the same level of control over scheduled downtime windows as they have on their own environment. I'm fine with that reason, and to some degree I'm ready to accept that as valid. But looking at those windows in GAE, while some of them could be inconvenient for your business, I'm sure that will compare favorably in terms of quantity and duration with the scheduled downtime that any local application you may have. Google environment is redundant and fail safe by design, that is, their need to bring the whole thing down to perform any chances is much, much less often than the typical environment. As for scheduled downtime due to application changes, this is more a factor of your application than the environment you choose to run it.

You also need to remember that GAE downtime history has been up until now more governed by it being a beta system than anything else. You should not expect to have similar degrees of downtime, scheduled or not, once they are fully up and running.

The second reason is truly, really, genuinely irrelevant. Unscheduled maintenance windows, that is, the time when the system crashes when nobody expects to, are the wors nightmares for any IT organization. And frankly, at any big consumer facing web business, be it Microsoft, Yahoo, eBay or Google downtime is way lower than anybody else in the closed environment data center world. Each time any of Google services, be it gmail or GAE, goes down, the critics raise their voices and say "that's why I'd never trust the cloud" I find it very hard to find environments where uptime is on par with the web-facing consumer giants, not even closer. Those folks have economies of scale that allow them to set up redundancy and fault tolerance on levels that the ordinary IT budgets cannot even dream of. So their unscheduled uptime is going to be lower than yours, period.

The most ironic part of it is that the people that make these kind of arguments against GAE are completely out of the GAE radar. The GAE proposition does not make sense in those enterprise environments that are above Google levels of security and uptime, that is, if they are actually meeting them outside of their dreams. But it's hard for me to find as many 99.99% uptime environments as GAE critics exist. My only possible explanation is that the criticism come mostly from armchair strategists. It is easy to "design" security and availability on paper. It is not easy or cheap to implement those designs.

And please, please, remember: I consider those points not being relevant in the context of discussing going with GAE or not. But that does not mean that they are not relevant. By all means they are, so always make sure that you have the right clauses in place whatever direction you choose.

The Bad

And yet I think there are genuine concerns you should be aware of when commiting to GAE, yet I don't see them widely discussed, perhaps because the bulk of the arguments belong to the the irrelevant ones.

Lock-in what you buy when you choose GAE is lock in. Good old lock in, of the deep variety. Any application you write for GAE will remain running under GAE forever as soon as it reaches any reasonable level of complexity. Don't fool yourself thinking that if you use Java your chances of being free are higher, because they are not. The GAE storage engine is different enough from anything else that you cannot run your application anywhere else. Period. If you want a good example, just look at GAE own framework, Django. The Django GAE has so many modifications over the standard Django framework that even Google is not making a sales argument of their Django heritage. Django is mentioned, but not promoted.

Rigid environment: by desing, GAE is not well suited to some applications. You may start your application with certain functionality in mind and later on want to expand it. In that expansion process, you may run into some of the limits imposed by GAE. By that time, by virtue of the lock-in, you'll have already invested some time and money in your application and you will then face the choice of basically throwing away most of that investment and switch to another, less restrictive platform, or start fitting your functionality to the GAE mindset: no analytical queries, no big batched transactions, no incoming mailbox, no …. I'm not completely aware of all the things you cannot do with GAE (or are extremely difficult and cumbersome to do as to not being practical) but it's likely that you'll run into any of them at some point. The problem with that, as I've already written somewhere else, is that Google itself does not have to deal with those as they have access to MapReduce and other technologies that allow them to sidestep any GAE limitations nicely. But you will not.

The good

Scalability. if there's one single thing they have mastered is how to massively scale everything they do. Massively. If you need that kind of scalability, Google is the way to go. And they keep commiting resources and capital to make their infrastructure even more big and scalable all the time. They have simply no competition in that front at this moment.

Uptime. Hey, all of us have heard of that server that has been running for the last eight years without a single reboot. Like urban legends, you never hear that from someone that has actually seen this machine, but only form people who know someone who know… While not breaking any records at this moment, Google uptime is among the best in the industry. Much more so taking into consideration the size of their operations. Also, I have the feeling that Google knows that this point is key to win the customer hearts.

In summary, I think that you should take a hard look at the bad and good before going to GAE. And, while not completely ignoring the irrelevant, take the time to measure them against what it is going to cost you to attain similar levels of uptime and security on your own. My bet is that what I've classified as irrelevant will actually look like, well, irrelevant, unless there are up front requirements (legal, for example) that prevent you to even pondering them in the decision. Then decide.