Databases as Monuments

In an earlier post, I mentioned wanting to transfer the focus of database developers from individual databases over to classes of databases.  Focusing on the care and feeding of individual database instances was the starting point for my journey to a class of databases.

I want to drill in to each one of the steps I took in the aforementioned post, including the place where I started.  However, rather than talk about what is wrong with each concept, I'm going to focus on what was right about it.  This is as much to challenge myself as it is to explore the problem in an interesting way.

Thousands of Years Ago, Before the Earth Had Cooled

you change your course for a
monument, not the other way around

Once, long ago, we thought of databases primarily in terms of the ultimately deployed instance.  This corresponds with the Lean concept of a monument - a piece of equipment that is so much a fixture that it controls your process rather than conforming to it.

Often, individuals were given caretaker positions and these caretakers quickly became responsible for the maintenance of both health and design.  The caretakers became gatekeepers in very short order.

Even early on, before agility was formalized, patterns-oriented development was codified, or test-driven development was understood, eye witnesses have told me this was a major impediment to the flow of new features in many software systems.

This is no surprise.  Monuments almost always impede flow.

When I was starting out as a developer, this was still a problem.  When I started capturing knowledge about test-driven database development, nearly a decade ago, this was still a problem.  I observe organizations today where this is still a problem.

The real question should never have been "How do we remove the impediment here?"  It should have been "Why does this problem exist everywhere I look?"

Lifeblood

The answer, as with many of the old ways that we so readily scorn today, is that there was something right about the gate-keeping behavior.  Not that it was completely right - I'm not saying that.  There was, however, one thing right about it: the motivation.

Databases are unlike most other kinds of software.  At least that is true in the case of production databases acting as the source of record for business data, which is what most of us think of when we hear the word "database."

Most kinds of software we write have two properties that are interesting in this post.  First, the designs are very complex.  Despite all our efforts to "keep it simple," we are addressing complex problems that demand sophisticated solutions.  Second, they contain little or no data other than their design.  In the old days there was some weird stuff about storing certain data as resources within binaries but that is mostly gone now.

By contrast, databases tend to have small, simple designs housing large, complex bodies of content.  That content is usually extremely important; like, "lose your job if it goes away" important.  As software becomes a more influential part of business, the contents of databases become more vital.

Cover Your Heart!  Cover Your Heart!

When something is vital, you protect it.  Sometimes, you protect it in a way that is otherwise detrimental to your health.

Imagine you are trapped under water but just barely.  Your wrist is caught in something and it is forcing you to face downward.  If you could just turn over, you can breath but you have to break your arm or dislocate your shoulder to do it.

You might think you're too weak to do anything about it - that you would drown.  That's certainly what we are all taught would happen.  Don't kid yourself: You're an animal.  Somewhere in your brain is a thing that wants to survive even if it hurts.  That thing doesn't care that you heard some words in school telling you that humans are different, special, and weak.

At some point, you'd start thrashing about until something gave and you'd stand a very good chance of flipping over and sucking in some air as a result.  It would hurt.  It would cost you something - maybe even a hand or a thumb - but you would live.

That is how I think of monuments, now.  Databases as monuments create gate keeping caretakers.  Those people are not malicious in their conservation of power.  They are not incompetents maintaining job security.

They are an organization's "lizard brain" (in a good way): the part of an organization that forces it to do anything - bend over backwards or even sacrifice a body part - in order to avoid the devastating effects of losing critical data.

Keeping the Faith

When I started getting good at "regular" TDD, I went through a phase where I believed that traditional testers with no programming skills were no longer needed.  While we may not need as many of them now as we did in the '80s (I hear), I freely admit that I was wrong in believing we didn't need them at all.

Testers were once essential to the software development process, providing vital feedback that ensured a product was at least worth showing a customer.  While we can get a lot of that feedback from automation, now, it is still difficult for a developer to have the same perspective as a real tester.

It's a skill.  As one of my coworkers at the time of this post would say, you have to be able to imagine what an end user might do.  So traditional testers have found a new place in the modern world.  One in which they are extremely effective; maybe even more so than before people did a lot of automation, because now they aren't stuck doing as many menial tasks.

Similarly, as we work to build a new practice of test-driven development in the database world, we should remember the vital function that the traditional DBA has filled.  That role and the perspective that comes with it should not be lost.

Part of learning how to create and test-drive classes of databases will be providing views into those structures that allow traditional DBAs to contribute, not to satisfy some political requirement, but to harvest the valuable knowledge and experience in their heads.

Do We Need Architects?

I'm still not certain that we need the role of architect but I'm coming around to support the need for the activity of architecture.

The kind of long-range planning an architect is asked to do, which so readily turns into a long-term commitment, triggers a sort of allergic reaction in most agile software developers these days.  I know it did for me.  It kind of still does.

However, I've found a way around that with value-stream-oriented architecture.  At least, I found a way that works for me.  Using a tool like VSOA, you can avoid creating the kinds of plans that become commitments or that interfere with implementation-decisions better made by individual  teams.

So I'm coming around to accepting the value of someone doing architecture.  The next question is should it always be the same someone?  Put another way: should there be "an" architect or should architecture be another kind of activity in which an agile team engages?  I'd like to warn you right now that I don't have the answer.

Pros

The most obvious argument on the pro side of the question is that if you have a designated architect, architecture will get done.  Architecture, while a valuable task under certain circumstances, is not usually a task critical to the completion of any given story in an agile environment.  If you don't have someone dedicating some portion of their time to it, then it is likely to fall by the wayside.  The effects of not doing long-term planning an coordination are felt much later and difficult to trace back to a lack of architectural attention.

Designating an individual as a team's or enterprise's architect ensures that time will be spent on the task.  Most agile teams have designated product owners for precisely this reason: someone needs to be thinking about the next steps.  In fact, my friend Mike Brown likes to refer to architects as product owners of technical value.

Just like a product owner, this also means that if coordination must be done with other sites or organizations, you already know who is going to head out there to do that.  While that may be distracting for the architect, it allows the rest of his team/enterprise to focus on creating valuable products.

It is also possible that concentrating the architectural tasks into one person allows them to gain a perspective on the whole that they otherwise would not have.  That perspective would then allow them to create more coherent visions for future development, better interact with product ownership, and better drive developers toward that vision.

Cons

The most obvious argument against having a designated architect is that putting someone in that kind of position is tantamount to handing them a bunch of dynamite and they might take themselves and their teams out just as easily as they could blast out a quarry.

It's to true to ignore.  In some respects, I wonder if the notion that someone who is interested in politics ought to be banned from politics also applies to architecture.  Most people who want the power of being an architect will not use it to create value.  Few architects today are architects because they wanted the influence.

The fact that an architect can be like a kind of product owner carries both edges of the product owner sword.  Sure, only one person has to travel but that means that the one person responsible for those kinds of decisions and that kind of learning is often not available for questioning.  Sure, they might get a better perspective than they otherwise may have had but it is still a single perspective and thus more vulnerable to confirmation bias.

Widening Our Options

In Decisive, it is said that dichotomies are a decision-making smell.  I've observed that for myself several times since having read the book.  So I'm going to add one more option.

The team that I was working on went through similar gyrations with product owners (POs) for several years.  These issues were exacerbated by the POs' very aggressive travel schedule.  What we settled on was a blend of the "have a PO" and "share PO responsibilities throughout the team" options.

We have product owners and they do own product design but they do not make all of the product design decisions.  They are responsible for the product design decisions and, for that reason, they can countermand one they don't like but we don't wait on our thumbs for all product innovation to flow from the product owner as someone preaching a very strict, very academic version of scrum might suggest.

Instead, the team as a whole contributes to product design.  The product owners guide that effort with the knowledge and perspective they have gained and they set direction by making big decisions and performing long-range planning.

Sound familiar?

So the other option I would like to put forth is that you can have a designated architect who is responsible for making sure that long-range technical planning is done and teams are coordinated and you can have architectural work being shared throughout a team or enterprise by all of the developers.

No Answer

I stand by my earlier statement that I have no answer to this question.  For one thing, I didn't have an answer when I started writing this entry.  For another, I'm not sure I do even now.  Certainly, the idea of decoupling ownership of architectural issues from execution of architectural tasks is an interesting one.  The success my current team has had by employing that strategy with product ownership makes it a tempting one.

So tempting I'm going to try it but I still don't have an answer.  Not yet.

A Solution to the Knockout Game

You can't stop this "knockout game" with words.  At least, you can't do it with words alone and not with kind or reasoned words of any kind.  It must be met with violence.  First, violence of the mind.  Then violence of the body.  Here's the plan.

Step 1: Dehumanize
The first thing we need to do is start denigrating the people who play this game.  We need a national media campaign to paint them as the animals they are.  These aren't young punks trying to deal with surplus adrenaline by picking a fight with another male of fighting age.  These are worthless little pukes picking on women and the elderly.

There should be nowhere for them to run from the message.  Nowhere to hide.  Everywhere they go, there should be billboards, marquees, and television advertisements shaming them and reinforcing one message: you are nothing - not even the lowest level of person - if you play this game against a woman, a child, or an old person.

The goal is not just to demonize these people in the eyes of others but to dehumanize them in their own mind.  People who play the knockout game in its present day form should be allowed to ego nor any measure of self esteem.  They need to be put in a place of utter despair.  They need to be willing to do anything - anything at all - in order to prove that they are not animals but men.

Step 2: Retaliate
Since kids - and it appears most people in general - are pretty stupid and easily swayed by the media, step 1 should turn their behavior a little.  Some of them will probably still choose the wrong targets but, hopefully, the culture of shame and humiliation will rob them of their will to live.  I'd like to see less kids killing themselves because they are gay and more killing themselves because they played the knockout game.

Some, however, will still want to play the game and will feel they have something to prove.  They will attack other men.  This is where the hard work starts.  I know it sounds crazy but, when they attack us because we are men of fighting age, we have to actually be men.

People who assault others on the street for no apparent reason must be dealt with.  That does't mean the cops get called.  That doesn't mean a gun gets pulled.  It also doesn't mean a single punch to the face or a little smacking around.

We need anchormen frozen with the chill of what they see.  We need mothers on the news collapsing to their knees and weeping when they discover what their child did and what happened to them as a result.  We need follow up stories months after attacks about the occasional knockout game player beating the odds and being transferred from the hospital into the county jail to await trial.

We need to teach the people playing this game that they still have something to lose and that the pain of playing the game is far greater than the pain of whatever fleeting angst they might be experiencing.

Step 3: Reprogram
The final step is to start teaching children to police themselves.  Juvenile humans are vicious, especially to other children.  We simply need to point that viciousness in another direction.  We need to re-align the culture of our schools so that people who attack random strangers are seen as different.  There's nothing that brings down the righteous fury of the masses like being different.

When I was a kid, schools made a practice of singling out children with certain attributes.  If you were exceptionally bright or artistically talented, there were structures in place to make sure everyone knew you were different.  This led to years of torture and many children tried to conform in order to fit in to the crowd.  Not me, but a lot of kids.

While this property of schools has traditionally been used for evil - as nothing more than a mechanism for small people to manipulate other small people into punishing the exceptional - I believe it can be used for good.  By the end of phase 2, acts of random violence should be cut down pretty close to the minimum.  Only the really broken kids would still be conducting them - the real "seed of evil" types.

The final stage is to identify these people early and sick the masses on them while they are young.  The pressure to fit in should drive many of them to try and conform which, in this case, means not playing the knockout game.  Those few who still engage in the behavior will serve as reminders of what happens when you threaten the fabric of society.  Better still: it's the truly evil kids being sacrificed to this cause, which is no real sacrifice at all.

If this plan is executed, the knockout game will cease to be a game.  It will lose its name.  It will instead be called what it really is: random acts of violence by misguided and defective teens.

Of course, it never will, so none of this really matters but I enjoyed writing it and, if even one person scolds a knockout game player as they run by because of this post, then I've made a difference.

I'll Be Speaking At DAMA SoCal in February

This is just a quick announcement that I will be giving the same talk I mentioned earlier again at DAMA SoCal on February 24th, 2014.

Click here for more details.

Robbery on the Road as a Sign of the Times

My father in aw bravely thwarted a robbery a while back.

It was taking place on the sidewalk.  A large man and a small man were struggling over a backpack.  My father in law got out of his car and asked them what was happening.

The large man said "I'm trying to take his stuff but he won't let me!"

My in law began calling the police at which time the large man trundled off into the city.

I think this is emblematic of the thinking in America.  Nobody is concerned with what is fair or right anymore.  Everybody is concerned with what they can get regardless of how they get it.

People like that large man have been robbing people like us for ever.  It's the kind of robbery we all know and it's what people imagine when you say "robbery."  But the thinking which underlies that behavior is flung a lot further and a lot wider than just hoodlums and ruffians.

You see this thinking in many welfare recipients, both individual and corporate.  I know more than a few people on "disability" who could easily work if they so desired.  Instead, because their easiest path is to steal from the taxpayer, they lounge around in their free apartment, watch their free cable, and eat their free food at my (and possibly your) expense.

Likewise, rather than earn a profit by offering a valuable product that people are willing to buy, corporations are becoming increasingly dependent on government contracts.  Why build something useful when you can just suckle at the teat of the U.S. government?  Why cover your risks when you know the government will just bail you out because you're too big to fail?  Why learn how to get people from point A to point B at a reasonable price when you know that the government will keep subsidizing your airline?

Another great example is the "false fee" industry.  In business schools, some have taught that adding fees onto an agreed-upon account is a "growth industry" and will represent some ridiculous amount of money (tens of billions per year) by the end of the decade.  Phone companies are notoriously bad about this.  They get you to agree to one price, then they tack on a bunch of extra fees and "taxes," some of which are legitimate and many of which are just plain old fashioned fraud.

I've also personally experienced a credit union stealing my money by charging me a fee for not using my account.  When I confronted them about this, they looked at me like I was crazy and said "We're just trying to fee down all the inactive accounts so we can close them."

In every one of these cases, the thief must justify his actions and his own existence; at least to him- or herself.  The welfare consumer must argue that they are "down on their luck" or otherwise unable to take care of themselves and that justifies, in their minds, the use of money taken at gunpoint.  The corporate welfare recipient is probably every bit as unenlightened in its thinking... it's probably something like "I've got a lot of jobs and shareholders to which I must answer so I've gotta do what I've gotta do."

People who perpetrate false fees also must justify their actions and probably say something like "I'm just trying to make a buck" to themselves and others who confront them about their nefarious acts.  Ten years ago, the credit union manager told me straight to my face, as though I was the one in the wrong, that they were just trying to take my money.

What's the big deal, right?

All of these people are about the same as our thug on the street.  They all want more than they have or could gain on their own by fair means.  They all want to take something from someone else; something that doesn't belong to them.  They all believe that they are in the right for "just trying to get ahead."

I've got no idea what to do about it.  It seems like the only corrective course of action is to radically change the makeup of the people who are deciding how the country I live in works.  There are lots of ways to do that.

You can increase the population of smart and thinking people, possibly by education and definitely by reproduction.  That takes time and a lot of energy.  Who wants to wait fifty years for a problem like this to get solved?

You can decrease the population of stupid or unthinking people but it's difficult and requires unethical activities.  A pogrom against the stupid and evil has one major roadblock: there are whole lot of stupid and evil people out there and none of them want to be pogrommed.

An increasingly viable impulse, for me, is exodus.  Instead of making more good people and instead of rounding up the bad ones,  just round up the few rational people left in this country and leave to make a new one.

That's probably very nearly as impractical as restoring freedom and reason to America but it seems unlikely that it would be any harder and it's probably at least a little bit easier.