Wednesday, December 04, 2013

My Journey to a Class of Databases

If you know anything about me or you read my article on InformIT, Ten Tips for Constructing an Agile Database Development Environment that Works, you know that I think the foundation of database TDD lies in creating a class of database.  That means transferring focus from the design of individual databases onto something that makes databases.  You want the ability to create or upgrade as many instances as you like and know that they all have exactly the same design.

Exactly the same design.

There are many possible ways to implement a concept like that, though.  When I began developing these concepts nearly a decade ago, before I even knew that I wanted a class of databases, I was just focused on controlling database creation.  I experimented with a lot of different kinds of infrastructure and a bunch of different patterns of database growth.

Since I started being a professional software developer, I've tried a bunch of different ways to control how databases grow and transform.  I think a lot of people have had similar thoughts and various times in their lives.  These are the ways and times I thought about these problems.

The Fool's Errand (pre-2005)

The most naive solution is the idea that you can specify what you want the design to be right now and have some tool that will update an existing database to have the new design.  Sometimes, it is a tool that compares two databases.  Sometimes, it is a diagramming tool that will inspect a database and figure out how to make it comply with a drawing.

magic will transmit design changes!
The problem is that this doesn't work.  It doesn't work for the same reason that you can't unscramble an egg.  There is no way for a software system to look at the current design, look at a new design, and figure out how to get from point A to point B.

At least, it's not possible to do that every time and with current technology.  Maybe, one day when we have computer systems that can infer intent, it will be possible.  Right now, however, that's too complex a task for a computer.

The Installer Fallacy (2005-2006)

Another way of thinking about the problem is the way we imagine installers.  Databases have components.  Components have dependencies.  You ask the installer to make sure the features you want are there and it ensures the dependencies are satisfied.

The problem is that there is always a meltdown.  In this case, I'm using that term a little less figuratively than usual.  In a healthy database design, things are changing.  Tables are splitting and recombining into newer, better shapes all the time.

The features all melt together more quickly than you could imagine.  Pretty soon, it's difficult to tell why you are creating separate features and components at all.  Eventually, all the components blend together and you wonder why you ever divided components in the first place.

Rise of the Versions (2006-2008)

After about my third database "feature" that depended on exactly one feature, which in turn depended on only one feature, I started to get the message.  I realized that the forces in the database world are telling us to organize around time, rather than around features.

It turns out that there is usually at least one database instance that as an extremely linear path of transformation, and it happens to be the absolute most important kind of database there is: a source of record database in production.

Production databases tend to metamorphose over time in a series of discrete transitions from one design to another.  At the same time, production databases are the most indispensable and long-lived databases of all.

Everything else (e.g.: test databases or development databases) tends to have a little more flexibility.  At the very least, nothing else has less flexibility.  So why shouldn't the most important and least flexible kind of database define how all databases of a particular kind are built.

Have a Little Class (2008-present)

When I started formulating these thoughts into something that I could start evangelizing, I realized there was more to this than just regulating the flow of design changes from a development environment out into a production environment.  That's an important feature but it's just an implementation detail of a much more critical shift in mindset.

a path of confidence
What really matters is having uniformity of design between all the different database instances filling the same role.  If you have that, tests executed against one instance allow you to make predictions about how another instance will behave.

That mechanism - that way of thinking - serves as a critical underpinning for test-driven development in the database world.

Tuesday, December 03, 2013

A Class of Databases for Non-Middle-Tier-Developers?

As I previously mentioned, you can use the sample code for my book as a starting point to establish a class of databases and you can also inexpensively write one for yourself.

However, if you don't want to do any non-SQL coding and you don't want to use the infrastructure I built and made freely available, it might get a little stickier.  At least it would for me because the test frameworks and expressiveness available for SQL-only solutions is kind of weak.

One option is to write something yourself.  If you do that, I'd love to hear about it and I bet a lot of other people would too.

Another option is to wait for the release of DataClass 3.  Maybe you could even participate in its design to make sure it meets your needs.  Right now, it is being made by a software developer for software developers.

If you need something now, though, you'll need to invest in developing something yourself or try downloading and compiling the sample code to see if it will work for you out of the box, which it easily could do.

Monday, December 02, 2013

Pivoting a Little on DataClass 3

I've found a smaller deliverable for DataClass 3 and will be releasing that, first.  As it is a side project, I cannot commit to a date but it shouldn't be too long.  I don't want to get in to the details but, basically, I'm going to rewrite the examples for my book incrementally.

The first deliverable should cover about a third of the book.  From there, I can add support for each practice independently.  After that, I will add the higher-level language and refine each of the examples as I go.

The initial output will still be C# 4.0 and the initial database platform will be still be SQL Server 2012.  Then I'll widen support from that point as demanded by my readers & followers.

Friday, November 29, 2013

Establishing a Class of Databases

A lot of people ask me about the infrastructure required to establish a class of databases.  In my opinion, that's not a lot of the work.  It takes a day or so here or there to write just what you need, just when you need it.

Nevertheless, I would like to remind everyone that the sample code for my book also contains sample infrastructure.  Feel free to take that as a starting point and go forward.  If you're not a .NET person, it could pretty easily be translated into Java.

Wednesday, November 27, 2013

Reminder: VSOA on DZone

I know it's lame but I don't care.  If you liked my article on value-stream-oriented architecture, please remember to go vote for it on DZone.  If you didn't like it... I don't know... go to hell or something?