Databases – often underestimated but essential

From Sequential Files to Databases

I still remember my early days as a software developer with COBOL. We all have been dealing with sequential or index-sequential files to store data of our applications, or read the necessary settings during application startup.

In the meantime, no one would use something else than a database. Databases became affordable and do a lot more than just storing data in tables and fields. The allow to store procedures and triggers to automatically execute steps that needs to be embedded into the application in the old days.

Database Management: Simple or Not?

So far, so good. Managing databases seems to be simple – boring (remembering my lectures in database normalization) but easy to handle. You are right and wrong at the same time.

Database and Application Design Go Hand in Hand

First of all, the design of databases and applications have to go hand in hand. Databases provide the lifeblood for applications, as they have no data that they can mingle and transform.

The Difference Between Design and Deployment

Second, applications are changed by the dev teams only, but databases can also be modified in operations. Database design and deployment are separate. A design is solely the blueprint for an instantiation of a database, but due to performance reasons or outages you might need to modify a specific instance, for example adding an additional index table, or defining a new stored procedure to fix performance problems. In the world of Geology a database would be a continent, many databases are the continents, shaped by the tectonic drift, but all come from a single, consolidated chunk of land. If you think about the different instances of single database scheme, you might be surprised by the differences, small and big, between development, test, staging and production instances. Identifying issues and chasing bugs can turn into a challenge, usually indicated by sentences like “it works on my database …”

Database Versioning Is Different from Code

Third, databases design and its versioning are different to code, although the commands to create a database instance are usually coded in ASCII. But there are many scenarios where the order of executing the creation commands doesn’t really matter – but it matters in application code. The traditional approaches to version control for software work partially for database schemes, but managing a single source of truth to be able to re-instantiate a database is a lot harder. Any change in operations has to be reported back to development to ensure, that the issue is not occurring in the next release again.

Releasing Application and Database Versions Together

Finally, you need a system that allows to release and deploy the correct versions of application AND database scheme to work smoothly. Separating the two is yet another source of problems. In case you offer your applications on different underlying databases it is even more essential, as database providers often have little but important differences in the command set.

Avoiding Database Drift with Liquibase Secure

In case the issues describe above have kept you up in the past (or still do) we highly recommend to have a closer look at Liquibase Secure. In addition to the downstream support also provided by the Community edition, Liquibase Secure eliminates with its roundtrip options the drift of database schemes automatically and avoids long-term root cause analysis.

Author: Rainer Heinold