Universal Programmer



Innovation in Database Technology

From a message to the NoSql group

Recently there has been some discussion about the meaning of the NoSQL movement, or at least it’s name.

For a group to thrive it should have a strong and clear idea of what it’s desires are. Breadth of applicability needs to be balanced with togetherness of aims, in order to reach a variety of people, and achieve something great.

The notion of “no sql” is strong, which creates togetherness, and is still broad enough to leave room for many to act on it.

Some have taken opposition to the name thinking that it demeans the concepts that are embodied in a sql database. I don’t believe this is truly why the term “no sql” rings so true to the issue. I believe that this confusion has largely been a result of the early days of the movement where it was only a small portion of the sql problem that finally prompted action and speaking out, specifically, the issue of scalability for massive loads, and the realization that moving away from sql could provide almost unexpected solutions.

In reality, the issue at play which lets “no sql” ring so true to us is that standardization often comes at the expense of innovation.

That, is why we have really moved away from sql, it is not for any specific approach to scalability or data storage, but rather just the ability to free ourselves from the standardized ideas encapsulated in the standard query language.

I’m going to say it again, moving away from sql allows us to innovate. I’m sure no one will have ill feeling towards the notion of innovation, and standardization is almost the exact opposite, it is the crystallization of previous innovation, so of course it would be what we stand against.

We do not necessarily stand against any specific idea encapsulated by the sql standardization, rather we just choose to open ourselves up to investigating the elements of the system for the sake of making design decisions which provide innovative solutions.

As more exploration into the options takes place we see different approaches to achieving innovative solutions. Currently the movement has been largely defined as “scalability, non-relational, schema-less, base etc” which has had it’s focus solely on scalability, and to worse detriment, often closed itself off to specific means of creating valuable solutions for scalability. The wikipedia page currently reads “promotes non-relational data stores that do not need a fixed schema, and that usually avoid join operations”, we need to increase the breadth of applicability to developers and give a clearer picture of this.

The issue of scalability is only one issue in the quest that starts with freeing ourselves from standardization. There are many unseen innovations to take place by freeing ourselves from these standardized tools, and I say “freeing” because that is really what it is, we are currently trapped with the tools that we have spent so many years putting our effort into, at the expense of having little to go to other than those tools.

As an application developer, and a framework developer, I have felt the negative impact of the sql ideas and interfaces on the ability to create efficient *and* beautiful database application code, whether that be for massive scale operation, or for embedded applications.

Creating scalable solutions is only a subset of the story, and one which will also be fueled by other tools which eschew the standardized concepts, even if those tools have little regard for scalability.

I believe we need to be truer to the name that we have felt fits the situation, “-no sql- is about re-opening the gates to innovation in databases” this is the definition we need to bind ourselves on, and make clear to those we want to attract.


Trackbacks & Pingbacks

Comments

  1. * Timothy says:

    I would almost have to disagree with this statement, even though I am in love with Redis, Mongo and others. SQL and relational databases are built upon a solid foundation of science and mathematics concepts which make them leaps and bounds better than what our current innovation could yield in the next couple years. The move to NoSQL data stores and map/reduce is based upon the challenges with the current computing model. Since Moore’s Law it not allowing us to scale vertically with the same compute power that scaling horizontally provides (which brings the added benefits of redundancy, load balancing, and hardware failover), the entire computing industry is looking at other products that are designed with multi-computer concurrency at the core of the design.

    SQL will be relevant far into the future if there was a software team that would undertake to rewrite an SQL engine from the ground up with the same horizontally scaling mentality that Basho is using when writing Riak.

    There is little incentive to abandon SQL engines when we can already use them as key-value or document stores (albeit with a different methodology than what is currently considered as de-normalization), except for the ease of horizontal scalability that is not currently provided by any of the vendors.

    | Reply Posted 5 years, 5 months ago
  2. * wllm says:

    Great post!

    I have also argued that we shouldn’t standardize things that, in some cases, haven’t even been developed yet. Developers need room to try things out- failing in some cases- to figure out what their users actually need. Hopefully the open source community will step up with adapters and exploratory common interfaces to inform the standards process, if things ever come to that.

    I also would like to hear more about the advantages of schema-less DB development. Basic queries can also be *much* more efficient and responsive with a NoSQL DB, to the point that it sometimes makes more sense to drop application artifacts, like logs, in the higher-availability DB instead of the filesystem.

    Although I appreciate the potential scalability afforded by a join-free DB, I agree that these emergent technologies shouldn’t be dismissed as neutered DBs for massive apps/sites. That’s turning your back on some enormous value propositions that will appeal to practically every web developer or developers from other fields.

    Thanks.
    ,Wil

    | Reply Posted 5 years, 5 months ago


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: