Universal Programmer

Innovation in Database Technology

From a message to the NoSql group

Recently there has been some discussion about the meaning of the NoSQL movement, or at least it’s name.

For a group to thrive it should have a strong and clear idea of what it’s desires are. Breadth of applicability needs to be balanced with togetherness of aims, in order to reach a variety of people, and achieve something great.

The notion of “no sql” is strong, which creates togetherness, and is still broad enough to leave room for many to act on it.

Some have taken opposition to the name thinking that it demeans the concepts that are embodied in a sql database. I don’t believe this is truly why the term “no sql” rings so true to the issue. I believe that this confusion has largely been a result of the early days of the movement where it was only a small portion of the sql problem that finally prompted action and speaking out, specifically, the issue of scalability for massive loads, and the realization that moving away from sql could provide almost unexpected solutions.

In reality, the issue at play which lets “no sql” ring so true to us is that standardization often comes at the expense of innovation.

That, is why we have really moved away from sql, it is not for any specific approach to scalability or data storage, but rather just the ability to free ourselves from the standardized ideas encapsulated in the standard query language.

I’m going to say it again, moving away from sql allows us to innovate. I’m sure no one will have ill feeling towards the notion of innovation, and standardization is almost the exact opposite, it is the crystallization of previous innovation, so of course it would be what we stand against.

We do not necessarily stand against any specific idea encapsulated by the sql standardization, rather we just choose to open ourselves up to investigating the elements of the system for the sake of making design decisions which provide innovative solutions.

As more exploration into the options takes place we see different approaches to achieving innovative solutions. Currently the movement has been largely defined as “scalability, non-relational, schema-less, base etc” which has had it’s focus solely on scalability, and to worse detriment, often closed itself off to specific means of creating valuable solutions for scalability. The wikipedia page currently reads “promotes non-relational data stores that do not need a fixed schema, and that usually avoid join operations”, we need to increase the breadth of applicability to developers and give a clearer picture of this.

The issue of scalability is only one issue in the quest that starts with freeing ourselves from standardization. There are many unseen innovations to take place by freeing ourselves from these standardized tools, and I say “freeing” because that is really what it is, we are currently trapped with the tools that we have spent so many years putting our effort into, at the expense of having little to go to other than those tools.

As an application developer, and a framework developer, I have felt the negative impact of the sql ideas and interfaces on the ability to create efficient *and* beautiful database application code, whether that be for massive scale operation, or for embedded applications.

Creating scalable solutions is only a subset of the story, and one which will also be fueled by other tools which eschew the standardized concepts, even if those tools have little regard for scalability.

I believe we need to be truer to the name that we have felt fits the situation, “-no sql- is about re-opening the gates to innovation in databases” this is the definition we need to bind ourselves on, and make clear to those we want to attract.

Javascript Wishlist

import and package keyword functionality. Lets get these implemented.


Direct manipulation over prototypal inheritance.

Functions that can be inherited, and whose function bodies can be manipulated. If you inherit from a function it’s body is not inherited and the resultant object is not a function. The actual function body should be a property of a function, the ‘call’ property, so that the resultant object of an inheritance operation will still contain that property, and could even override it. This way you can create a constructor object that inherits from a parent constructor.

Object definition block syntax. So that we can define an object using a block of code instead of just object literal syntax. Such as:

var tim = new Object() {
   this.myname = "Tim Lind"


var Myconstructor = new Function() {
   this.call = function() {
     // the constructor body
   this.prototype = new Object() {
       var someprivate = "secret"
       this.somemethod = function() {}

Functions that can be called with a user determined context object. The object that will contain the resultant variable of the statement “var hi”.

Custom iteration (maybe also index notation overriding). The ability for a type to determine how it is iterated over. I assume this will also lead to sorting out alot of the messy problems we encounter with iteration, especially with using Object as an associative array, if someone has tried to add functionality to Object (and no there shouldn’t be anything wrong with doing that – currently there is though). We could have an [no enumeration] thing when we set properties, but I think there’s a much better solution. Instead, I’d like to see a Map object be the root of the hierarchy, which Object inherits from.

The Driver pattern as a seemless enabler of Remoting

Any software designer is well aware of the driver pattern, in fact, any computer user is well aware of the driver pattern. It enables functionality, usually the same functionality that is available through some other means as well. When I put it like that, what I’m about to say becomes so obvious to me, I don’t know why I didn’t realize sooner.

The driver pattern is how we can almost effortlessly make functionality available from a remote location; a location remote from where the functionality is “really” running.

Sure, it doesn’t provide a completely effortless way (that’s why I said ‘almost’), because sometimes we’ll still need to make some adjustment to the functionality in order to make it feasible to let the driver do the work at the points we expect it to if the functionality was really running at our location, but most of the time that driver can choose when it will actually make the remote call and let the programmer go unaware of any differences to the design of the functionality.

So what I’ve realized is that the driver pattern is what we want to design so that remoting can be just another driver. Being able to access your api can be as direct as it is from the server side, with almost no effort. You just need to create a different driver (and make sure you were using the driver pattern in the first place).

Simple Subclassing?

When we work with class based languages, focus seems to be placed on the same basics, extending a class with a new class and overriding a method. Each manual we read for different languages gives the same basic mechanism to achieve this, but there seems to be no focus on different ways we could make subclasses, there is little depth about even the obvious internal workings that these basic mechanisms rely on. When working with Javascript and trying to overcome some deficiencies with it’s prototype scheme these internal working become more apparent.

Language authors would have us think that subclassing is a straight forward operation and there is only one way to do it, but I think we could empower ourselves better if we looked for different ways we could do it, and looked for a language that allowed many of those different ways.

An example of the internal issues I’m talking about is whether a method runs as though it is coded in the class it is called on, or on the class it is inherited from. Our language should allow us to specify things like this, but few do. We can use these issues to carve out more “types” of subclassing mechanisms.

Project (or developer) Stagnation

This is a common and understandable problem, easily solved.

As we know software development is driven by requirements and features, and is process of building something to meet the requirements while taking into account the issues, and this process is always motivated by some reason for achieving a feature.

You need to get yourself thinking about the issues and reasons for the decisions you’ve made in your design so far.

Having a simple document which walks you through the desires, the options, the issues and the decisions you’ve made on them so far will be all you need to boot your brain with the details necessary to start executing the development process, not least of which is your personal motivation for achieving a certain feature.

This is similar to the option someone gave above of talking to a friend, your documentation is your friend which includes sticking to the decisions you’ve made so far (important for progress).

Some recommendations have been to start with the small aspects, this is also taken into account here. The walk-through of your design should obvious start with simple premises and build up to more complicated rationale. There should always be a fairly simple motivation (rationale) for implementing a given feature, but often getting to that feature means booting yourself with the decisions that build up to that one…it’s really just a matter of knowing the motivation for any unit of work, and that is an important aspect of system design that has not been well supported by the tools our industry has created for themselves.


Get every new post delivered to your Inbox.