A golden ratio of data management

Until now we learned that Unidatab is a complicated engine that makes the act of looking for information slower and clumsier.

Our last query for instance involved isolating a set of values sharing the same type from a much larger set (symbols). This kind of query is slow when the sets are numerous. No comparison with a classical db. But there, each individual value in its column would be reentered, growing the database to the google-size. The operating time is usually shortened by the amount of data, and the work of developers can be resumed by finding a good ratio between both quantities.

In Unidatab this same problem arises at a different level and forces us to investigate the way we describe things. Jean-Claude Gardin – among others – was well aware of these problems and stated somewhere that there must be a fine limit between describing a thing as an object and creating an object to denote something comfortably enough for our needs, but then becoming cheaty to the thing.

The extrinsic-intrinsic dichotomy is helpful as long as we keep conscious of its imperfection, a trait that in my opinion will be shared by all binary perspectives – though we seem to need one of them. So we can find means of speeding up Unidatab by adding extrinsic properties to our object. How much are we wanting to “cheat” this way will decide how close we’ll come to some hypothetical golden ratio in data management.

For SQL can be fast as long as we deal with identification numbers (row ids), and on a smaller scale with integers. If we add a special symbol with a unique chosen value (say 0 or whatever) and type ‘description’ (which denotes here only one property) – or even ‘has description’ (to distinguish clearly from the type of the original property) – to all the records that do contain this type of property, then our last request can become:

? select distinct find_record(lnk) from unid_links where a = -145 limit 2
find_record(lnk)        :

provided -145 is the id of our new symbol, and this is much quicker. We can even stop using views in this case and query directly the table, and as our property is unique in its type for each record, we can also forget about distinctness:

? select find_record(i) from unid_node where a = -145 limit 2
find_record(i)        :

and this is as fast as SQL can be. What happened to our object in the while? It simply ceased to describe the thing alone but is now telling something about the way we describe it.

[2018 edit]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s