Dynamic Containers™: The Key To Superior Performance
Introduction: Building mission critical systems using object oriented technology is done in an environment of high pressure and extreme urgency. Combine this with the architectural choices available in the ODBMS arena and the pressures in evaluating and selecting an appropriate object persistence solution and we have a perfect opportunity to either miss a key advantage or succumb to uncertainty and confusion.
In the object oriented design of software applications, architecture is one of the issues at the forefront. That same focus on good architecture has been brought to Objectivity/DB in order to lay a foundation that will enable software developers to build solutions that will stand the test of time. One basic commonality amongst all ODBMS is that they all store and retrieve objects. However, we strongly suggest that how a given ODBMS achieves that basic goal of storage is the key to choosing an ODBMS. Relational Database Systems (RDBMS) have had decades in which to refine and perfect their ways of storing rows and columns of data. B-Trees and indexing methods have become a staple in almost all RDBMS. In the ODBMS world, there are several competing ideas on how storage of objects should be achieved and we will show that Objectivity/DB has an excellent solution to that very crucial question.
This document will give the background to Objectivity/DB storage design decisions and cover key aspects of Objectivity/DB architecture. We will specifically deal with the powerful concept of Objectivity/DB containers and how they may be used to achieve superior designs. We will also show that traditional aspects of software development such as data structures can be used as well as current object oriented development methods.
BACKGROUND: Objectivity introduced containers into the storage hierarchy for four main reasons: to increase system throughput; to ease the migration path from file-based storage to an ODBMS; to make it easier to use standard off-line (hierarchical) mass storage devices; and to make it easier to use standard operating system security features. This White Paper is primarily about increasing system throughput, but the ability to treat a container in a way that is analogous to a conventional file is very useful in some applications, particularly in workgroup environments and in large scale data acquisition or warehousing applications.
Many alternative logical and physical storage models were considered and discarded during the requirements analysis and systems design phase. These ranged from the use of the “Black Hole” approach favored in traditional DBMSs to the explicit declaration of both logical and physical storage structures, either in a data diction-ary or at runtime. In the traditional database approach there is a single address space and the application programmer has little or no control over the physical place-ment of data. This task is delegated to a Database Administrator who uses projected and actual system statistics to map the logical structures, such as records or rows, indices and hash tables onto the appropri-ate physical devices and storage structures (generally files or partitions, but sometimes even down to the cylinder/track level). In a conventional file environment, almost universally favored in the engineering, scientific and PC applications world, the structure of the data is often hidden in the applications and the essential metadata components of a DBMS are missing. However, early ODBMSs often used structured files, sometimes capitalizing on the performance of virtual memory mapped files, because UNIX programmers and users were comfortable with the filesystem paradigm. Some of these ODBMSs have physical object identifiers (OIDs) and this can make space reclama-tion, cross database references and object migration (as the schemas change) very cumbersome. They also tend to scale very badly as the volume of data and the amount of concurrent access increases.
The evolution of DBMSs is interesting in that successive technologies did not always capture all of the good features of the previous ones. Many users tend to be more comfortable with files that they can see or own than with information locked in a closed DBMS. The CODASYL (network) DBMS standard spawned products that were extremely efficient at navigating through linked lists of related data. The RDBMS world generally ignored these factors and so RDBMSs still offer relatively crude data clustering features and are clumsy and inefficient at performing complex “JOIN” operations. Objectivity tried to meet the demands of object-oriented applications without losing the advantages that each of the previous technologies, including RDBMSs, had introduced. So, although conventional file structures, tuple engines, dataflow techniques and many other alternatives were considered, Objectivity settled on a hierarchical logical storage model that simplifies database administration tasks and meets the four main goals outlined above.
The physical model reflects the logical model. It provides a means to safely and efficiently store and manipulate both very small and very large objects, large collections of objects and extremely complex networks of related objects. There is a single logical address space supported by a hierarchically structured physical space. The application programmer has some control over the physical space. For instance, a programmer may create a new database and specify the hostname and file pathname for the initial database file. Similarly, any object may be initially or dynamically clustered near any other object (or an existing group of objects). However, like its counterparts in earlier DBMSs, Objectivity/DB’s storage manager automatically manages critical tasks, such as space reclamation in the cache and filesystem during and across transactions.
This section also available in Adobe Acrobat PDF format 