M4S Home Page

MetaKit - FAQ #1

The following list contains questions which have been asked about MetaKit and other issues which did not fit anywhere else in the documentation: [ Back to the MetaKit Home Page ]

How reliable is the MetaKit software?

The library is under development, and is currently (March 1996) distributed as a pre-release. The beta 3 version (m4kit10b, of Mar 12, 1996) is quite solid, this is definitely not a "toy" release. The software has undergone a substantial amount of testing, with more to come. Reference counting and automatic object cleanup are fully functional, there are no known memory leaks. Regression tests are performed after every major addition / change to ensure that no new problems are introduced. Corruption of data files has never been detected, probably also due to the inherently fail-safe design of MetaKit.

How complete is MetaKit?

Some of the functions, such as rollback and searching, have not been completed in this pre-release version (March 1996). Further changes and extensions to the library API will probably be made once the library is used in more production-level / commercial applications.

Is MetaKit an object oriented database?

Yes - you can store structured data in a MetaKit data file. No - this is not identical to persistent C++ data structures. The main distinction here, is that MetaKit manipulates run-time data structures (views, rows, properties), which have no exact counterpart in C++ (which is a statically typed language). Furthermore, MetaKit is vector-oriented - you must use numeric indexes instead of pointers if you which to store object relationships (or you can use the key-based model of relational databases). As for full scale object storage, no - there is no support for encapsulation, polymorphism, inheritance, and similar concepts. This version is a data- (and structure-) oriented package.

How does MetaKit compare to relational databases?

The data model supported by MetaKit includes relational tables (as views of rows of properties), but nested views allow more strictly structured data without having to work with common key prefixes and table joins. When compared to the expressive power of SQL, MetaKit clearly has less functionality. The data model of MetaKit is closer to object-oriented concepts such as containers / collections / iterators.

What data types are supported by MetaKit?

In the current version, the following data types are supported: integer (long), floating-point (float), and ASCII strings are supported. There are no separate data types for 1- and 2-byte integers, because MetaKit will automatically adjust the stored size as needed (this is not yet implemented in the beta 2 pre-release). Strings can be of any length, and are based on the underlying implementation of the CString class in MFC. Version 1.0 of the library does not accept zero bytes inside strings.

Can you describe the fail-safe storage and recovery features?

As a storage manager, MetaKit is extremely robust. Changes to stored information are only saved to file when explicitly committed, and will never overwrite the current version of the data. This means that - until the file header is replaced as the very last step - nothing will affect the currently stored information. It also means that the file size may need to grow to twice the amount of all stored data (or less if only a few changes are made). Free space is automatically reclaimed, using a form of garbage collection which can recover from partially completed commits. A benefit of this "stable storage" algorithm is that during development - when crashes can occur more frequently - no stored data is lost or corrupted by MetaKit.

On which platforms is MetaKit available?

The first version of MetaKit relies on the CFile, CString, and several C...Array classes of the Microsoft Foundation Classes (MFC) as underlying framework. By building on this popular "industrial-grade" C++ application framework, a significant amount of development and testing effort has been eliminated. Currently, this framework is supported by Microsoft on all flavors of Windows and MS-DOS. There are ports of MFC to Macintosh and Unix platforms, but the MetaKit library has not been tested on them (nor on the RISC versions of Windows NT). A future version of the MetaKit library will be based on the Standard Template Library (STL), making it portable to any platform which supports this new standard (including Macintosh, and several Unix versions).

Can I use MetaKit in combination with other libraries?

Yes - All globally visible names of MetaKit are chosen to avoid conflicts with any other package. The small code footprint of the library makes it well suited as an add-on to store some structured data on file or to send structured data over a communications channel. Even information which could be stored in INI files or in the Windows registry may be easier to access and manage using MetaKit.

What are the primary application areas for MetaKit?

The first version of this library is primarily intended for those areas where INI files and TEXT files are too limited, too inefficient, or too cumbersome, and where the overhead and complexity of full scale database packages are considered excessive. This includes single-file executables such as indexing software, PIMs, disk / file / system utilities, and all applications which should be easy to deploy and should have minimal requirements for installation. Since the size of data files is not limited, and since access to individual items can be fast regardless of total file size, even applications such as Web-indexing, CD-ROM catalogs, and medium-sized financial applications can benefit from using MetaKit. The robustness of data storage and the fact that the library can be linked statically to an application can make MetaKit transparent to end-users - they will never have to repair damaged data files and they cannot disrupt operation by removing some "required component".

Does MetaKit require C++ templates?

No, this version of MetaKit does not use templates. Only a small number of classes in the library would benefit from using templates. Nevertheless, to simplify the source and to benefit from strong type-checking, templates are planned for one of the future versions. This will probably coincide with the development of an STL-based version of MetaKit.

Does MetaKit require exception handling?

Yes, although exceptions are rarely thrown from the MetaKit code itself. In this version, whatever MFC supplies as mechanism is also used by the library. For Windows 16-bit and MS-DOS versions, this implies that the THROW/CATCH macros based on setjmp/longjmp of MFC 2.5 are used. Because of the automatic clean-up of stack-based objects, standard C++ exception handling will be adopted in a future version of MetaKit to make sure no memory leaks are introduced by run-time exceptions.

What are the implementation limits?

The first version of MetaKit has some limitations in the Windows 16-bit and MS-DOS large-model versions because the size of several internal data structures is limited to 32 Kb (signed ints). The most restrictive limitation is that all strings of a single property of a single view must fit within 32 Kb. If you think about this for a while, you will note that data is stored column-wise in MetaKit. The library API completely hides this fact in normal use. The implications of this implementation are extremely far-reaching, and have the just-mentioned limitation as consequence. This limitation is temporary: future revisions will allow virtually unlimited view sizes, while maintaining fast on-demand loading. The 32-bit implementations have no limits on size, but do suffer from performance degradation with large views.

What are the origins of MetaKit?

The MetaKit library is the result of several years of experience with commercial software development. These concepts have been used in commercial applications which manipulate very large data-sets (millions of mostly numeric results in scientific computations). The performance achieved have been several orders of magnitude higher than traditional database-oriented approaches used in similar products. MetaKit is the result of "starting over", to build a general-purpose engine which implements these concepts for other application areas. Version 1.0 is obviously only the beginning of this second-generation software package.

I desperately need feature XXX, will MetaKit support this?

This library is being developed with a large commitment in time and effort by a single person. One guy cannot possibly pretend to offer marketing, technical, and sales-support on a massive scale. On the other hand, when you ask a question, you do not have to wade through a chain of support technicians who do not have the time to figure out how everything works, let alone how to fix problems that may occur. The bottom line is: if you want to change MetaKit to suit your purposes - GET IN TOUCH. Simple issues are simple to deal with - and probably useful for others as well. More complex issues - well... perhaps it's so important that it will be considered right away, otherwise there might be other ways of achieving your goals. Just remember, the MetaKit programmer / designer / architect / whatever is only an email or fax away from you - and listening.

Why is MetaKit currently only available for use with MFC?

There are several reasons: MFC is widely used, extremely robust, available for all variations of Windows and for MS-DOS, and works well with the equally robust Visual C++ product range. The dependency of MetaKit is very limited, but development was substantially speeded up by the string, file I/O, and collection classes of MFC. There is no fundamental dependency, the predecessor of MetaKit has been used in products for Macintosh and Windows. Porting is only a matter of time and priorities, and will probably be done when MetaKit is adjusted to work with the Standard Template Library (STL).

Why should I register my copy of MetaKit?

This one is simple: because use of this product beyond the initial 30-day trial period is illegal. If you only use MetaKit for personal or non-profit use, you can obtain a registered version. If you intend to use this library in a business setting (i.e. if you are paid for the software development you do) or if you intend to sell products for profit, including shareware, you must purchase the source code version - which includes the right to use MetaKit commercially. If you don't register, this shareware product will quickly cease to exist.

What will the next releases of MetaKit be like?

Future updates to this library will be guided by several priorities: sufficient functionality for its intended use, simple to understand and to work with, time and space efficiency, removing implementation limits, new features - roughly in that order.

Why does MetaKit use it's own format for the data files?

The design and implementation of MetaKit is a radical departure from current database and object persistence libraries. The on-demand loading and fail-safe commit/rollback features pose some unique requirements on the data structure on file. Future plans such as cross-platform portability and memory-mapped file access further complicate this situation. Existing packages have too little to offer to simplify these issues, and generally consist of a substantial amount of code. Since the source code of this library is available to anyone, this data format can hardly be called "proprietary". Anyone can create special purpose utilities to access the data in these files, or extend the generic DUMP.EXE which is included in source code in the shareware version.

How does MetaKit deal with multi-user access?

It doesn't. Data files can be read simultaneously by multiple applications, even until one of the processes executes and fully completes a call to commit changes to file. There is no mechanism to synchronize information after this point, however. With the current version, you have three options: implement a server with exclusive access to the data file and which handles client requests in serial order, solve the synchronization issue between processes outside of MetaKit, or stay away from this version of MetaKit for all shared data.

Can MetaKit be used in other languages, such as Delphi or Visual Basic?

The library is strongly oriented toward C++, and relies heavily on object constructors and destructors to maintain its reference-counted data structures. The only way in which you can integrate MetaKit in other languages right now, is to write a C++ wrapper, possibly as a DLL, which encapsulates the library with a procedure-call based interface.
For questions or comments, please contact Jean-Claude Wippler @ Meta Four Software.