Saturday, March 21, 2009

Protobuf - Data Interchange Format

It is a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage etc

Protocol Buffers are a way of encoding structured data in an efficient yet extensible format.Google uses Protocol Buffers for almost all of its internal RPC protocols and file formats.

You define how you want your data to be structured once, then you can use special generated
source code to easily write and read your structured data to and from a variety of data streams
and using a variety of languages

You specify how you want the information you're serializing to be structured by defining protocol
buffer message types in .proto files. Each protocol buffer message is a small logical record of
information, containing a series of name-value pairs.Once you've defined your messages, you run
the protocol buffer compiler for your application's language on your .proto file to generate data
access classes.

These data access classes can be in any of the languages such as C , Java ,Python etc.These provide simple accessors for each field as well as methods to serialize/parse the whole
structure to/from raw bytes. These generted classes can be shipped along with any client where
serialisation or deserialsation is required.

You can even update your data structure without breaking deployed programs that are compiled
against the "old" format.

Protocol buffers have many advantages over XML for serializing structured data.

Protocol buffers:
* are simpler
* are 3 to 10 times smaller
* are 20 to 100 times faster
* are less ambiguous
* generate data access classes that are easier to use programmatically

Ref :

Java DataBases - Derby(Java DB) & HSQLDB

These are those two 100% Java databases, Derby and HSQLDB.

Apache Derby is developed as an open source project under the Apache 2.0 licence.
Derby was previously distributed as IBM Cloudscape.It is currently distributed as
Sun Java DB and comes packaged along with Java.

HSQLDB (Hyperthreaded Structured Query Language Database) is a relational database
management system written in Java.

Derby aims to be a fully functional relational database, and as such could be considered
as an alternative to "grown up" databases like Oracle and MySQL in some cases.

HSQL is more like a "toy database", providing a subset of the features of a fully
functional databases. But that subset of functionality may be all you need.

Benefits of HSQL over Derby.
- light weight (smaller libraries, less complex)
- supports in memory databases
- Simple to use

Benefits of Derby over HSQL
- much larger development community
- secured future (is part of JDK, is sponsored by IBM and Sun)
- More complete feature set (e.g. proper handling of transactions, encryption...)
- Scalability
- Performance with large Data Set

Memory Use

Derby is setup like a traditional database system where data is stored
on disk and a subset of that data is cached in a buffer cache or pool.

HSQL by default uses MEMORY tables (for standard CREATE TABLE
statements), this always stores the entire data in memory. Thus this is
fast, but obviously consumes memory. It is useful to look a process
sizes as well as absolute times when running benchmarks.


SQLite - LightWeight DB

SQLite is a software library that implements a self-contained, serverless,zero-configuration, transactional SQL database engine.It is opensource.

It is used by Firefox,Adobe,google,Microsoft etc

SQLite is different from most other SQL database engines in that its primary design goal is to be simple:

* Simple to administer
* Simple to operate
* Simple to embed in a larger program
* Simple to maintain and customize

It has Zero-Configuration, is Server Less ,Single Database File ,Stable Cross-Platform Database
File and gives easy user friendly APIs to write into and access from the database.

Situations Where SQLite Works Well

Embedded devices and applications
Replacement for ad hoc disk files
Low Concurrency

SQLite home page

Free Domain Names @!