Thread Context Classloader in Deserialization - ObjectInputStream

We had uses cases, especially with OSGI where we had to use TCCL in deserialization of Object InputStream.

Sample method

public static Object getObject(byte[] bArr){
try{
ByteArrayInputStream bais = new ByteArrayInputStream(bArr);
ObjectInputStream ois = new ObjectInputStream(bais);
return ois.readObject();
}catch(Exception err){
err.printStackTrace();
}
return null;
}


return ois.readObject() - fails because ObjectInputStream doesnt take into consideration TCCL when deserializing.

In OSGi we need to deserialize this using TCCL because objects being deserialized are in another bundle and framework bundle canot see these classes.

Java has provided a way to solve this . We need to extend ObjectInputStream and overide the
public Class resolveClass(ObjectStreamClass desc) method.

[Since resolveClass class is protected method in ObjectInputStream ]

Code


public class MyObjectInputStream extends ObjectInputStream {

@Override
public Class resolveClass(ObjectStreamClass desc) throws IOException,
ClassNotFoundException {
ClassLoader currentTccl = null;
try {
currentTccl = Thread.currentThread().getContextClassLoader();
return currentTccl.loadClass(desc.getName());
} catch (Exception e) {
}

}
return super.resolveClass(desc);
}

public MyObjectInputStream(InputStream in) throws IOException {
super(in);
}

}

Now we can use MyObjectInputStream which ensures that TCCL will have visibility to deserialize the class.

MyObjectInputStream ois = new MyObjectInputStream(bais);
return ois.readObject();

Using Bundlor for automated generation of Osgi manifest files

One of the most useful utilities which helps in automated creation of manifest files. This gives all the hooks required for customization.

A sample template file used for generation

Bundle-Name: ${bundle.name}
Bundle-Description: ${bundle.description}
Bundle-ManifestVersion: 2
Bundle-SymbolicName: ${bundle.symbolic.name}
Bundle-Vendor: ${bundle.vendor}
Import-Template: org.apache.log4j.*;bundle-symbolic-name="com.springsource.org.apache.log4j", *
Excluded-Imports: com.sun.*,javax.xml.*,org.apache.xerces.jaxp.*,org.w3c.*,org.xml.*,sun.*,com.tc.*


Here the Import template functionality gives complete flexibility to control my imports> This ensures that all subpackages of org.apache.lo4j if imported by bundle will have bundle symbolic names attached to it.

If there is any private package , based on some naming convention we can exclude this from being exported. Also boot delegated packages should be excluded for exporting.

Boot delegated package exports can be avoided by passing osgi profile file to bundlor.
At this moment - Bundlor 1.0.0 had a Bug with this feature which is fixed in the daily snapshot version.
Ref : http://forum.springsource.org/showthread.php?t=87312

Hence bundlor automates a lot of these things and there is no need for static commit of manifest files.

This can be easily integrated with any build for the project as follows

<target name="bundlor.init">
<taskdef resource="com/springsource/bundlor/ant/antlib.xml"
uri="antlib:com.springsource.bundlor.ant">
<classpath id="bundlor.classpath">
<fileset dir="${bundlor.home}/dist"/>
<fileset dir="${bundlor.home}/lib"/>
</classpath>
</taskdef>
</target>


<bundlor:bundlor
inputPath="${basedir}/xxx_jar.jar"
outputPath="${basedir}/xxx_bundle.jar"
manifestTemplatePath="${basedir}/${bundlor.config.dir}/xxx.mf"
propertiesPath="${basedir}/${bundlor.config.dir}/xxx.properties"
osgiProfilePath="${basedir}/${bundlor.home}/profile/java6-server.profile"/>

Ref: http://www.springsource.org/bundlor

Integration of Terracotta with Spring Dm server

At this moment I am using the following version of the softwares

Spring DM server(or DM Kernel) - 2.0.1 version
Terracotta- version 3.2.1

Steps

In Dm server

1. Modify springsource-dm-kernel-2.0.1.RELEASE\bin\dmk.sh to start with terracotta cleint.

Change $JAVA_HOME/bin/java to $TERRACOTTA_CLIENT_PATH/bin/dso-java.sh

2. Change springsource-dm-kernel-2.0.1/lib/java6-server.profile to include terracotta classes in boot delegation path

Add under org.osgi.framework.bootdelegation = \
com.tc.*,\
com.tcclient,\
com.tcclient.*

Changes in Terracotta

1. change tc-config.xml to have the following tims

<modules>
<module name="tim-equinox-3.5.1" version="1.1.1"/>
<module name="tim-ehcache-2.0" version="1.5.2"/>
<module name="tim-distributed-cache" version="1.3.2"/>
<module name="tim-concurrent-collections" version="1.3.2"/>
</modules>

2. Add the following app group element in tc-config.xml

<app-group name="osgiAppGroup">
<named-classloader>Standard.system</named-classloader>
<named-classloader>dso-shared.0.0.0</named-classloader>
</app-group>


Thw app group element in only required if you want to share dso classes between standalone JVM and Spring Dm server(where dso classes are laoded by bundle classlaoder).

Also important thing to be noted is that inside a JVM all shared classes(dso's) have to be laoded by a single classloader.[Inside spring Dm server all dso classes should be a single bundle]

Soft Reference /Weak Refernce /Phantom Reference / finalize()

Using teh above types we get a refernce to a object which ensure that teh object can still be GCed.

Main purpose of all the above types is to ensure that the reference is not held and hence GC can
collect all these objects from the heap.

The order in which the reference - from Strong to Weak are as follows
Strong reference,Soft Reference ,Weak reference and Phantom Reference

Strong Reference - are normal references which will not be garbage collected because some objects will have reference to it.

SoftReference -
Having a soft Reference to a object will not stop GC on that object. By using SoftRefernce the virtual machines does not clear this references until there is shortage of memory.But this ensures that before the system crashes with OutOfMemory errors all soft references will be GC'ed.

Hence this becomes a very good candidate for maintaing cache.Thus objects will remain in cache until there is enough memory. Once there is shortage they will be collected.


Weak Reference - Is similar to soft reference with the only difference that it will be GC'ed as soon as there are no other references . This will be very handly in most cases where there exists explicit refernce to objects which stop garbage colection.

If Weak Reference is not used , we need to explicitly clear the contents as part of cleanup.

WeakHashMap is one such data structure which will automatically clear references when there are not other references to objects. WeakHashMap works exactly like HashMap, except that the keys (not the values!) are referred to using weak references .If a WeakHashMap key becomes garbage, its entry is removed automatically.


Both weak and Softreference have constructors of format

WeakReference(Object referent, ReferenceQueue q) & WeakReference(Object referent)
SoftReference(Object referent, ReferenceQueue q) & SoftReference(Object referent)

where ReferenceQueue is queue which will be populated with objects(Weak/Soft references) which are GC'ed.[The objects they are pointing to will be GC'ed after which these references become dead]

Hence we can use this ReferenceQueue info for cleanup purposes.


Reference queues

Once a WeakReference starts returning null, the object it pointed to has become garbage and the WeakReference object is pretty much useless. This generally means that some sort of cleanup is required;

The ReferenceQueue class makes it easy to keep track of dead references. If you pass a ReferenceQueue into a weak reference's constructor, the reference object will be automatically inserted into the reference queue when the object to which it pointed becomes garbage. You can then, at some regular interval, process the ReferenceQueue and perform whatever cleanup is needed for dead references.

The remove() method can be used to get notifications on objects Garbage Collected.
http://java.sun.com/j2se/1.4.2/docs/api/java/lang/ref/ReferenceQueue.html

To access WeakReference or SoftReference we have to use the public Object get().
If this reference object has been cleared, either by the program or by the garbage collector, then this method returns null.


A soft reference is exactly like a weak reference, except that it is less eager to throw away the object to which it refers. An object which is only weakly reachable (the strongest references to it are WeakReferences) will be discarded at the next garbage collection cycle, but an object which is softly reachable will generally stick around for a while


Finalize()


This is Called by the garbage collector on an object when garbage collection determines that there are no more references to the object.

This has to be carefully used because sometime improper use of this method can get teh object back to life.

This is also costly because garbage collection in its every traversal has to keep account of the finalize method variables and exclude it from considering as normal reference.Also until finalize method is called other objects will not be GC'ed because of its changes to come back to life. This can be solved with Phantom reference.

Finalize() method is costly -hence phantom reference can be used for the same purpose.

Phantom Reference

Phantom references are most often used for scheduling pre-mortem cleanup actions in a more flexible way than is possible with the Java finalization mechanism.

In order to ensure that a reclaimable object remains so, the referent of a phantom reference may not be retrieved: The get method of a phantom reference always returns null.

A phantom reference is quite different than either SoftReference or WeakReference. Its grip on its object is so tenuous that you can't even retrieve the object -- its get() method always returns null.

The only use for such a reference is keeping track of when it gets enqueued into a ReferenceQueue, as at that point you know the object to which it pointed is dead and perform some cleanup.

Difference b/ Weak-Soft & Phantom Reference

The difference is in exactly when the enqueuing happens. Weak/Soft References are enqueued as soon as the object to which they point becomes weakly reachable. This is before finalization or garbage collection has actually happened; in theory the object could even be "resurrected" by an unorthodox finalize() method, but the WeakReference would remain dead.

PhantomReferences are enqueued only when the object is physically removed from memory, and the get() method always returns null specifically to prevent you from being able to "resurrect" an almost-dead object.

2 Usecases with PhantomReference

1. Only way to determine exactly when an object was removed from memory based on which some actions can be performed

2.PhantomReferences avoid a fundamental problem with finalization: finalize() methods can "resurrect" objects by creating new strong references to them. Hence garbage collection of teh object is postponed untill finalize metdod is called.(because finaize object may
reconstruct the object ).With phantom reference no way the objects can be reconstructed and Gc will happen soon


Reference.
http://weblogs.java.net/blog/enicholas/archive/2006/05/understanding_w.html
http://mindprod.com/jgloss/weak.html
http://www.javalobby.org/forums/thread.jspa?threadID=16520&messageID=91822406&tstart=0
http://onjava.com/pub/a/onjava/2001/07/09/optimization.html?page=2