Using Bundlor for automated generation of Osgi manifest files

One of the most useful utilities which helps in automated creation of manifest files. This gives all the hooks required for customization.

A sample template file used for generation

Bundle-Name: ${bundle.name}
Bundle-Description: ${bundle.description}
Bundle-ManifestVersion: 2
Bundle-SymbolicName: ${bundle.symbolic.name}
Bundle-Vendor: ${bundle.vendor}
Import-Template: org.apache.log4j.*;bundle-symbolic-name="com.springsource.org.apache.log4j", *
Excluded-Imports: com.sun.*,javax.xml.*,org.apache.xerces.jaxp.*,org.w3c.*,org.xml.*,sun.*,com.tc.*


Here the Import template functionality gives complete flexibility to control my imports> This ensures that all subpackages of org.apache.lo4j if imported by bundle will have bundle symbolic names attached to it.

If there is any private package , based on some naming convention we can exclude this from being exported. Also boot delegated packages should be excluded for exporting.

Boot delegated package exports can be avoided by passing osgi profile file to bundlor.
At this moment - Bundlor 1.0.0 had a Bug with this feature which is fixed in the daily snapshot version.
Ref : http://forum.springsource.org/showthread.php?t=87312

Hence bundlor automates a lot of these things and there is no need for static commit of manifest files.

This can be easily integrated with any build for the project as follows

<target name="bundlor.init">
<taskdef resource="com/springsource/bundlor/ant/antlib.xml"
uri="antlib:com.springsource.bundlor.ant">
<classpath id="bundlor.classpath">
<fileset dir="${bundlor.home}/dist"/>
<fileset dir="${bundlor.home}/lib"/>
</classpath>
</taskdef>
</target>


<bundlor:bundlor
inputPath="${basedir}/xxx_jar.jar"
outputPath="${basedir}/xxx_bundle.jar"
manifestTemplatePath="${basedir}/${bundlor.config.dir}/xxx.mf"
propertiesPath="${basedir}/${bundlor.config.dir}/xxx.properties"
osgiProfilePath="${basedir}/${bundlor.home}/profile/java6-server.profile"/>

Ref: http://www.springsource.org/bundlor

Integration of Terracotta with Spring Dm server

At this moment I am using the following version of the softwares

Spring DM server(or DM Kernel) - 2.0.1 version
Terracotta- version 3.2.1

Steps

In Dm server

1. Modify springsource-dm-kernel-2.0.1.RELEASE\bin\dmk.sh to start with terracotta cleint.

Change $JAVA_HOME/bin/java to $TERRACOTTA_CLIENT_PATH/bin/dso-java.sh

2. Change springsource-dm-kernel-2.0.1/lib/java6-server.profile to include terracotta classes in boot delegation path

Add under org.osgi.framework.bootdelegation = \
com.tc.*,\
com.tcclient,\
com.tcclient.*

Changes in Terracotta

1. change tc-config.xml to have the following tims

<modules>
<module name="tim-equinox-3.5.1" version="1.1.1"/>
<module name="tim-ehcache-2.0" version="1.5.2"/>
<module name="tim-distributed-cache" version="1.3.2"/>
<module name="tim-concurrent-collections" version="1.3.2"/>
</modules>

2. Add the following app group element in tc-config.xml

<app-group name="osgiAppGroup">
<named-classloader>Standard.system</named-classloader>
<named-classloader>dso-shared.0.0.0</named-classloader>
</app-group>


Thw app group element in only required if you want to share dso classes between standalone JVM and Spring Dm server(where dso classes are laoded by bundle classlaoder).

Also important thing to be noted is that inside a JVM all shared classes(dso's) have to be laoded by a single classloader.[Inside spring Dm server all dso classes should be a single bundle]

Soft Reference /Weak Refernce /Phantom Reference / finalize()

Using teh above types we get a refernce to a object which ensure that teh object can still be GCed.

Main purpose of all the above types is to ensure that the reference is not held and hence GC can
collect all these objects from the heap.

The order in which the reference - from Strong to Weak are as follows
Strong reference,Soft Reference ,Weak reference and Phantom Reference

Strong Reference - are normal references which will not be garbage collected because some objects will have reference to it.

SoftReference -
Having a soft Reference to a object will not stop GC on that object. By using SoftRefernce the virtual machines does not clear this references until there is shortage of memory.But this ensures that before the system crashes with OutOfMemory errors all soft references will be GC'ed.

Hence this becomes a very good candidate for maintaing cache.Thus objects will remain in cache until there is enough memory. Once there is shortage they will be collected.


Weak Reference - Is similar to soft reference with the only difference that it will be GC'ed as soon as there are no other references . This will be very handly in most cases where there exists explicit refernce to objects which stop garbage colection.

If Weak Reference is not used , we need to explicitly clear the contents as part of cleanup.

WeakHashMap is one such data structure which will automatically clear references when there are not other references to objects. WeakHashMap works exactly like HashMap, except that the keys (not the values!) are referred to using weak references .If a WeakHashMap key becomes garbage, its entry is removed automatically.


Both weak and Softreference have constructors of format

WeakReference(Object referent, ReferenceQueue q) & WeakReference(Object referent)
SoftReference(Object referent, ReferenceQueue q) & SoftReference(Object referent)

where ReferenceQueue is queue which will be populated with objects(Weak/Soft references) which are GC'ed.[The objects they are pointing to will be GC'ed after which these references become dead]

Hence we can use this ReferenceQueue info for cleanup purposes.


Reference queues

Once a WeakReference starts returning null, the object it pointed to has become garbage and the WeakReference object is pretty much useless. This generally means that some sort of cleanup is required;

The ReferenceQueue class makes it easy to keep track of dead references. If you pass a ReferenceQueue into a weak reference's constructor, the reference object will be automatically inserted into the reference queue when the object to which it pointed becomes garbage. You can then, at some regular interval, process the ReferenceQueue and perform whatever cleanup is needed for dead references.

The remove() method can be used to get notifications on objects Garbage Collected.
http://java.sun.com/j2se/1.4.2/docs/api/java/lang/ref/ReferenceQueue.html

To access WeakReference or SoftReference we have to use the public Object get().
If this reference object has been cleared, either by the program or by the garbage collector, then this method returns null.


A soft reference is exactly like a weak reference, except that it is less eager to throw away the object to which it refers. An object which is only weakly reachable (the strongest references to it are WeakReferences) will be discarded at the next garbage collection cycle, but an object which is softly reachable will generally stick around for a while


Finalize()


This is Called by the garbage collector on an object when garbage collection determines that there are no more references to the object.

This has to be carefully used because sometime improper use of this method can get teh object back to life.

This is also costly because garbage collection in its every traversal has to keep account of the finalize method variables and exclude it from considering as normal reference.Also until finalize method is called other objects will not be GC'ed because of its changes to come back to life. This can be solved with Phantom reference.

Finalize() method is costly -hence phantom reference can be used for the same purpose.

Phantom Reference

Phantom references are most often used for scheduling pre-mortem cleanup actions in a more flexible way than is possible with the Java finalization mechanism.

In order to ensure that a reclaimable object remains so, the referent of a phantom reference may not be retrieved: The get method of a phantom reference always returns null.

A phantom reference is quite different than either SoftReference or WeakReference. Its grip on its object is so tenuous that you can't even retrieve the object -- its get() method always returns null.

The only use for such a reference is keeping track of when it gets enqueued into a ReferenceQueue, as at that point you know the object to which it pointed is dead and perform some cleanup.

Difference b/ Weak-Soft & Phantom Reference

The difference is in exactly when the enqueuing happens. Weak/Soft References are enqueued as soon as the object to which they point becomes weakly reachable. This is before finalization or garbage collection has actually happened; in theory the object could even be "resurrected" by an unorthodox finalize() method, but the WeakReference would remain dead.

PhantomReferences are enqueued only when the object is physically removed from memory, and the get() method always returns null specifically to prevent you from being able to "resurrect" an almost-dead object.

2 Usecases with PhantomReference

1. Only way to determine exactly when an object was removed from memory based on which some actions can be performed

2.PhantomReferences avoid a fundamental problem with finalization: finalize() methods can "resurrect" objects by creating new strong references to them. Hence garbage collection of teh object is postponed untill finalize metdod is called.(because finaize object may
reconstruct the object ).With phantom reference no way the objects can be reconstructed and Gc will happen soon


Reference.
http://weblogs.java.net/blog/enicholas/archive/2006/05/understanding_w.html
http://mindprod.com/jgloss/weak.html
http://www.javalobby.org/forums/thread.jspa?threadID=16520&messageID=91822406&tstart=0
http://onjava.com/pub/a/onjava/2001/07/09/optimization.html?page=2

Useful OSGi(Spring DM) utilities

1.
Using OsgiServiceProxyFactoryBean to create proxy(equivalent to osgi:ref)

I had usecase where i could not use osgi:ref or osgi:list to reference a service ,because when the proxy is created it
creates on the generic interface given, which that particular bundle can see.

To get away with this usecase , we created proxies using API's.

http://static.springsource.org/osgi/...orter/support/OsgiServiceProxyFactoryBean.html

OsgiServiceProxyFactoryBean proxy = new OsgiServiceProxyFactoryBean();
proxy.setBundleContext(bundle.getBundleContext());
proxy.setBeanClassLoader(cls.getClassLoader());
proxy.setInterfaces(new Class[] {cls});
proxy.afterPropertiesSet();

2.

There is a distinction between the OSGi bundle state and the state of the BundleInstallArtifact. The latter is displayed by the shell "bundle" commands.

BundleInstallArtifact only transitions to ACTIVE when any Spring DM application context has been published in the service registry. The OSGi bundle may
transition to ACTIVE earlier than this as it does not take into account publication of any Spring DM application context.

The transition to ACTIVE state starts either when the bundle is started or, in the case of lazy activation, after the bundle has been started and then
the first class is loaded from the bundle.

3.

Bundle-Version: 10.0.0

Export-Package: com.test

There is no relationship between the bundle and the package version. It's fairly often that a bundle version increases while the bundle package doesn't
(for example the new bundle provides only extra resources or extra packages).

Exported packages by default are versioned at 0.0.0

4.

When I hit a bundles list command on the dm shell it shows as follows

DM Shell output

43 com.springsource.osgi.webcontainer.core 1.0.0.M1 ACTIVE
44 S com.springsource.osgi.webcontainer.tomcat 1.0.0.M1 ACTIVE
45 S com.springsource.server.web.core 2.0.0.RELEASE ACTIVE
46 com.springsource.server.web.dm 2.0.0.RELEASE ACTIVE
47 com.springsource.server.web.tomcat 2.0.0.RELEASE ACTIVE
48 com.springsource.javax.annotation 1.0.0 ACTIVE

S means "Spring powered", i.e. with Spring (DM) XML files.

Useful Spring Utilities(API's)

1. Default behaviour of Spring is that , when same names beans are loaded in a single applicationContext , the first bean will
be overridden by the latter.

We had usecase where we required to get an error when multiple beans had the same name.


I have to extend ClassPathXmlApplicationContext and overide this method as it was protected.

@Override
public void setAllowBeanDefinitionOverriding(
boolean allowBeanDefinitionOverriding) {
super.setAllowBeanDefinitionOverriding(allowBeanDefinitionOverriding);
}


ctx.setAllowBeanDefinitionOverriding(false);
ctx.setConfigLocation("classpath*:/META-INF/spring/conf*.xml");
ctx.afterPropertiesSet(); //will initialize the context


2. To get teh number of beans initialized by the application Context.

ctx.getBeanDefinitionCount());


3. To get the resouces read by created ApplicationContext based ona regular Expression.
[This is very handy in an integratde environment to exactlt know the resources ]

I had to extend ClassPathXmlApplicationContext as this method was protected.

@Override
protected Resource[] getConfigResources() {
// TODO Auto-generated method stub
return super.getConfigResources();
}


Resource[] resources = ((ResourcePatternResolver) ctx).getResources("classpath*:/META-INF/spring/conf*.xml");

where ctx is the initialized applicationContext.


4. If we need multiple definitons for <context:property-placeholder/>

Only one context:property-placeholder (the last one defined in the configuration files) will prevail.
You can define two separate PropertyPlaceholderConfigurers and work, only if the second one has a different placeholderPrefix and placeholderSuffix.


< bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" >
< property name="location" value="classpath:/com/foo/2.properties" / >
< property name="placeholderPrefix" value="#[" / >
< property name="placeholdersuffix" value="]" / >
< / bean >

< bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer" >
< property name="location" value="classpath:/1.properties" / >
< / bean >


Normal way of replacing is ${property1} and with new prefix and suffix #[]

Difference between Classpath:* & Classpath*: in Spring

The difference between these two is the way the resources are found when a regular expression is given for match.

classpath*: -> This special prefix specifies that all classpath resources that match the given name must be obtained
(internally, this essentially happens via a ClassLoader.getResources(...) call), and then merged to form the final application context definition.

Ex classpath*:app*.xml && classpath:app*.xml is

With spring xmls, app1.xml and app2.xml and beans in app2.xml referring to app1.xml

- classpath:app*.xml will give errors if app1.xml and app2.xml are in two different classpath folders(It tries to find a matching app1 or app2 in classpath , and when
found doesnt continue to search further with other classpath locations.)

- classpath*:app*.xml will never give errors(searches for all classpath folders)

classpath* will search all classpath folder whereas classpath: will concentrate on one resource.

This also works for location attribute of PropertyPlaceHolderConfig

<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations" value="classpath*:jdbc.properties"></property>
</bean>

But important thing to be noted here is that property name is locations and not location.


One more difference with classpath*:config.xml and classpath:*config.xml

is that

classpath:*config.xml - if file(matching *config) isnt present it shows error in console

classpath*:*config.xml just ignores teh error if no file is present in the classpath.

classpath*: basically refers to a list of resources and a list can be empty.
classpath: simply points to a certain resource and if this is empty (where one would have expected it to exist) it would generate an error.


Note:

Note that "classpath*:" when combined with Ant-style patterns will only work reliably with at
least one root directory(IMPORTANT) before the pattern starts, unless the actual target files
reside in the file system.This means that a pattern like "classpath*:*.xml" will not retrieve
files from the root of jar files but classpath*:META-INF/* will search as expected.

Spring3 Features -TaskExecutor & TaskSchedulor

The Spring Framework provides abstractions for asynchronous execution and scheduling of tasks with the TaskExecutor and TaskScheduler interfaces, respectively.

http://docs.google.com/fileview?id=0B-0gqvlsl_VvNTBlMDdmZmUtYWE4Ni00YmRhLTk0ODAtYWNhYjk4OGM5YTk1&hl=en
[Download - rename from pdf to rar]
[Attached is the sample code for both –with annotations]

1. TaskExecutor

Spring's TaskExecutor interface is identical to the java.util.concurrent.Executor interface.

ConcurrentTaskExecutor : This implementation is a wrapper for a Java 5 java.util.concurrent.Executor.

ThreadPoolTaskExecutor : Spring's TaskExecutor implementation.

[org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor – Namespace: ]

Parametres : corePoolSize,maxPoolSize,queuecapacity,rejection-policy

The executor will first try to use a free thread if the number of active threads is currently less than the core size. If the core size has been reached, then the task will be added to the queue as long as its capacity has not yet been reached. Only then, if the queue's capacity has been reached, will the executor create a new thread beyond the core size. If the max size has also been reached, then the executor will reject the task.

By default, when a task is rejected, a thread pool executor will throw a TaskRejectedException. CallerRunsPolicy. Instead of throwing an exception or discarding tasks, that policy will simply force the thread that is calling the submit method to run the task itself. The idea is that such a caller will be busy while running that task and not able to submit other tasks immediately. Therefore it provides a simple way to throttle the incoming load while maintaining the limits of the thread pool and queue.

2. TaskScheduler

Spring 3.0 introduces a TaskScheduler with a variety of methods for scheduling tasks to run at some point in the future.

- can be associated with a pool size.

Taskschedulor.schedule() takes a trigger expression based on which tasks will be scheduled.

Annotation Support

Both have very easy annotation support.

@Async - This can be marked on any POJO method

@Scheduled- This annotation can be marked on any POJO method

Just add  in the xml and give respective scheduler or executor as properties.

Currently with spring 3 RC2 and Spring Dm 2.0M6 – I could not use namespace support as they were giving some issues. It could be a BUG and I have raised this issue in the forum.

http://forum.springsource.org/showthread.php?t=81676

In the attached sample code instead of using namespace I have used the entire class name.

Namespace should work with Spring 3 RC3 version.