Issues during migration of platform to OSGi

1. Certain framework classes had to be part of boot delegation like Terracotta.

Added com.tc.* to org.osgi.framework.bootdelegation = \ in

/home/pg/thirdparty/springsource-dm-kernel-2.0.0.RELEASE/lib/java6-server.profile

2.Certain jboss imports were not getting part of package imports when jars were run through bundlor

So had to explicitly add these to bundles which required JMS(JBoss MQ) functionalities.

org.jboss.logging,
org.jboss.mq.referenceable,
org.jnp.interfaces,
org.jboss.util,


3. If there is any Custom Class Loader created in the code ,special care has to be taken to ensure that you set the custom class loaders parent to Bundle Classloader.


4. Special care needs to be taken that classes loaded through custom class loader should see all its imports and exports through its parent Bundles manifest.


5. Where ever class was loaded using Class.forname() we had to change it to ensure these where done through Thread Context Classloader where classes were laoded from other bundle.

6. Since we use jbossallclient.jar for JMS(Jboss MQ) we had issues as this was conflicting with packages in bootdelegation.

We could either solve this using explicit imports from system bundle(bundle 0).

a.

BootClasspath was exporting javax.management

Removed javax.management from jboss all client jar - Using bundlor excluded exports manifest

b.

Because jboss-allclient.jar also exports javax.jms which is also exported by javax.jms_1.1.0 bundle

Ensured jbossallclient.jar imports this and doesnt export it by changing manifest.


7.Since there were multiple bundles exporting org.apache.log4j , i had to explicitly mention bundle
symbolic names for the imports.

There were certain cases of cyclic dependency since it was legacy code.

Code in one bundle was initialized(spring application context) by code in another bundle. This cyclic dependency was solved by
explicitly marking this imports as optional(resolution="optional").


8. Wherever Application Context was read using regular expression had to change classpath: to classpath*: to ensure that all classpath resources are read.

BootStrap Initialization GuideLines (For same code base to work in both OSGi and Non-OSGi environment)

During the migration of the platform into OSGi we had use cases where we needed the platform to work in both OSGi and non OSGi cases until the OSGi platform had stabilized.But our mail goal was we needed the same code based and packaging to work in both OSGi and non OSGi use cases.

We had come up with list of guidelines and conventions to achieve the same.

One major assumption was that in a non OSGi case there has to be a single application Context as against OSGi where there is application context
per bundle.


1. Separate xml files for OSGi service definitions and bean definitions.

2. Naming convention for spring xml files are as follows

<MODULE_NAME>-osgi-context.xml
<MODULE_NAME>-bean-context.xml
3.Naming convention for beans to be exported as services

Service Exposing side
<bean id="<MODULE_NAME>service" class="XXX.YYY.ZZZ"/>
<osgi:service ref=""<MODULE_NAME>service"" interface="bla.bla.IService"/>

Service Reference side
<osgi:ref interface=" bla.bla.IService" id="<MODULE_NAME>service"/>
Now this <MODULE_NAME>service can be used as regular bean.
This convention will help us run the same code base both in OSGi and non OSGi cases without changing these xmls. Note that in non OSgi cases we will initialize META-INF/spring/*-bean-context.xml. This will help in unit testing so that only required xmls can be initialized.

4. All spring bean definition files have to be in the following location - META-INF/spring/*

5. Ensure that in NON-OSGI cases individual products/services/modules do not independently initialize the application Context.There should be a independent initialization moduel which initilialises the app context across all products(jars) in the Vm based on some naming convention followed by the products.This will help easy wiring of beans across products which in other cases are done through OSGi services.Also the common module will expose a method in for accessing the beans present in the context.

6. Conventions have to be followed so that there are no same named beans across products as there is a single application context and objects will get overridden.

7.Boot Strap initialized xmls with this regular expression - [*-bean-context.xml]

[** Ensure that if any module in not following all these conventions the spring xml file has to be in any other folder than META-INF/spring ***[so that DM server doesnt initialize it ]* . *Ensure that in any case the spring xmls** are not initialized twice *[all the above guidelines are just to ensure that ] ***]

8. Property File initialization

Property placeholder config definitions containing files should be seperated from bean and osgi definition files and should be present in META-INF/spring folder.

Naming convention pgconfig-<MODULE_NAME>.properties

This will be initialized in bootstrap based on this expression in teh bootstrap propertyPlaceHolconfigurator : "classpath*:pgconfig*.properties"

{"classpath*:META-INF/spring/bootstrapconfig.xml","classpath*:/META-INF/spring/*-bean-context.xml"};

bootstrapconfig.xml contains --> <context:property-placeholder location="classpath*:pgconfig*.properties"/>

9.Avoid using Bundle Activator and instead use spring config (init method) for initialization as this will give us a common approach in both OSGi and Non OSGi cases.

OSGi Guidelines

1.Interaction between non-core bundles which can be hot-deployed should be through services (Interfaces).

2.Bundle should Import packages only for utilities, datatypes packages / classes and interfaces.

3.Interface and implementation should be separately bundled. The datatypes associated with service contracts can be bundled along with the interfaces.

4.All third party jars should be converted into bundles. Avoid using "external" in Bundle-classpath.

5.Avoid split package. Proper package structure should be followed. Two bundles should not have same package structure. We can have package structure like -.

6.TC shared classes to be bundled into separate bundles.

7.Avoid static variables and blocks.

8.Avoid Class.forname instead use Classloader.loadclass(). Class.forname may cause issues when bundle is hot deployed as initiating classloader caches the loaded class.

9.Classes present in separate dependent bundles should be loaded using TCCL. For example in jdbcframework all TYPE classes would be bundled with the respective bundle. These classes have to be loaded by jdbcframework bundle. Since classes present in respective bundle will not be visible to jdbcframewok bundle, so jdbcframework bundle has to use TCCL to load these classes.

10.Cyclic dependency should not exist between bundles.

11.Resource clean up - Each bundle / bean should clean up its resources like threads, sockets, cache, file handles, connections etc using its life cycle method (using "destroy" of bean or "stop" of "bundle activator")

12.Use the bean initialization method to initialize the properties and avoid putting initialization logic into constructor.

13.Native library to be loaded once. The code that is loading the native library should be part of core bundle which should not be hot deployed.

14.Prefer composition over inheritance.

15.Use AOP for common concerns - prefer compile time weaving over load-time / runtime weaving.

16.Use DM kernel when web container is not required.

17.Prefer import package over import-bundle.

18.All In-House jars should not have resolution optional for package imports.[Just to ensure that in production enviroment dependencies are resolved at deployment time rather than run time.]

19.All third party converted jars have resolution=optional.Else this pulled a not of other jars which were never used.

20.All log4j imports should have bundle symbolic name for lo4j to resolve properly to log4j instead of slf4j - something like

org.apache.log4j.*;bundle-symbolic-name="com.springsource.org.apache.log4j"

This can be added as part of Bundlor template during building the jar.

21.Manifest is generated at build time based on template(Spring Utility like Bundlor can be used for the same).

Useful OSGi(Spring DM) utilities

1.
Using OsgiServiceProxyFactoryBean to create proxy(equivalent to osgi:ref)

I had usecase where i could not use osgi:ref or osgi:list to reference a service ,because when the proxy is created it
creates on the generic interface given, which that particular bundle can see.

To get away with this usecase , we created proxies using API's.

http://static.springsource.org/osgi/...orter/support/OsgiServiceProxyFactoryBean.html

OsgiServiceProxyFactoryBean proxy = new OsgiServiceProxyFactoryBean();
proxy.setBundleContext(bundle.getBundleContext());
proxy.setBeanClassLoader(cls.getClassLoader());
proxy.setInterfaces(new Class[] {cls});
proxy.afterPropertiesSet();

2.

There is a distinction between the OSGi bundle state and the state of the BundleInstallArtifact. The latter is displayed by the shell "bundle" commands.

BundleInstallArtifact only transitions to ACTIVE when any Spring DM application context has been published in the service registry. The OSGi bundle may
transition to ACTIVE earlier than this as it does not take into account publication of any Spring DM application context.

The transition to ACTIVE state starts either when the bundle is started or, in the case of lazy activation, after the bundle has been started and then
the first class is loaded from the bundle.

3.

Bundle-Version: 10.0.0

Export-Package: com.test

There is no relationship between the bundle and the package version. It's fairly often that a bundle version increases while the bundle package doesn't
(for example the new bundle provides only extra resources or extra packages).

Exported packages by default are versioned at 0.0.0

4.

When I hit a bundles list command on the dm shell it shows as follows

DM Shell output

43 com.springsource.osgi.webcontainer.core 1.0.0.M1 ACTIVE
44 S com.springsource.osgi.webcontainer.tomcat 1.0.0.M1 ACTIVE
45 S com.springsource.server.web.core 2.0.0.RELEASE ACTIVE
46 com.springsource.server.web.dm 2.0.0.RELEASE ACTIVE
47 com.springsource.server.web.tomcat 2.0.0.RELEASE ACTIVE
48 com.springsource.javax.annotation 1.0.0 ACTIVE

S means "Spring powered", i.e. with Spring (DM) XML files.

Spring Integration+JMS(JBOSS)+OSGi

I have a POC for spring integration to be used along with JBoss and OSGi in Spring Dm server 2.0M6.

Code Sample
http://docs.google.com/fileview?id=0B-0gqvlsl_VvNzhiMzVmMzgtY2Y4Zi00YjJmLWJmNWQtMTIwMDZjNDViMmJl&hl=en
[Download this pdf - Rename file from pdf to rar and extract]


SwitchBoardBundle – Exposes connection factories as service

Exposes common thread pools as service

[Jboss ip values for connection factories are picked from property files]



Ara Bundle - Start a timertask which puts message into a login channel.

This channel is exposed as a service which is imported by PamClient Channel

Ara bundles defines its own thread pool and receives connectionFactory service from switchboard bundle.



PamClient Bundle - Listens to local events from ara bundle and remote events from Pamserver bundle

Imports login channel service from Ara bundle

Uses common thread pool defined in switchboard bundle

[Property file has values which can point each topic or queue to be hosted on a specific Jboss server]



PamServer Bundle – generates events on regular time intervals which will be put into Jms queue using jms adapters.

Also tested a use case where we could change connection factory without getting down the server

AOP - AspectJ,Spring - Compile Time ,Load Time ,Run Time(Proxy)

Spring AOP - Runtime - Using Proxy

Steps

1.Spring Application Context - Bean Declarations

For enabling this proxy approach add
<aop:aspectj-autoproxy />

Note :

Do not be misled by the name of the element: using it will result in the creation of Spring AOP proxies. The @AspectJ style of aspect declaration is just being used here, but the AspectJ runtime is not involved

Aspect Declaration
<bean id="myAspect" class="com.test.aop.MyAspect"></bean>

Method where aspect will be applied
<bean id="test" class="com.test.aop.Test" init-method="init" />

Wrapper calling test
<bean id="testB" class="com.test.aop.Wrapper" init-method="init">
<property name="test" ref="test"></property>
</bean>


2. Classes

• Aspect Class

@Aspect
public class MyAspect {

@Before("execution(* com.test.aop.Test.*())")
public void myMethod()
{
System.out.println("calling my before aspect");
}

}

• public class Test {

public void init()
{
System.out.println("init in test method");
}

}

• public class Wrapper {

private Test test;

public void setTest(Test test) {
this.test = test;
}

public void init() {
test.init();
}

}

3. OutPut(In Spring DM)

System.out I init in test method
System.out I calling my before aspect
System.out I init in test method

Only when the init method is called through testB bean ,the aspect is bound.

Now when init method of test bean is called , the aspect is still not bound.(init methods cannot be proxied).

This is because the way spring life cycle works :

Spring life cycle

Phase 1 - Validate bean definitions
Phase 2 - Bean definition post processor(replacing property values)
Phase 3 - Bean Instantiation
Phase 4 - Bean post initialization(setters and init methods -in order) & Proxy creation

In the above example bean 'test' first cycles through phase 1,2,3 and 4. Hence after the init method is called, the proxy is created.But when testB is created in its init(phase) call to bean test will call teh aspect (as bean test lifecycle is complete)

One more important thing to note is that Cg lib will be used to create proxy here as teh class doesnt implement any interface.(Else jdk dyanmic proxies would have been used)

Aspect-J Load Time Weaving
(This may have issues with Spring DM
http://forum.springsource.org/showthread.php?t=80504
)

Same code as above except some change in the configurations.

1. Replace <aop:aspectj-autoproxy /> with <context:load-time-weaver/>.

2. Add aop.xml in META-INF directory

<!DOCTYPE aspectj PUBLIC
"-//AspectJ//DTD//EN" "http://www.eclipse.org/aspectj/dtd/aspectj.dtd">
<aspectj>

<!--works without this -->
<weaver>
<include within="com.test.aop.*" />
</weaver>

<aspects>
<!-- weave in just this aspect -->
<aspect name="com.test.aop.MyAspect" />
</aspects>

</aspectj>


3. Class to test the application in standlone mode

public class StandAloneLoadTimeWeaving {

public static void main(String args[])
{
ApplicationContext ctx = new ClassPathXmlApplicationContext("config.xml",StandAloneLoadTimeWeaving.class);
Test t =(Test)ctx.getBean("test");
t.init();
}

}

Ref: http://www.springbyexample.org/examples/aspectj-ltw.html
http://static.springsource.org/spring/docs/2.5.x/reference/aop.html#aop-aj-ltw


4. Add the following parameter when running

-javaagent:spring-agent.jar


Aspect J Compile Time Weaving(Build Time)

Same as Loadtime weaving needs aop.xml (but no need to add spring agent while running)

Command to compile the code

java org.aspectj.tools.ajc.Main –inpath <src_code_directory> -outjar <output_jarname.jar>

Ex:java -cp aspectjtools.jar;aspectjrt.jar org.aspectj.tools.ajc.Main -inpath . -outjar test.jar

The output jar will have code which is instrumented by the aspect.

You can download aspectj from this site

http://www.eclipse.org/aspectj/downloads.php

TCCL in OSGi(Spring DM)

In order to better understand how TCCL works in OSGi i have done a sample POC.

I created 3 bundles.

Bundle A - calls Bundle B (imports Bundle B)

Bundle B -calls Bundle C (imports Bundle A)

Bundle C - instantiates class in Bundle A (No imports)

In normal cases Bundle C will fail to instantiate class in Bundle A, but this can be achieved using TCCL as follows.

ITest test=(ITest)Thread.currentThread().getContextClassLoader().loadClass("com.bundleA.external.ActionClass").newInstance();

Conclusion: Spring Dm sets TCCL to the bundle classloader where the thread initiated.

Now what if we want to instantiate class of bundle B in Bundle C , but the thread flow doesnt change. ?

This can be easily done by resetting the TCCL in bundle B's method as follows

Thread.currentThread().setContextClassLoader(Bundle2.class.getClassLoader());

BootClassPath and Sytem ClassPath in OSGI(Spring DM)

System Package

java.* can be directly used without any imports. This is neither part of boot delegation nor system packages as they are part of jdk itself.For loading these fundamental types, the OSGi platform uses parent delegation instead of the networking model; that is, it uses the class loader that started the OSGi framework to load classes instead of an OSGi class space.

None of the other public packages available in the JDK such as the javax.net or javax.xml are parent delegated which means they have to be resolved within the class space that is, bundles need to import the packages (which means there needs to be a provider) since they are not implied. The OSGi spec allows the Framework (through its system bundle) to export
any relevant packages from its parent class loader as system packages using the org.osgi.framework.system.packages property.

As repacking the hosting JDK as a bundle isn't a viable option, one can use this setting to have the system bundle (or the bundle with id 0) export these packages itself.

Extension Bundles

Another possible option is to enhance the system bundle through extension bundles. These act as fragments; they are not bundles of their own but rather are attached to a host.
Extension bundles are a special kind of fragments that get attached only to the System bundle in order to deliver optional parts of the Framework

Boot Delegation

This allows the user to create 'implied' packages that will always be loaded by the framework parent class loader, even if the bundles do not provide the proper imports:

This option is mainly created to accommodate various corner cases especially in the JDK classes that expect parent class loading delegation to always occur or that assume that every class loader on the system has full access to the entire boot path.There are cases where boot delegation is quite handy; a good candidate being instrumentation, such as profiling or code coverage.

Ref:http://blog.springsource.com/2009/01/19/exposing-the-boot-classpath-in-osgi/

I have done samples to better understand how classloader comes into picture for loading classes at different levels by printing parent classloaders upto 4 levels.

Class loader for one of the classes of bundle in pick up.

bundle classloader ServerBundleClassLoader: [bundle=ClassPathTest_1.0.0]
bundles parent java.net.URLClassLoader@126e85f
bundles parent sun.misc.Launcher$AppClassLoader@7d772e
bundles parent sun.misc.Launcher$ExtClassLoader@11b86e7


Class loader hierarchy for org.osgi.framework.BundleActivator

**************System bundle 0*******************
bundle classloader java.net.URLClassLoader@126e85f
bundles parent sun.misc.Launcher$AppClassLoader@7d772e
bundles parent sun.misc.Launcher$ExtClassLoader@11b86e7
I bundles parent null


Created a dummy class and added this in the classpath of dm server startup.bin.Also modified to include this java6-server.profile as org.osgi.framework.system.packages = \
(I did import this package in my test bundle)

*************SYSTEM PATH**************
bundle classloader sun.misc.Launcher$AppClassLoader@7d772e
bundles parent sun.misc.Launcher$ExtClassLoader@11b86e7
bundles parent null
I parent null


Created a dummy class and added this in the classpath of dm server startup.bin.Also modified to include this java6-server.profile as org.osgi.framework.bootdelegation = \
(Did not import this package in my test bundle)


*************BOOT PATH**************

bundle classloader sun.misc.Launcher$AppClassLoader@7d772e
bundles parent sun.misc.Launcher$ExtClassLoader@11b86e7
I bundles parent null
I parent null

Configuring Component based logging in OSGi using Log4j

Initialize reading of the log4j xml in any of the core bundles. Ensure that this bundle doesnt get bounced often.Configure the values to include log4j xml and reload delay.The init method can look something similar to the one below.

public void initialize() {
try {
try {
String logConfig = "D://temp//poc_log_config.xml";
long configReloadDelay = (long) (1 * 30 * 1000);
if (logConfig != null && (new java.io.File(logConfig).exists())) {
if (logConfig.endsWith("xml")) {
org.apache.log4j.xml.DOMConfigurator.configureAndWatch(
logConfig, configReloadDelay);
} else {
org.apache.log4j.PropertyConfigurator
.configureAndWatch(logConfig, configReloadDelay);
}
} else {
org.apache.log4j.BasicConfigurator.configure();
}
} catch (Exception initErr) {
initErr.printStackTrace();
}
} catch (Exception err) {
}

}


The log4j xml will look as follows

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">

<appender name="PRODUCT1" class="com.test.anyappender">
<param name="File" value="D:\\temp\\product1\\product1log.log"/>
<param name="MaxFileSize" value="100000000"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{HH:mm:ss,SSS} - %m%n"/>
</layout>
<param name="Threshold" value="DEBUG"/>
</appender>

<appender name="PRODUCT2" class="com.test.anyAppender">
<param name="File" value="D:\\temp\\product2\\product2.log"/>
<param name="MaxFileSize" value="100000000"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{HH:mm:ss,SSS} - %m%n"/>
</layout>
<param name="Threshold" value="DEBUG"/>
</appender>

<root>
<level value="OFF" />
</root>

<logger name="test1">
<appender-ref ref="PRODUCT1"/>
<level value="INFO" />
</logger>

<logger name="test2">
<appender-ref ref="PRODUCT2"/>
<level value="DEBUG" />
</logger>

</log4j:configuration>


Now in any bundle code you can access any component logger as follows

static final Logger TEST_LOGGER = Logger.getLogger("test1");

use TEST_LOGGER.debug("any debug message");

By the above approach we can just have a single log4j bundle in the repository and not only avoid issues occuring from static variables but also can take full benefit of log4j API's.

Extending OSGi telnet for custom commands

OSGi telnet command provided by equinox can be easily extended to implement custom commands.

First we need to write a class with which implements the custom commands.the methods should start with _ to be identified as osgi telnet commands.The getHelp command needs to be overwritten ,which will display the command when help is hit on the telnet console.

import org.eclipse.osgi.framework.console.CommandInterpreter;
import org.eclipse.osgi.framework.console.CommandProvider;

public class CustomCommands implements CommandProvider {
@Override
public String getHelp() {
return "***********CUSTOM COMMANDS*******************\n";
}

public void _testcommand(CommandInterpreter ci) {
ci.println("arg 1" + ci.nextArgument());
ci.println("arg 2" + ci.nextArgument());
}
}


Next step is to write a BundleActivator and register the command Provider service.

@Override
public void start(BundleContext ctx) throws Exception {
customCmd = new CustomCommands();
//created a instance of the implementation of our custom commands
ctx.registerService(CommandProvider.class.getName(),customCmd, null);

}

Dont forget to add the bundle activator class to the manifest. Manifest should include the following.

Import-Package: org.eclipse.osgi.framework.console;version="[1.0.0,1.0.0]",
org.osgi.framework;version="[1.4.0,1.4.0]"
Bundle-Activator: com.command.extender.CommandBundleActivator

Spring Training -Day 6

http://www.infoq.com/presentations/colyer-keynote-springone2gx

Telnet console 2401 – ssh shell -2402 in DM2 .

2.0 release will have both osgi console and dm shell (osgi console will be disabled by default)

Dm Kernel – lightweight – no web profile.

No plans to support EJB

Bundles in usr directory is not exposing service because – not in active state – need to get some code to move it to active state.

Bundles in pick up move to active state by default.

Import – org.osgi.framework – will not work , as jar is in lib directory .Work Around is to copy paste it into repository/bundles/usr.

Spring dm automatically creates a property for service by name – bean name.

Also application context itself is exposed as service – can be verified with osgi console

Logback – for logging in dm 2.0

http://logback.qos.ch/

In JVM failure - can be achieved using ranking attribute

Load time weaving – more complex when many bundles are there , so hardcoded in DM server - just below boot Classloader (this is not possible with aspectj being a bundle)

Require Bundle(used for Split packages)

Require bundle – all exported packages are imported

Deployment pipeline – Like wars and non osgi headers still works under spring DM because –this is processed by DM server to convert it into osgi complaint bundle - End result is visible in work directory

When Dm server starts all bundles are not seen in osgi console – They move to resolved state , only when required(looking to find required imports) they move to resolved state.

Osgi reference has bean name attribute which can be used as default filter.

Bundles have version , services have ranking – Service ranking is part of OSGi spec

Custom osgi commands – implement Command Provider – equinox specific

2.0 modules – has attribute - sticky for service.

Spring integration

Enterprise Integration Pattern –implementers are SI and Camel

http://www.ibm.com/developerworks/library/os-ecl-osgiconsole/index.html

Spring Training -Day 5

Create new platform (may be equinox 3.5) – eclipse may be using 3.4

Eclipse takes ini file – which when bundles added to a folder adds it to the file –Eclipse can do it easily.

Uses conflict may be ClassCastExceptions or Linkage Exceptions

http://blog.springsource.com/2008/11/22/diagnosing-osgi-uses-conflicts/

Uses is whatever the bundle is resolved to .(Ex If struts has import;log4j and in any Export blabla;uses log4j – no issues because it will resolves to log4j in applications classpath just forcing us to use the same log4j.. When uses is with version becomes very critical)

Ideally 3rd party guys should not be ideally using uses –unless they want to restrict , but still relaxed using import in struts lib without version – so whatever 3rd party resolves to in our lib.

Import without version no – resolves to latest.

Class loader performance may increase -steps reduced for loading from the conventional model.

Osgi command – packages - searches for the package in all bundles.

BundleContext- used to install/uninstall bundles, deal with events and services programmatically

When multiple services have same ranking – will be solved based on service id – least one – the first one registered.

Extender bundle

Spring extender bundle creates bundle context for bundles which are in ACTIVE state . May be this is the reason bundles in usr lib cannot exposes services.(because they don’t move to active state)

Application context for bundles are created asynchronously when DM server starts up. It can be changed to get a sync behaviour.

When synchronous, it does not spawn a new thread for creating context and waits until context is created.(If any bundle is picked up before by the extender that may run in a separate thread)

Appcontext fails – it uninstalls the bundle – hence cannot verify errors from osgi console.

Service mandatory wait is asynchronous.

Extender Bundle – is like SpringContextLoadListener in web Apps

Extender bundles creates appContext for all bundles which are in ACTIVE state.

Accessing BundleContext – implements BundleContextAware

Either put appcontext file in META-INF / spring or use Spring-Context header in manifest file.

Mandatory Service

Timeout

Cardinality - 1:1 ,0:1, 0:n,1:n – renamed as availability attribute in Dm 2.0

osgi: set and osgi:list

Consumers can listen to bind-unbind events

Publishers can register to register and unregister events.

RFC 124 – Moving towards component and service than spring specific osgi:service and bean .

Spring Training - Day 4

Spring Security

http://blog.springsource.com/2009/01/02/spring-security-customization-part-1-customizing-userdetails-or-extending-grantedauthority/

http://blog.springsource.com/2009/01/02/spring-security-customization-part-2-adjusting-secured-session-in-real-time/

Principal –User,Device or system that performs action

Authentication –Establishing principals credentials are proper.(LDAP ,Database etc)

Authorization –Deciding if a principal is allowed to perform an action.

Secured Item – resource being secured.

Spring security is Portable across containers.

Secure item – method , web page(url) or flow(web flow)

Voters – 2 default- role based and authenticate based

3 types of AccessDecision manager (judges)

1. AffirmativeBased (anyone says yes –it’s a yes) -Default

2. Unanimous based – very conservative – one voter says no – it’s a no

3. Consensus based – based on majority says yes – majority is configurable – all these are configurable



Configuration in Web Application

<security:http> - configures default accessmgr, default filters, voters, security interceptor etc

Configures a servlet filter in web.xml

<security intercept-url pattern=”…” filters=”” acess=””/> can be configured

Spring security Taglibs since spring 2.0

Method Security(based on AOP)

Xml based - point cuts in xml with access role

Configuration based – have to register to get this enabled. , also has JSR specific annotations

Bad to put security constraints inside class – use xml approach over annotations – security is infrastructure

Advanced Security In web –

HttpSessionContextIntegrationFilter – stores context in session - if exists gets it and binds to thread local –way back binds it to session

LogoutFilter –clears security context on logoff

AuthenticationProcessingFilter - If not authenticated then send to login page

ExceptionTranslationFilter – converts exception into http responses and redirects

FilterSecurityInterceptor - authorises web requests based on url patterns

Any of these can be replaced and any can be added in the required order.

So filters for all these purposes are already placed.

Security context can be obtained anywhere in the code using – spring class SecurityContext.getCtx();

Obfuscating – is just setting bean – to avoid knowing technology




Spring Remoting

Consistency across all protocols and flipping from one protocol to another is simple.

Supported Protocols

RMI
HttpInvoker – is a protocol –over http
EJB
Hessian/Burlap(for sending XML over HTTP –Less predictable with complex types)

Hessian –uses binary xml
Burlap –uses textual xml

Spring on Server & Client –Use Http Invoker
Java Env but no Web Server –Use RMI
Interop with other languages Using HTTP – Hessian

Spring JMS

Message,Destination,Connection,Session,MessageProducer,MesageConsumer

Spring JMSTemplate

DynamicDestinationResolver

JMSTemplate – takes connectionFactory,MessageConvertor,DestinationResolver,

Spring JMX

MBean is a standard java bean with an additional management interface – attributes (with getters and setters) and Operations

Rmi is default protocol – can be changed

JVM runs a default MBean server , we can also override this by one given from spring

<context mbean –export>

Default –export everything – mostly a bundle will have only mbeans exposed

Export bean as MBean.

Strategies –

objectNamingStratergy- keyNamingStratergy(Default) , IdetitynamingStratergy(naming by objected from JVM),MetaDataNamingStratergy (annotations) – not recommended

One strategy can be overriden by the other.

MBeanInfoAssembler- by implementations of the MBeanInfoAssembler interface

To run with JMX add -Dcom.sun.management.jmxremote as VM param


OSGI

Single Version mean atleast that version

Resolves to the greatest -

Running in eclipse

Added vm param for clean

In manifest given space for the next line – because its reserved for headers

Import package , not mentioned – binds to the latest

Resolved phase- doesn’t mean classes are loaded

When classloader gets GC’ed - when objects are Gc'ed and all bundles importing from this bundle are refreshed(osgi refresh command)


Using Eclipse to simulate Equinox

Run configurations - > OSGi Configurations – new - Uncheck target platform - In filter search org.eclipse.osgi(any version ) – select it – apply – run

ss will show only equinox bundle.

Change run config as - -Dosgi.clean=true

Create new project – plugin – new plugin project - select run environment as equinox

Can run this new project under equinox with the created run configurations – Run as ...

Issues with loading Native libraries in OSGi Bundles

The Java virtual machine (JVM) allows only one class loader to load a particular native library.
(which means two bundle cannot load same native library)

There is no application programming interface (API) to unload a native library from a class loader.

Native libraries are unloaded by the JVM when the class loader that found the library is collected from the heap during garbage collection.

If a shared library that is configured with a native library path is associated with an application, whenever the application is restarted or dynamically reloaded the application might fail with an UnsatisfiedLinkError indicating that the library is already loaded. The error occurs because, when the application restarts, it invokes the shared library class to reload the native library. The native library, however, is still loaded in memory because the application class loader which previously loaded the native library has not yet been garbage collected.

A native library is unloaded only when the class loader that loaded it has been garbage collected. When a bundle is uninstalled or updated, any native libraries loaded by the
bundle remain in memory until the bundle's class loader is garbage collected. The garbage collection will not happen until all references to objects in the bundle have been garbage collected, and all bundles importing packages from the updated or uninstalled bundle are refreshed. This implies that native libraries loaded from the system class loader always remain in memory because the system class loader is never garbage collected

Spring forum : http://forum.springsource.org/showthread.php?t=80063

Solution – load this library in a bundle which won’t be refreshed or set a flag which shoudl be checked when reloading any bundle.

The option of changing native code name may result in memory leaks, because multiple native code accumulate in the process space.

Only the JVM class loader can load a dependent native library.

For example, if NativeLib1 is dependent on NativeLib2, then NativeLib2 must be visible to the JVM class loader. The path containing NativeLib2 must be specified on Java library path defined by the LIBPATH environment variable.

If a native library configured in a shared library is dependent on other native libraries, the dependent libraries must be configured on the LIBPATH of the JVM hosting the application server in order for that library to load successfully.

Ref: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.express.doc/info/exp/ae/tcws_sharedlib_nativelib.html

Service Monitoring - OSGI

There are 2 approaches for doing this.


1. Class implementing Service Listener gets all service lifecycle Events .

public class PGServiceListener implements ServiceListener, BundleActivator {
……..

@Override
public void start(BundleContext cntxt) throws Exception {
cntxt.addServiceListener(this);

}

@Override
public void stop(BundleContext cntxt) throws Exception {
// TODO Auto-generated method stub

}

public void serviceChanged(ServiceEvent event) {
-à this is called for every service life cycle event
}


}

Here this class will be mentioned in the manifest file as a bundle Activator

2. Listening to the market Interface . Assuming that all the services implement a interface IService we can have the following listener which get all service related events.

<osgi:list id="listService"

interface="com.partygaming.service.monitor.external.IService"

cardinality="0..N">

<osgi:listener ref="serviceListener1" bind-method="bindService"

unbind-method="unBindService"></osgi:listener>

</osgi:list>



Looks like the second approach is better, as we get only 2 events, service register and unregister which we are interested in .It also gives all the properties of the service which will help us maintain other info like version.

Distibuted OSGi

DOSGi helps in interaction between services which exists in different services in the same way as when they are co-hosted.

Specifications for DOSGi: http://www.osgi.org/Specifications/HomePage

Well Known implementations are

Apache CXF implementation : http://cxf.apache.org/distributed-osgi.html

Eclipse ECF implementation : http://wiki.eclipse.org/Distributed_OSGi_Services_with_ECF

Apache Tuscany implementation


Apache CXF implementation
-------------------------

http://cxf.apache.org/roadmap.html : Nothing more other than web services even in roadmap.

DOSGi Webinar : http://download.progress.com/open/adobe/prc/psc/042809_fuse_osgi_webinar/index.htm

Some answers to the questions on the webinar

http://coderthoughts.blogspot.com/2009/05/questions-from-rfc-119-webinar.html

Exposing service -has to mention the interace which can be remotely invoked.

Based on this Apache CXF creates a wsdl - which can be called by any web service client

(RFC 119) complaint discovery service - http://cxf.apache.org/dosgi-discovery.html
---------------------------------------
If discovery service is present - it will aurmtically detect , else we need to statically configure end points.

To enable automatic hooking of these remote services we need to have Service Registry Hooks -OSGi 4.2 spec(RFC 126)

Service Registry Hook Functionality(RFC 126 -OSGi spec 4.2) -Should be part of OSgi Core

Service Hook Functionalities
----------------------------

1. On Demand Registration of remote Services

2. Limiting Service Visibility

3. Service Proxification




How does it work [Refer Attached Diagram]
-----------------
Interface bundles are present in both the interacting containers.

Both teh containers have RFC 119 complaint discovery service and RFC 126 complaint registry hook

Service implementer registers with teh service with local OSGi container.

If this service has to be remotely accessed it needs to mention the property of remote interface.

Once this interface is mentioned the Discovery Service comes into picture which creates a remote end point for the service

It also register some meta data info with teh discovery service.

On the client side(in another Vm) when it expresses its interest in consuming this service , this information is passed
from the local osgi registry into the Registry hook which consults the discover service.

Now the discovery service will see if any one has remotely registered and if so creates a proxy for this service which will be reffered by the local osgi registry.

Current Apache CXF doesnt have either discovery service nor registry hook , hence end points need to be statically mentioned.

Apache is working on a discovery service under withthe ZooKepper Project

http://hadoop.apache.org/zookeeper/

http://wiki.apache.org/hadoop/ZooKeeper/ProjectDescription


[I think ZooKeeper is an alternative to Jgroups for sharing registry information between two nodes]

Working Demo is present in the link I have mentioned above, but without discovery service or service registry hook and it has a web service based implementaion for communication.

So to use Apache CXF we need minimum of these version - Equinox 3.5 / Felix 1.4.1 or later

Dmversion of equinox currntly- Dm 1.0.2 uses equinox 3 and 2.0 version uses equinox 3.5

Eclipse ECF implementation
----------------------------

http://wiki.eclipse.org/Distributed_OSGi_Services_with_ECF

http://eclipsesource.com/blogs/2008/12/19/rfc-119-and-ecf-part-1/

http://wiki.eclipse.org/Getting_Started_with_ECF%27s_RFC119_Implementation

Here changing the protocol for communication between services in diffrent VM's looks very simple

They claim to support the following protocols

Distribution

* r-OSGi (http) -r-OSGi is based upon a TCP-based protocols
* XMPP
* ECF Generic
* Skype
* Java Messaging Service (JMS)
* JavaGroups

Ways to get Spring ApplicationContext -In Spring DM

Couple of other ways to get hold of ApplicationContext (rather than doing new ClasspathXmlApplicationContext()).

1. Create a bean which is ApplicationContextAware


MessageChannel msgCh = (MessageChannel)ApplicationContextHolder.getApplicationContext().getBean("eventChannel");

msgCh.send(msg);


public class ApplicationContextHolder implements ApplicationContextAware {

private static ApplicationContext appContext;

public void setApplicationContext(ApplicationContext ctx)

throws BeansException {

System.out.println("*** ApplicationContextHolder::setApplicationContext()...");

ApplicationContextHolder.appContext = ctx;

}

public static ApplicationContext getApplicationContext() {

return appContext;

}

}


This one is in the case of Spring DM

2. Accessing the ApplicationContext from BundleContext (By default every bundle publishes the Application Context as a Service).



public static ApplicationContext getApplicationContext() {

if (appCtx == null) {

try {

ServiceReference[] svcRef = bundleContext.getAllServiceReferences("org.springframework.context.ApplicationContext", "(org.springframework.context.service.name=ContextInClass)"); // Service name is the Bundle Symbolic Name

ServiceReference ref = svcRef[0];

appCtx = (ApplicationContext) bundleContext.getService(ref);

} catch (Exception e) {

e.printStackTrace();

}

}

return appCtx;

}

This is the preferred approach if you wish to use the Application Context from another bundle.

Thread Context ClassLoader / Buddy ClassLoading in OSGi.

Bundles like Struts,hibernate,Spring need to instantiate classes in custom application bundle.In normal cases this can be directly done because there is a single class space(single class loader).

Consider a case where there are more than one class loaders .Lets take a simple case in which the struts/spring jar is kept under tomcat common lib and your application classes are under the web-inf lib. These two are jars now are loaded by two different classloaders.
Tomcat common lib class loader being the parent of tomcat web-inf/lib class loader.Also classloader of struts jar cannot see application classes in web-inf lib.

Now how can struts/spring see application classes ?

To solve such problems java Java came up with a solution called Thread Context Class Loader(TCCL). Thread context class loader was introduced in Java 2 to enable frameworks running in application servers to access the "right" application class loader for loading application classes.

This is accessed using Thread.getCurentThread.getContextClassloader().Its as simple as, It is the classloader associated with each thread.There are setter methods which can be called to set this TCCL.

By the definition in tomcat the TCCL is set to the context class loader from where the thread begins.So TCCL has the classloader visibity of web-inf lib,solving our problem.

Now since the thread started from that application ,these third party jars can use TCCL to instantiate application classes. The control starts from our application bundle and then flow moves to some third party bundle like struts or hibernate,which internally in their
source code use TCCL to instantiate our classes.

TCCL becomes slightly more complicated in the case of OSGI.

I have a small POC for the Reflection based Class loading using TCCL.

Source Code : http://www.4shared.com/file/123739699/3b6dd30e/TCCLosgi.html

Bundle 3(Core bundle) – has reflection code to instantiate application classes. Uses TCCL to instantiate classes.

Bundle 2 (Service) –Using the core bundle to instantiate its classes (imports bundle 3 classes directly)

Bundle 1 (Application) – uses the OSGi service provided by bundle 2.


Control Flow:

Bundle 1 makes a OSGi service call to bundle 2.

Bundle 2 now uses core to instantiate its(bundle 2) classes.

Result: ClassNotFoundException

Reason:

Bundle 2 is able to instantiate classes only in Bundle 3 and not in Bundle.

This is because TCCL has visibility only to class loader from where the thread started.

Solution:

1. PAR(Spring DM solution)

Spring Dm comes with a concept of PAR,which is grouping of bundles.All bundles inside the PAR will contribute together to form a synthetic context.And hence bundles inside it have visibility to all others exisiting inside it.

http://blog.springsource.com/2008/05/02/running-spring-applications-on-osgi-with-the-springsource-application-platform/

2.Buddy Class Loading (this is equinox specific solution and not part of OSGi spec)

http://wiki.eclipse.org/index.php/Context_Class_Loader_Enhancements#Buddy_Class_Loading


Now bundle 3, that is the core bundle can be a buddy to bundle 2 (service bundle).

Syntax

In bundle 3 manifest we need to mention Buddy-Policy :registered

In bundle 2 manifest we mention Eclipse-RegisterBuddy : bundle symbolic name


Now at run time we can add new slots(Bundle 4) which registers itself as a buddy to bundle 2(core) ,without affecting any other bundles.

3. We can set the TCCL in bundle 2 to bundle class loader when the deligation goes through it.I guess TCCL may not be the right approach(not good to tamper the class loader prior to making calls across bundles).

We can choose a solution based on the requirement.

Cyclic Dependency in OSGi & Dynamic Imports

Since in OSGi we have multiple class loaders as compared to single Class Loader case ,there may be occur the problem of cyclic dependency.

This cyclic dependency may not be compile type instead most of the cases its runtime. Some piece of code may try to instantiate classes in another bundle using reflection.

Consider such a example

Bundle A requires Bundle B at runtime and vice versa.

Bundle A has to mention import in its manifest for bundle B classes and Bundle B has to do the same for Bundle A classes.

Now in such case when Bundle A is deployed it says class not found for Bundle B classes,hence to solve this we need to write

Import Package : package-from-BundleB;resolution=optional

Now Bundle A doesn't give error . Follow with deployment of Bundle B. Later when bundle A tries to access classes in Bundle B it says ClassNotFoundException.

This is because when we mentioned resolution=optional , it doesn't try to link again when class is requested. And the linking happens only initial time.

To solve this we need to use Dynamic imports in such cases.

This cyclic dependency in most of the cases can be solved using Thread Context Class Loader or Eclipse Buddy Class Loading(refer next blog for details),and there will rarely occur a case to use dynamic imports.

Example: Spring Integration + Spring DM OSGi + Spring Remoting

I have just come up with a example for Spring integration along with OSgi.

I basically have 4 bundles

1. Exchange bundle which has definitions of channels

This bundle is spring integration aware, and hence has channel definitions in it.

Channel definition.

<publish-subscribe-channel id="login"/>

This channl is wrapped around a gateway and exposed as a OSGi service.This is to ensure that the event generating bundle is unaware of spring integration apis

Wrapping channel around gateway

<gateway id="loginProxy" default-request-channel="login"
service-interface="com.pg.exchange.event.IEvent" />

Exposing gateway as OSGi service

<osgi:service id="loginChannel" ref="loginProxy"
interface="com.pg.exchange.event.IEvent" />


Also the channel is directly published as a OSGi service for listener listening to it.

<osgi:service id="loginAnnouncementsChannel" ref="login"
interface="org.springframework.integration.channel.SubscribableChannel" />



2. Login bundle - bundle which generate login event and publishes it using gateway.

Accepting the Gateway from the exchange bundle as a OSgi service

<osgi:reference id="eventPublisher"
interface="com.pg.exchange.event.IEvent"/>

This bundle generates periodic events using a spring timer task as shown below

<bean id="logineventGenerator" class="com.pg.ara.LoginEventGenerator">
<property name="eventPublisher" ref="eventPublisher"></property>
</bean>

<bean id="loginTask" class="org.springframework.scheduling.timer.ScheduledTimerTask">
<property name="period" value="10000" />
<property name="timerTask" ref="logineventGenerator" />
</bean>


<bean id="timerFactory" class="org.springframework.scheduling.timer.TimerFactoryBean">
<property name="scheduledTimerTasks">
<list>
<ref bean="loginTask" />
</list>
</property>
</bean>

Timer Task class generating events

public class LoginEventGenerator extends TimerTask {

IEvent eventPublisher;

public void setEventPublisher(IEvent eventPublisher) {
this.eventPublisher = eventPublisher;
}

@Override
public void run() {
System.out.println("login event published");
eventPublisher.login("Logged In");
}

}


3. Pam-proxy bundle is a bundle which listens to the event using OSGi refered pub-sub-channel

When the message is obtained it makes a spring remoting call uisng remoting adaptors

Consumed OSGi service

<osgi:reference id="loginAnnouncementsChannel"
interface="org.springframework.integration.channel.SubscribableChannel"/>


Activator which makes a remoting call when message is put on a channel

<si:service-activator input-channel="loginAnnouncementsChannel"
ref="exampleService"
method="testRemoting"
output-channel="responseChannel"/>


Channel which gets the spring remoting response
<si:channel id="responseChannel" />


Activator listening on the response
<si:service-activator input-channel="responseChannel"
ref="responseHandler"
method="handleResponse" />

ProxyFactory bean for making spring remoting call
<bean id="exampleService"
class="org.springframework.remoting.httpinvoker.HttpInvokerProxyFactoryBean" >
< property name="serviceUrl" value="http://localhost:8088/test/pamService" />
< property name="serviceInterface" value="com.pam.external.IExampleService" />
</bean>


Response handler for the spring remoting call
<bean id="responseHandler" class="com.pam.proxy.sample.ResponseHandler" />


All the above bean can be avoided and instead we can use http-invoker inbound and outbound adaptors as explained in spring integration reference guide.

But i did not want the spring remoting hosting side to be unaware of spring integration.

Also i did not want to put any java code in the pam-proxy bundle.


4. Pam web bundle publishing a spring remoting service

<bean id="exampleProxy" class="com.pam.ExampleService" />

<bean name="/pamService"
class="org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter">
<property name="service" ref="exampleProxy" />
<property name="serviceInterface" value="com.pam.external.IExampleService" />
</bean>



Source Code Location :http://www.4shared.com/dir/18401573/8c9d7e8d/springintosgiremoting.html