Spring Training - Day 4

Spring Security

http://blog.springsource.com/2009/01/02/spring-security-customization-part-1-customizing-userdetails-or-extending-grantedauthority/

http://blog.springsource.com/2009/01/02/spring-security-customization-part-2-adjusting-secured-session-in-real-time/

Principal –User,Device or system that performs action

Authentication –Establishing principals credentials are proper.(LDAP ,Database etc)

Authorization –Deciding if a principal is allowed to perform an action.

Secured Item – resource being secured.

Spring security is Portable across containers.

Secure item – method , web page(url) or flow(web flow)

Voters – 2 default- role based and authenticate based

3 types of AccessDecision manager (judges)

1. AffirmativeBased (anyone says yes –it’s a yes) -Default

2. Unanimous based – very conservative – one voter says no – it’s a no

3. Consensus based – based on majority says yes – majority is configurable – all these are configurable



Configuration in Web Application

<security:http> - configures default accessmgr, default filters, voters, security interceptor etc

Configures a servlet filter in web.xml

<security intercept-url pattern=”…” filters=”” acess=””/> can be configured

Spring security Taglibs since spring 2.0

Method Security(based on AOP)

Xml based - point cuts in xml with access role

Configuration based – have to register to get this enabled. , also has JSR specific annotations

Bad to put security constraints inside class – use xml approach over annotations – security is infrastructure

Advanced Security In web –

HttpSessionContextIntegrationFilter – stores context in session - if exists gets it and binds to thread local –way back binds it to session

LogoutFilter –clears security context on logoff

AuthenticationProcessingFilter - If not authenticated then send to login page

ExceptionTranslationFilter – converts exception into http responses and redirects

FilterSecurityInterceptor - authorises web requests based on url patterns

Any of these can be replaced and any can be added in the required order.

So filters for all these purposes are already placed.

Security context can be obtained anywhere in the code using – spring class SecurityContext.getCtx();

Obfuscating – is just setting bean – to avoid knowing technology




Spring Remoting

Consistency across all protocols and flipping from one protocol to another is simple.

Supported Protocols

RMI
HttpInvoker – is a protocol –over http
EJB
Hessian/Burlap(for sending XML over HTTP –Less predictable with complex types)

Hessian –uses binary xml
Burlap –uses textual xml

Spring on Server & Client –Use Http Invoker
Java Env but no Web Server –Use RMI
Interop with other languages Using HTTP – Hessian

Spring JMS

Message,Destination,Connection,Session,MessageProducer,MesageConsumer

Spring JMSTemplate

DynamicDestinationResolver

JMSTemplate – takes connectionFactory,MessageConvertor,DestinationResolver,

Spring JMX

MBean is a standard java bean with an additional management interface – attributes (with getters and setters) and Operations

Rmi is default protocol – can be changed

JVM runs a default MBean server , we can also override this by one given from spring

<context mbean –export>

Default –export everything – mostly a bundle will have only mbeans exposed

Export bean as MBean.

Strategies –

objectNamingStratergy- keyNamingStratergy(Default) , IdetitynamingStratergy(naming by objected from JVM),MetaDataNamingStratergy (annotations) – not recommended

One strategy can be overriden by the other.

MBeanInfoAssembler- by implementations of the MBeanInfoAssembler interface

To run with JMX add -Dcom.sun.management.jmxremote as VM param


OSGI

Single Version mean atleast that version

Resolves to the greatest -

Running in eclipse

Added vm param for clean

In manifest given space for the next line – because its reserved for headers

Import package , not mentioned – binds to the latest

Resolved phase- doesn’t mean classes are loaded

When classloader gets GC’ed - when objects are Gc'ed and all bundles importing from this bundle are refreshed(osgi refresh command)


Using Eclipse to simulate Equinox

Run configurations - > OSGi Configurations – new - Uncheck target platform - In filter search org.eclipse.osgi(any version ) – select it – apply – run

ss will show only equinox bundle.

Change run config as - -Dosgi.clean=true

Create new project – plugin – new plugin project - select run environment as equinox

Can run this new project under equinox with the created run configurations – Run as ...

Spring Training -Day 3

Transaction Management

ACID-Atomic,Consistent,Isolated,Durable

Only way to handle transaction is hold on to the same connection –only way of doing it.

JTA – not good , scalability becomes difficult .

Ex: EBay – develop undo behaviour –compensatory action – 3rd part its the only option- for every method need a undo method.

Can use any datasource – no need to tieing it to spring jdbc datasource

Lots of transaction management – datatSourceTransactionMgr,HibernateTransactionMgr,Jps…,Jta..

Connection pool – use apache DBCP or c3p0

<tx:annotation driven> - registering for annotations

<bean id=”transactionManager” class=”…:> … datasource as property </>

transactionManager – should be the name – else have to explicitly mention in the <tx:annotation ……transaction manger=””.>

@Transactional at class level – all public method(because proxy sees only public method) will be transactional

Can have timeout – default is 60 secs

Datasource , transaction – better use xmls – infrastructure components (Aspects annotation are better)

Xml based approach - define transaction mgr, then define transaction interceptor- <tx:advice… > define pointcut

Tx:advice – is just used for overiding , not required

Xml approach advantage - can take these point cut definition from property files.

Set isolation levels – else though not committed , one thread can see the other

For recursive methods with @transaction – the same transaction context will be used.

Transaction Propagation – Propagation.REQUIRES_New – new one transaction context will be created per transaction.

So for iterative ones – if inner @Transaction is marked for required_new ,inner one will be committed irrespective of the other.

Can avoid rollback –with noRollBack option

Annotations
---------------

@Autowired –field level ,method level and constructor level

<context:annotation-config/>

@Autowired default by Type .-for all auto wired injections including and filed injections.

@Autowired
Test x; though x is name of any bean it will look for type Test.

@Autowired for field injection – Doesnt require setters.

Multiple types or no types will give error.

@Autowired not only for setter – for any method,field,constructor

Autowired for collections – List<Types> - all these types will be autowired into the list.

@Autowired(required=false) – does not fail when no type is found

@Qualifier – filtering when injecting by type – properties have to be mentioned in the bean
[default is name – when no name for bean is mentioned it is taken as id- That is why we say ctx.getBeanByName() and still use id it works]

@Resource –JSR 250 – injection by name – equivalent to ref in xml – not specific to Spring its in JSR 250

@Resource – also for JNDI references

@Component - alternative for writing bean definition in xml can be use with @Scope

@Component can also have bean name-else classname will be taken as bean name with first letter as small letter

<context:component scope> is required to identify components. – no need for annotation config as this is a superset of it.

<context:component scope> - can have filters – includes , excludes etc – without this Component will not work

Component scanner is above annotation processor. – so need to register annotation processor when using component scanner.

@value annotation will work with spring 3.0 – no support in spring 2.5

Both xml and annotation beans - if they have same name xml will override annotations , if the bean names are different 2 instances will be created.

Testing
-----------

Junit 4 – has annotations

Test runner - @Runwith (SpringJunit4ClassRunner.class)

@ContextConfiguration - with location of appContext.xml

Follow naming convention to avoid giving location - <testcasename>-context.xml

In case we need access to applicationCtx beans write private field and write @Autowired over it.

Name of bean if annotated with @Component –same as class name with first letter as small letter

With map it will put key as bean name when autowired.

@autowire with qualifier is very useful

Stubs and Mocks are different

Mock libraries - EasyMock,Jmock ,Mokito

Mockito: http://mockito.org/

Example with mockito – works by proxy- how to ensure method is called n number of times

During db tests – data inserted is automatically deleted , default can override if required.

SpringWeb
------------------

Web Layer should be very thin layer speaking to the backend service layer.

Spring Web Flow - Same url will be used , server maintains a state based on which flow will continue.

ContextLoadListener in web.xml - Root ApplicatonContext

WebApplicationContextLoader - register in servletContext – For WebApplicationContext

Using DispatcherServlet

2 contexts , RootApplicationContext and WebApplicationContext.

Web (DispatcherServlet) context can rootContext but not the other way round.

Spring MVC – annotation suited

WebApplicationContextUtils – static helper for accessing spring beans in web context.

Processing Flow -> Request -> Dispatcher Servlet -> Handler (Controller) -> Result View(send to DispatcherServlet) -> View

@Controller- equivalent to struts actions classes but can have multiple mapping for different url’s per method inside it.

@RequestMapping -for mapng urls

@RequestParam – mapping fields from HttpRequest.

Controller returns View name – default is path to jsp page

If controller returns null – then default view based on request url.

DispatcherServlet delegates to viewResolver to map returned view names to view implementations.

Default View resolver treats view name as Web –App relative path.

View Resolver – can take prefix and suffix as params.

If returned as string from handler – its taken as a view name

Spring Training -Day 2

Bean name will be generated by bean id. Context.getBean(name) hence works by id.

Writing custom annotation – write custom annotation and register pre-processor to add custom logic to it.

BeanPostProcessor- deals with objects – postInit & preInit

Interceptor- aspect with single advice[will be called when bean is created]

isSingleton method inside factory will be called only once if isSingletonMethod() set to true –Doesn’t make sense if factory itself is prototype

Inheritance – bean can be marked as

abstract=true - instance wont be created

Parent – parent=”” – for overriding

PropertyEditors- how properties are converted in xmls which always are mentioned as string ex – toDate,Integer etc –Spring has many of its own

We can register our own propertyeditors - For any type –call our propertyEditor – this is not like set of interceptors instead editor is picked from a map.

Following conventions - put customEditor in same package as code , and name should be <type>+Editor - no need to explicitly register the editor.

Import configuration file - <import resource=”ddsdsd.xml” />

P namespace – allows properties to be set as attributes in bean definition.

Util namespace – loading properties – diff from propertyplaceholder – util :properties , set,list,Map,reading constants

AOP
------

2 technologies

Spring Aop – Runtime –Using Dynamic Proxy

AspectJ- Compile Time OR Load Time –byte code injection

Performance – aspectj build time is best but complex.

For AspectJ : http://www.infoq.com/articles/Orchestration-Oleg-Zhurakousky;jsessionid=E64F684641DCF76CEC10633EFF108059

Spring Aop – uses (hides this complexity)

a. uses JDK dynamic proxy when interface is mentioned

Ref: http://java.sun.com/j2se/1.4.2/docs/guide/reflection/proxy.html

Method invocations on an instance of a dynamic proxy class are dispatched to a single method in the instance's invocation handler, and they are encoded with a java.lang.reflect.Method object identifying the method that was invoked and an array of type Object containing the arguments.

b. uses CGlib when no interface is mentioned

CGLIb can be forced in both cases using Proxy-target-class=false in definition.

Creating proxy using APIs - org.springframework.aop.framework.ProxyFactory;

Can add advice to the proxy – which recognizes either annotations or anything else

Adding aspectj weaving adds a context weaving classloader

Hierarchy --- BootClassLoader,Aop Context weaving Classloader,Extension loader,Application loader

The above happens for only Aspectj Based viewing

Hence all classes other than in boot path can be intercepted.

Delegation Of Classloader – first goes to parent if class def is not found and then comes down

When using jdk dynamic proxy –cast to interfaces

CGlib works by extending the class definition and then creating instance of that , old object is GC’ed.

Hence with final classes it throws error

Constructor gets called twice.- because extended.

Internal methods don’t get called.

JdkDynamic uses the same instance

JointPoint- execution context

PointCut - Expression

Advice-method

Aspect- class

Advice -@Before,@Afterreturning,@AfterThrowing,@After,@Around

Spring JDBC
---------------

No checked exception – all run time exceptions –rarely Db exceptions are recoverable

Callback pattern – give ur reference to some to call u at right times

Does from Getting connections, Participating in Transaction, Execution of statement, Processing ResultSet, Handling Exceptions, Closing connection.

Need to provide a datasource along with transactionManager. – Datasource manages the connection pooling

We can use Apache DBCP or C3po for pooling.

JdBCTemplate methos like queryForMap,queryForList

RowMapper – Each Row maps to a Doman Object
RowCallBackHandler – No return value
ResultSetExtractor - Multiple rows map to a single object.

Spring handles all exceptions irrespective of the vendor. It keeps a internal mapping of all error codes specific to the vendor, and gives very descriptive errors.

Spring Training -Day 1

Trainer : Oleg.zhurakousky@springsource.com

Spring deals with plumbing so that we can focus on domain problem.

We analyse what part Is business logic & what’s rest ?

Creation pattern - Strategy Pattern – Dependency Injection – Interface based.

Conventions over configuration – ex maven etc

Annotations – when dependency is mostly static , and also confusion can be solved using conventions.

Generic at compile time Vs runtime – This may be to support old Java runtimes

Implement FactoryBean – the getObject() – will give result whenever the bean is injected.

How to get factory itself - &<beanname> in property field.

FilePathXmlContex(“classpath:dhgadhs.xml”,c:/sdsd/sdsd/asds.xml); - classpath will ensure its searched in classpath for the first xml

Bean Life Cycle

BeanFactoryPostProcessor – just after dependency tree is constructed but before objects are created.Can change definition of any bean here.

One usecase would be propertyplaceholderconfigurer – to replace values with exact properties from file.

once configurer like placeholder is register it will be replaced with property files. - <context:property placeholder/>

<context:annotation-config/> - needs to be registered for annotations.

@Required - throws error if property is not set

Enforcing init method – by implementing initializingBean

Proxy – Decorator pattern

<bean class=”Org.springframework.beasn.factory.config.PropertyPlaceholderConfigurer” >

<property name=”location” value”…”/>

</bean>

With namespace – no need to remember the class

<context:property-placeholder location=””/>

Giving class as part of application context – helps identify where the xml is located

Bean Initilization (3 ways)

@PostConstruct ,init-method,implement interface initializingBean

Bean Post processor example - @Required – have to add contextannotation config

Destroying the application Context

ConfigurableApplicationContext.close()

Bean Destruction(3 ways)

@PreDestory,destroy-method,implement interface DisposableBean

Scopes – request and session makes sense only when they are in a web continer.

Issues with loading Native libraries in OSGi Bundles

The Java virtual machine (JVM) allows only one class loader to load a particular native library.
(which means two bundle cannot load same native library)

There is no application programming interface (API) to unload a native library from a class loader.

Native libraries are unloaded by the JVM when the class loader that found the library is collected from the heap during garbage collection.

If a shared library that is configured with a native library path is associated with an application, whenever the application is restarted or dynamically reloaded the application might fail with an UnsatisfiedLinkError indicating that the library is already loaded. The error occurs because, when the application restarts, it invokes the shared library class to reload the native library. The native library, however, is still loaded in memory because the application class loader which previously loaded the native library has not yet been garbage collected.

A native library is unloaded only when the class loader that loaded it has been garbage collected. When a bundle is uninstalled or updated, any native libraries loaded by the
bundle remain in memory until the bundle's class loader is garbage collected. The garbage collection will not happen until all references to objects in the bundle have been garbage collected, and all bundles importing packages from the updated or uninstalled bundle are refreshed. This implies that native libraries loaded from the system class loader always remain in memory because the system class loader is never garbage collected

Spring forum : http://forum.springsource.org/showthread.php?t=80063

Solution – load this library in a bundle which won’t be refreshed or set a flag which shoudl be checked when reloading any bundle.

The option of changing native code name may result in memory leaks, because multiple native code accumulate in the process space.

Only the JVM class loader can load a dependent native library.

For example, if NativeLib1 is dependent on NativeLib2, then NativeLib2 must be visible to the JVM class loader. The path containing NativeLib2 must be specified on Java library path defined by the LIBPATH environment variable.

If a native library configured in a shared library is dependent on other native libraries, the dependent libraries must be configured on the LIBPATH of the JVM hosting the application server in order for that library to load successfully.

Ref: http://publib.boulder.ibm.com/infocenter/wasinfo/v6r1/index.jsp?topic=/com.ibm.websphere.express.doc/info/exp/ae/tcws_sharedlib_nativelib.html

Garbage Collection - Log Analysis

25646.328 :[GC 325407K->83000K(776768K), 0.2300771 secs]
25646.328 : [GC 325816K->83372K(776768K), 0.2454258 secs]
25646.328 : [Full GC 267628K->83769K(776768K), 1.8479984 secs]


25646.328 –time since JVM started

325407K->83000K :combined size of live objects before and after garbage collection
After minor collections the count includes objects that aren't necessarily alive but can't be reclaimed, either because they are directly alive, or because they are within or referenced from the tenured generation.

(776768K) -> is the total available space, not counting the space in the permanent generation, which is the total heap minus one of the survivor spaces

0.2300771 secs –time taken

Here we see two minor collections and one major one.

Good Link:http://java.ociweb.com/mark/other-presentations/JavaGC.pdf


Sample GC logs:
25646.328: [GC [PSYoungGen: 53934K->1783K(55744K)] 139043K->86893K(521792K), 0.0091130 secs] [Times: user=0.02 sys=0.01, real=0.01 secs]
25650.658: [GC [PSYoungGen: 55735K->2232K(54208K)] 140845K->91675K(520256K), 0.0140890 secs] [Times: user=0.06 sys=0.00, real=0.02 secs]
[2009-11-03 10:02:23.092] fs-watcher Processing 'DELETED' event for file 'xxx.jar'. ? Application Undeployed
[2009-11-03 10:02:24.495] fs-watcher Undeployment of ‘xxx' version '0' completed.
[2009-11-03 10:02:48.592] fs-watcher Processing 'CREATED' event for file 'xxx.jar'.
[2009-11-03 10:02:52.915] fs-watcher Deployment of 'xxx' version '0' completed.
25705.888: [Full GC (System) [PSYoungGen: 416K->0K(53632K)] [PSOldGen: 122447K->122819K(466048K)] 122863K->122819K(519680K) [PSPermGen: 56927K->56927K(58496K)], 0.4994360 secs] [Times: user=0.50 sys=0.00, real=0.49 secs]
Total time for which application threads were stopped: 0.5127720 seconds

Approx 2.20 mins later after the redeployment the memory in OldGen starts piling up

33705.323: [Full GC (System) [PSYoungGen: 3584K->0K(51328K)]
[PSOldGen: 162686K->166028K(466048K)]
166270K->166028K(517376K)
[PSPermGen: 61486K->61486K(61824K)], 0.6978710 secs]
[Times: user=0.70 sys=0.00, real=0.70 secs]

33766.045: [Full GC (System)[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor305]
[Unloading class sun.reflect.GeneratedSerializationConstructorAccessor306]
[PSYoungGen: 3424K->0K(51456K)] [PSOldGen: 175451K->178649K(466048K)] 178875K->178649K(517504K) [PSPermGen: 61487K->61487K(61824K)], 0.7389690 secs] [Times: user=0.73 sys=0.01, real=0.74 secs]
36459.779: [Full GC [PSYoungGen: 41152K->0K(57664K)] [PSOldGen: 554317K->557726K(699072K)] 595469K->557726K(756736K) [PSPermGen: 61442K->61442K(61824K)], 1.6296560 secs] [Times: user=1.62 sys=0.00, real=1.63 secs]
Total time for which application threads were stopped: 1.6310400 seconds
Total time for which application threads were stopped: 0.0011620 seconds

After approx 3hrs 40mins after the redeployment, application starts doing Full GC every 2 secs virtually unable to collect anything. The GC pause times are 2secs. So application is not
running at this point.

38208.203: [Full GC [PSYoungGen: 42047K->40198K(53952K)]
[PSOldGen: 699068K->699068K(699072K)]
741115K->739266K(753024K)
[PSPermGen: 61376K->61376K(61568K)], 2.1202220 secs]

[Times: user=2.16 sys=0.00, real=2.13 secs]
Total time for which application threads were stopped: 2.1220930 seconds
38210.406: [Full GC [PSYoungGen: 42048K->38287K(53952K)] [PSOldGen: 699068K->699072K(699072K)] 741116K->737359K(753024K) [PSPermGen: 61376K->61369K(61568K)], 2.2874650 secs] [Times: user=2.28 sys=0.00, real=2.28 secs]
Total time for which application threads were stopped: 2.2889430 seconds
38213.678: [Full GC [PSYoungGen: 42043K->40523K(53952K)] [PSOldGen: 699072K->699072K(699072K)] 741115K->739595K(753024K) [PSPermGen: 61369K->61369K(61568K)], 2.1083830 secs] [Times: user=2.11 sys=0.01, real=2.11 secs]
756 MB -Xmx

Ref :http://java.sun.com/docs/hotspot/gc1.4.2/

Each blue box represents a collector that is used to collect a generation. The young generation is collected by the blue boxes in the yellow region and the tenured generation is collected by the blue boxes in the gray region.

"Serial" is a stop-the-world, copying collector which uses a single GC thread.

"ParNew" is a stop-the-world, copying collector which uses multiple GC threads. It differs from "Parallel Scavenge" in that it has enhancements that make it usable with CMS. For example, "ParNew" does the synchronization needed so that it can run during the concurrent phases of CMS.

"Parallel Scavenge" is a stop-the-world, copying collector which uses multiple GC threads.

"Serial Old" is a stop-the-world, mark-sweep-compact collector that uses a single GC thread.

"CMS" is a mostly concurrent, low-pause collector.

"Parallel Old" is a compacting collector that uses multiple GC threads.

Using the -XX flags for our collectors for jdk6,

UseSerialGC is "Serial" + "Serial Old"

UseParNewGC is "ParNew" + "Serial Old"

UseConcMarkSweepGC is "ParNew" + "CMS" + "Serial Old". "CMS" is used most of the time to collect the tenured generation. "Serial Old" is used when a concurrent mode failure occurs.

UseParallelGC is "Parallel Scavenge" + "Serial Old"

UseParallelOldGC is "Parallel Scavenge" + "Parallel Old"

Ref : http://blogs.sun.com/jonthecollector/entry/our_collectors



These 2 references clears most of teh doubts pertaining to jvm internals and gc tuning.

http://blogs.sun.com/jonthecollector/entry/our_collectors

http://www.nbl.fi/~nbl97/java/tuning/jvm_internals.pdf

Service Monitoring - OSGI

There are 2 approaches for doing this.


1. Class implementing Service Listener gets all service lifecycle Events .

public class PGServiceListener implements ServiceListener, BundleActivator {
……..

@Override
public void start(BundleContext cntxt) throws Exception {
cntxt.addServiceListener(this);

}

@Override
public void stop(BundleContext cntxt) throws Exception {
// TODO Auto-generated method stub

}

public void serviceChanged(ServiceEvent event) {
-à this is called for every service life cycle event
}


}

Here this class will be mentioned in the manifest file as a bundle Activator

2. Listening to the market Interface . Assuming that all the services implement a interface IService we can have the following listener which get all service related events.

<osgi:list id="listService"

interface="com.partygaming.service.monitor.external.IService"

cardinality="0..N">

<osgi:listener ref="serviceListener1" bind-method="bindService"

unbind-method="unBindService"></osgi:listener>

</osgi:list>



Looks like the second approach is better, as we get only 2 events, service register and unregister which we are interested in .It also gives all the properties of the service which will help us maintain other info like version.