Monday, 13 November 2017

Scaling database connection pools using LazyConnectionDataSourceProxy


While working on a server side application we ran into a problem where every API invocation was taking a database connection out of the connection pool. This was due to our services being "transactional" through use of  Spring's @Transactional annotation.The connection was being held till the transactional method completed execution.

This is the default behaviour and fine for most of the cases. However, it was problematic for us for a couple of reasons

a) Most of the time Hibernate's second level cache were sufficient and there was actually no need to fetch the data from the database.
b) We were invoking some other micro-services in our ecosystem at the start of the flow and did not require database connection until those micro-services returned.

We explored for solutions and chanced upon Spring's LazyConnectionDataSourceProxy. It has the property that it defers fetching of actual connection from the pool until it is absolutely required. This meant that we could continue to use Spring's Transaction demarcation on our services and not pay a performance penalty unless there was a cache miss or we got to a point inside the transactional method where data had to be loaded from the database.

Do check it out as it can significantly improve the scalability of database connection pools.

Monday, 31 December 2012

How to get the best out of online classes

MOOCs and online education have been one of the most revolutionary phenomenon to hit the web in 2012 (perhaps, ever!!). There are quite a few (for-profit and non-profit) companies like Coursera, Udacity, edX etc. offering online learning opportunities. Most of them provide a platform for the renowned universities of the world to offer their courses online, while others develop their own content in niche areas. If you are a avid learner, committed to life long learning, or just want to pick up a necessary skill for your job, head over to these sites and chances are that you will find a course that fits your requirements. My rough estimate is that there are more than 400 courses offered by these sites with topics covering everything under the sun. If you are in a hurry to find one, go to class-central.com which shows an aggregated list of all courses, when they are about to start and all other details. Highly recommended!!.

I want to use this post to talk about my experience I gained during completing the first course on coursera. To be honest, I could not complete the first course I started with. I had started with Functional Programming in Scala. However, I un-enrolled quickly as I could not keep up with it. The course was tougher than I expected and pressure of my day job prevented me from spending quality time on it. I fell behind and quickly became disengaged. However, this was a huge learning experience, some of which I am mentioning below
  1. Read the course page carefully: This is the most important page related to the course. Go through it carefully to understand what the class is all about, what you can expect to learn from it, what the prerequisites are for this course, recommended readings and other general instructions. Usually, there is a small video about the course on the page and I recommend that you watch it.
  2. Understand the prerequisites: If you are missing most of them, chances are that you will struggle during the course. Consider filling the gaps before the course starts. Sometimes, the course contains material that will help you fill some of them, but its better that you have a head start atleast.
  3. Set aside quality time: You will need time to go through the course material, usually presented in form of short videos. You will also require time to complete the assignments, quizzes and tests. Usually the course page will mention how many hours are required per week. Take that with a pinch of salt. You might need much more if you are short on the prerequisites. There is no point in overburdening yourself and burning out midway.
  4. Participate in forums/form local groups/meetups: Feeling engaged (and remaining so) in an online course is one of the biggest challenges. Thankfully, every online course provider recognizes it and provided tools such as discussion forums to address this. Coursera and other providers provide access to tools to make forming local groups and doing meetups easier.Use them to the fullest to derive maximum advantage.
  5. Choose carefully: Lastly, choose your course carefully. If the course adds value to your personal and professional life, chances are greater that you will complete it.
I hope that you will find the above post useful.

EDIT : Laura Cushing has written an excellent post on getting most out of Coursera, read it over here.

Saturday, 31 December 2011

Getting started with Spring JMS integration

Here is a quick introduction on how to get started with Spring JMS integration. We are using an example of asynchronous email system to understand the concepts which is a common case of Spring-JMS integration.

In our example, our application will be generating email objects. Instead of sending them to a mail server directly, it will send them to a jms broker for it to be placed in a queue. Our receiver will pick this message from the queue and do the actual job of sending the email through a mail server.

Let us start setting up pieces of our example one by one. The entire source code used in this example can be downloaded in form of a maven project from this location. SpringSource Tool Suite was used to develop this example.

1) ActiveMQ
Before we can start our producers/receivers, we need to start our jms broker first. We will using ActiveMQ for our example. Head over to ActiveMQ site and download one of the final releases. Unzip the download, go to bin directory and start ActiveMQ by running activemq.bat on Windows system (or activemq for *nix systems).When you start ActiveMQ, you will see output similar to that shown below. Pay particular attention to formatted line. This is the broker url and in this case it says that ActiveMQ is listening for connections on port 61616 on my local machine.


 INFO | For help or more information please see: http://activemq.apache.org/
 INFO | Listening for connections at: tcp://27411:61616
 INFO | Connector openwire Started
 INFO | ActiveMQ JMS Message Broker (localhost, ID:27411-51085-1325482616565-0:1) started
 INFO | jetty-7.1.6.v20100715
 INFO | ActiveMQ WebConsole initialized.
 INFO | Initializing Spring FrameworkServlet 'dispatcher'
 INFO | ActiveMQ Console at http://0.0.0.0:8161/admin
 INFO | ActiveMQ Web Demos at http://0.0.0.0:8161/demo
 INFO | RESTful file access application at http://0.0.0.0:8161/fileserver
 INFO | Started SelectChannelConnector@0.0.0.0:8161

Head over to http://localhost:8161/admin/ to access the admin console. Click on "Queues" and add a queue by entering "email.queue" in "Queue Name" textbox and clicking on "Create" button.

2) Message Generator (JMS Producer)
Our main application would be the source of the messages.  Given below is the spring bean configuration file for the producer

<beans>
    <bean id="connectionFactory" class="org.apache.activemq.spring.ActiveMQConnectionFactory">
        <property name="brokerURL" value="tcp://localhost:61616" />
    </bean>
    <bean id="emailQueue" class="org.apache.activemq.command.ActiveMQQueue">
        <constructor-arg value="email.queue" />
    </bean>
    <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionFactory" />
    </bean>    
</beans>

The beans are pretty much self explanatory.
"connectionFactory" : A factory that provides connection to the JMS broker. Pay attention to "brokerURL" property which points to the host:port combination pointed out earlier in ActiveMQ section.
"emailQueue": This bean represents the queue to which we want to send our Email message. Pay attention to the constructor-arg which specifies the queue name.
"jmsTemplate": This bean provider the Spring framework helper class that simplifies JMS access code.

The java code  which shows how to send Email messages to jms broker is shown below. The sample code uses jmsTemplate to send 5 Email objects to JMS Broker to be placed in the queue specified by "emailQueue" bean.

    ApplicationContext ctx = new ClassPathXmlApplicationContext(
                "applicationContext.xml");
        JmsTemplate jmsTemplate = ctx.getBean("jmsTemplate", JmsTemplate.class);
        Destination emailQueue = ctx.getBean("emailQueue", Destination.class);
        for (int i = 0; i < 5; i++) {
            final int j=i;
            jmsTemplate.send(emailQueue, new MessageCreator() {
                public Message createMessage(Session session)
                        throws JMSException {
                    Email email = new Email();
                    email.setToAddress("test@test.com");
                    email.setEmailSubject("Test Subject-"+j);
                    email.setEmailBody("Test Body-"+j);
                    return session.createObjectMessage(email);
                }
            });
        }


3) Synchronous Message Receiver

Given below is the code that implements a JMS Receiver that pulls the Email Messages that have been put in the queue by our sender above. Notice how this uses the same Spring configuration file used by our sender. Receiver loops continually checking for new messages. JMSTemplate.receive method will block till a message is available in the Queue. Also notice that we will encounter javax.jms.JMSException while reading the Email object from JMS message. To convert this into one of the Spring's unchecked exception, we are using Spring's JmsUtils.

        ApplicationContext ctx = new ClassPathXmlApplicationContext(
                "applicationContext.xml");
        JmsTemplate jmsTemplate = ctx.getBean("jmsTemplate", JmsTemplate.class);
        Destination emailQueue = ctx.getBean("emailQueue", Destination.class);
        for (;;) {// jmsTemplate.receive will block until a message is available in the Queue
            ObjectMessage msg = (ObjectMessage) jmsTemplate.receive(emailQueue);
            try {
                Email email = (Email) msg.getObject();
                System.out.println(email);
                // Send the email with Java Mail Api
            } catch (javax.jms.JMSException e) {
                // Catch checked JMSException and convert it into one of
                // unchecked org.springframework.jms.JmsException
                throw JmsUtils.convertJmsAccessException(e);
            }
            
            try {
            // Sleep for sometime before hitting the queue again
            Thread.sleep(10000L);
            }catch(InterruptedException ie){
                Thread.currentThread().interrupt();
                break;
            }
        }

4) Asynchronous Message Receiver

We can delegate the responsibility of monitoring the Queue to Spring framework. When a message is received, Spring can invoke any specified method that you have configured. Take a look at the Spring configuration file for this setup below. Notice jms:listener-container element and bean "emailReceiver"configured as a listener in it.

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:jms="http://www.springframework.org/schema/jms"
    xsi:schemaLocation="http://www.springframework.org/schema/beans    http://www.springframework.org/schema/beans/spring-beans.xsd
     http://www.springframework.org/schema/jms http://www.springframework.org/schema/jms/spring-jms-3.0.xsd">

    <bean id="connectionFactory" class="org.apache.activemq.spring.ActiveMQConnectionFactory">
        <property name="brokerURL" value="tcp://localhost:61616" />
    </bean>

    <bean id="emailQueue" class="org.apache.activemq.command.ActiveMQQueue">
        <constructor-arg value="email.queue" />
    </bean>

    <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionFactory" />
    </bean>
    
    <bean id="emailReceiver" class="edu.vpandey.spring.AsyncEmailReceiver"/>
    
    <jms:listener-container connection-factory="connectionFactory">
        <jms:listener destination="email.queue" ref="emailReceiver" method="emailHandler"/>
    </jms:listener-container>
</beans>

Whenever a message arrives in JMS queue "email.queue", method "emailHandler" is invoked on "emailReceiver" bean. Java code for AsycnEmailReceiver is shown below. As you can see it is just a POJO. Also notice the main method which simply loads the Spring config. Spring automatically creates threads that monitor JMS queues(or topics) and invoke configured methods on specified beans.

package edu.vpandey.spring;

import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

import edu.vpandey.spring.model.Email;

public class AsyncEmailReceiver {
    public void emailHandler(Email email) {        
        System.out.println(email);
    }
    
    public static void main(String[] args) {
        ApplicationContext ctx = new ClassPathXmlApplicationContext("applicationContext-asyncReceiver.xml");        
    }
}



Thats the end of this post. Please leave your comments/feedback in comments section. Dont forget to download the code from this location.

Tuesday, 27 December 2011

Setting up your personal website through Google Apps

So you want to setup your personal space on the great WWW? How about free email and hosting to go with it?  "Google Apps for free" comes to your rescue. You can use it to get free email, docs, calendar, appengine, blogger accounts on Google servers for you and your family.

Click here to get started. You can purchase a new domain name(at a cost of $10) or use your existing domain to setup Google Apps. Google has partnered with GoDaddy/eNom to provide domain registration service.

You can login into your domain dashboard by going to this url : http://google.com/a/<your-domain-name>. After you have logged in, you can create additional user accounts. You will find your domain registrar credentials at DomainSettings -> Domain Names -> Advanced DNS Settings
You might also want to change the urls of google services to more user friendly urls. One particular handy setting is "Catch-all address" (Settings-> Email->General) which allows you specify a default email address for mails sent to any non-existent email address on your domain.


You have multiple options to create your website
1) Google Sites : It is a tool provided by Google which makes creation of websites very easy. You can create a site through this tool and map it www.<your-domain-name>
2) Google App Engine :  You can deploy any application in GAE and map it to any url on your domain. You can deploy a CMS like Vasao as an application in GAE and use it as a starting point
3) Blogger :  If you just want to run a blog, you can point your blogger blog with a simple configuration change. You can also integrate your existing blogger blog as a website.

Hope you will find the above information useful.