|
|
A common use case is doing something immediately after booting up or prior to shutting down. Examples of these could be sending an email to yourself whenever a server is up, or perhaps starting a webserver.
I needed to do the following at boot time of a cloud machine that we acquire on demand.
- Download a war file from a certain location, place it it tomcat webapps folder.
- Start tomcat independently at boot time
- Send an email indicating that the server is up.
I created three separate scripts to do each of these. Lets look at how to do something like this in Linux. I am using Fedora 17 by the way.
Change directory to init.d like this.
Next, use vi or gedit to create a file. Call it whatever you want to. In my case, the script is called ‘update-scraper’
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
| #!/bin/bash
#
# chkconfig: 2345 95 12
# description: download and deploy update scraper
#
# Get function from functions library
. /etc/init.d/functions
# Download update scraper
start() {
cd /home/apache-tomcat-7.0.34/webapps
wget http://dl.dropbox.com/u/123456879/scraper-content-extraction.war
}
stop() {
cd /home/apache-tomcat-7.0.34/webapps
rm -f scraper-content-extraction.war
echo
} |
Save this file. I ll explain what the configurations are:
On line 3, you have chkconfig: 2345 95 12
2345 are basically specifying the ‘run levels’ on which this script can run. A run level is a mode of operation of an operating system. For example, a run level of 3 is ‘Multi-User mode, console logins only’. More on run levels here.
95 is the order in which the start function of this script will be called at boot time relative to other scripts.
12 is the order in which the stop function of this script will be called at shutdown.
In lines 10-13, we have a start function. This will be called during boot time. I am simply downloading a war file from a publicly accessible location and placing it in tomcat’s webapp folder. This way whenever the system (in my case an on demand cloud machine) boots up, it downloads the latest version of my application from a URL. Tomcat is started by a separate script.
In lines 15-18, we have the stop function which will be executed at shutdown. I am simply removing the war downloaded by the start function although it is not entirely necessary.
Next, we need to give rights to execute this script.
1
| # chmod +x update-scraper |
Finally, register the service/script with chkconfig.
1
| # chkconfig --add update-scraper |
To see if the registration was successful, check with the following command and update-scraper should be in the list.
Now reboot and test if your script works!
 
I saw someone reading this book at some airport a while back and the title seemed interesting enough. In this post, I will try and provide a brief review of what this book is all about. All in all, I will give it about 3.5 stars out of 5.
This book is written by George Friedman, founder of STRATFOR, a private intelligence and forecasting firm based out of Austin Texas. Apparently Friedman has a knack for predicting the future (primarily Geopolitical) without using the crystal ball. The book is written from an American perspective but the reasoning Friedman provides to support his claims makes his predictions seem plausible.
Friedman predicts that contrary to popular belief, the 21st century is the American century. He argues that the American military machine and economy is far too powerful to be overcome by any other country including Russia and China for at least the next 100 years. Here are some of the reasons quoted in the first chapter:
- Any nation that controls the world’s oceans is going to be a super power because all major international trade, especially oil, happens through sea routes. The United States Navy is incomparable to any in the world in terms of numbers, reach and sheer power.
- The US economy is the by far the largest in the world. In 2007, the US GDP was $14 trillion out of the world GDP of $54 trillion which is a massive 26% of the world economy. The economies of the next 4 countries (Japan, Germany, China and UK) combined still do not surpass the scale of the US economy. Even though over the last couple of decades, the American auto and steel industry has declined or moved overseas, America’s industrial output is double than Japan’s, the next in line. Even if we assume American economic dominance to be in decline, it is going to take a significant amount of time for anybody else to take over.
- As compared to China, Asia or Europe, America is largely uninhabited leaving vast areas of land available for agriculture.
Next, Friedman talks about America’s so-called “Long war on terror” and how it has and will America. I ll update this post with details on that soon.
In this multi-part post, I’ll show you how to integrate your spring mvc web application with Facebook and do all sorts of interesting things like accessing your data, your friend’s data, posting to walls among other interesting things. In this this post however, we will focus on a pre-requisite, which is allowing people to login into your application using their facebook accounts and asking them for various permissions while doing it.
Integration with Facebook can happen either at the client-side, where integration typically would happen via javascript directly from the page or it can happen at the server-side where server-side php/java/ruby code performs the integration. Since, we will be discussing in context of Java, Spring and Spring MVC, ours will be a server side implementation.
Registering a new application in Facebook
The first step of course is registering your application with Facebook. All you need is an account on facebook. You can register you application by going to the following URL https://developers.facebook.com/apps
 Registering an application on facebook
While there are lots of configurations you can make while setting up a Facebook application (things like setting up your logos, configuring an administrator email address etc) we will stick to configuring the most important one: the call back URL. In order to understand why this is important, we need to understand a typical OAuth flow which we will do in a bit.
 Setting up your facebook application
Understanding the typical OAuth flow. Case in point: Facebook
Accessing any API that is OAuth compliant typically follows the same 3 step process. This process is commonly referred to as the OAuth dance. The pre-requisite to the OAuth dance is registering the application/API consumer as we have already discussed above in context of facebook. The steps of the OAuth dance are:
- Use the application id and client secret to generate a authorization code
- Use the authorization code to generate an access token
- Use the access token to access the API
The terms application id, client secret, authorization code and access token are used by Facebook and maybe different for different implementers but the overall idea and flow would be similar if not exactly the same.
The code
I am going to paste the code here and then walk you through the OAuth dance. Be aware though. This is not production quality code. It is just to give you an idea of how things works. Tweak it according to your needs.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
| @RequestMapping(value = "/social/facebook")
@Controller
public class FacebookController {
private static final String SCOPE = "email,offline_access,user_about_me,user_birthday,read_friendlists";
private static final String REDIRECT_URI = "http://localhost:8080/platform-services/social/facebook/callback";
private static final String CLIENT_ID = "Your Client Id";
private static final String APP_SECRET = "Your App Secret";
private static final String DIALOG_OAUTH = "https://www.facebook.com/dialog/oauth";
private static final String ACCESS_TOKEN = "https://graph.facebook.com/oauth/access_token";
@RequestMapping(value = "/signin", method = RequestMethod.GET)
private void signin(HttpServletRequest request, HttpServletResponse response)
throws Exception {
try {
//TODO: if already have a valid access token, no need to redirect, just login
response.sendRedirect(DIALOG_OAUTH+"?
client_id="+CLIENT_ID+
"&redirect_uri="+REDIRECT_URI+
"&scope="+SCOPE);
} catch (Exception e) {
e.printStackTrace();
}
}
@RequestMapping(value = "/callback", params = "code", method = RequestMethod.GET)
@ResponseBody
private void accessCode(@RequestParam("code") String code,
HttpServletRequest request,
HttpServletResponse response) throws Exception {
try {
response.setContentType("text/html");
String responseString = IntegrationBase.readURLGET(ACCESS_TOKEN,
new String []{"client_id","redirect_uri","code", "client_secret"},
new String[]{CLIENT_ID,REDIRECT_URI,code,APP_SECRET});
response.getWriter().write(responseString);
response.flushBuffer();
} catch (Exception e) {
e.printStackTrace();
}
}
@RequestMapping(value = "/callback", params = "error_reason", method = RequestMethod.GET)
@ResponseBody
private void error(@RequestParam("error_reason") String errorReason,
@RequestParam("error") String error,
@RequestParam("error_description") String description,
HttpServletRequest request, HttpServletResponse response) throws Exception {
try {
response.sendError(HttpServletResponse.SC_UNAUTHORIZED, description);
System.out.println(errorReason);
System.out.println(error);
System.out.println(description);
} catch (Exception e) {
e.printStackTrace();
}
}
} |
Use the application id and client secret to generate a authorization code
According to the code above, you can see that we have the following service endpoints
- /social/facebook/signin
- /social/facebook/callback?code=x (where user gives permissions)
- /social/facebook/callback?error_reason=x&error=y&error_description=z (where user does not give permissions)
The service client calls /social/facebook/signin service. The call is received in the controller from lines 13-34. We simply redirect the HTTP call to
Let’s breakdown this URL:
https://www.facebook.com/dialog/oauth – This is the URL to which our application would redirect to. This will show the facebook login page for our users.
The URL takes in 3 query string parameters.
- client_id: This is the ‘App Id’ we got when we created our app in facebook (refer to the second image in this post)
- redirect_uri: Facebook calls this URL once user has successfully logged in and accepted or rejected permissions. The URL must be the same as mentioned in the ‘Site URL ‘ field while creating the app (again refer to the second image). If your redirect URL needs to have query string parameters, you don’t have to specify those in the ‘Site URL’ field as it only requires the base URL to be the same.
- scope: Using the scope parameter, you specify the list of permissions you require from the user. For example: ‘user_birthday’ which give us access to the users birth date he has entered in facebook. A list of permissions is available at https://developers.facebook.com/docs/reference/api/permissions/
Redirecting to the complete URL takes you to the facebook login page in case you are not logged in. After entering your facebook username and password, you get presented with the following pages asking you add your application and provide/reject requested permissions.
Clicking the ‘Go to App’ button opens up the following screen.
When you click on either Allow or Skip, facebook is going to make a HTTP call on the redirect_uri that we sent in as part of our redirect to the facebook oauth dialog.
If you allow the application the permissions it requires, facebook calls the redirect uri with an additional code parameter. In our controller, this corresponds to the function at Line 28 being called. Otherwise, the function at Line 46 is called with error, error_reason, and error_description parameters.
The code is the authorization code that is an intermediate code required to get the access token. From lines 33-37, we make an HTTP call (using our own class IntegrationBase) to the URL https://graph.facebook.com/oauth/access_token?client_id=<your-client-id>&redirect_uri=<the-url-facebook-will-redirect-to>&code=<the-authorization-code-we-received-earlier>&client_secret=<the-client-secret>
the redirect_uri must be the same as in the earlier call to facebook.
As a result of this call, if all is well and your code, along with other parameters are valid, you will get the a response similar to this:
access_token=<accesstoken>&expires=<number-of-seconds-till-this-token-expires>
This access token can now be used to access facebook data you required access to and asked permission for in the scope query string parameter sent earlier to facebook while asking for the authorization code.
In a later post, we will see how to post on a wall and get access to friend information among other actions using our access token.
More information about the Oauth dance can be found on the facebook developer site at http://developers.facebook.com/docs/authentication/
Configuring a connection pool for applications is a fairly common requirement however acquiring database connections during a request-response cycle is not a good idea as getting a database connection takes a fair bit of time. Try connecting to a database in debug mode and you will experience a significant delay. Connection pooling comes to the rescue by pre-building a number of connections, having the application use them, and have the application return them back to the pool after use. Now, if you were writing EJBs deploying in a jee container, pooling would be provided by the container. In our case, we use spring, hibernate and tomcat and have to manage pools using something like c3p0 or apache dbcp. I chose c3p0.
c3p0 is an easy-to-use library used for providing connection pooling capability. You can find more details here http://www.mchange.com/projects/c3p0/index.html
In our application, we tried configuring c3p0 connection as an afterthought which didn’t work at first. I’ll explain why.
This is what our spring/hibernate configuration looked like in the beginning.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
| <bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource"
p:driverClassName="${jdbc.driverClassName}" p:url="${jdbc.url}"
p:username="${jdbc.username}" p:password="${jdbc.password}" />
<!-- ADD PERSISTENCE SUPPORT HERE (jpa, hibernate, etc) -->
<!-- Hibernate SessionFactory -->
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource">
<ref local="dataSource" />
</property>
<property name="configurationClass">
<value>org.hibernate.cfg.AnnotationConfiguration</value>
</property>
<property name="configLocation">
<value>classpath:hibernate.cfg.xml</value>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
<prop key="hibernate.dialect">${hibernate.dialect}</prop>
<prop key="hibernate.format_sql">${hibernate.format_sql}</prop>
<prop key="hibernate.use_sql_comments">${hibernate.use_sql_comments}</prop>
<prop key="hibernate.auto_close_session">${hibernate.auto_close_session}</prop>
<!-- configuration pool via c3p0 -->
<prop key="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</prop>
<prop key="hibernate.c3p0.acquire_increment">5</prop>
<prop key="hibernate.c3p0.idle_test_period">1800</prop>
<prop key="hibernate.c3p0.max_size">600</prop>
<prop key="hibernate.c3p0.max_statements">50</prop>
<prop key="hibernate.c3p0.min_size">5</prop>
<prop key="hibernate.c3p0.timeout">1800</prop>
</props>
</property>
</bean> |
This apparently did not work. There were no c3p0 specific logs at start up of the application for me to believe that that c3p0 was configured properly. Also, immediately after start up, a quick log in into mysql and checking the number of processes (one per connection) using show processlist; came up with nothing. A little reading up made things clear.
Apparently if you are using Spring and Hibernate, there are two ways you could use c3p0. You could let either spring or hibernate control the connection pooling. You could also configure both but that is redundant and a waste of resources and can cause issues with spring’s transaction management. The above configuration tells me we were attempting to have hibernate control the pool but having it controlled by spring is preferred which is why eventually I changed my configuration.
Have Hibernate Manage the Pool
In the configuration above, c3p0 settings are being made in the session factory which means that hibernate is being responsible for managing the pool instead of Spring. I then looked for the org.hibernate.connection.C3P0ConnectionProvider class which should serve as the connection provider in case hibernate wants to manage the pool. This class wasn’t on my class path either. I had to add the following dependency to include the above mentioned class.
1
2
3
4
5
6
7
8
9
10
| <dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-c3p0</artifactId>
<version>3.6.3.Final</version>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-commons-annotations</artifactId>
<version>3.3.0.ga</version>
</dependency> |
Adding these two did increase my initial DEBUG and INFO log drastically but a quick show processlist; on mysql still showed me that connection pool was not being made ( I didn’t investigate any further) .
Have Spring Manage the Pool
As mentioned earlier, in a spring controlled environment, have spring manage connection management along with connection pooling. This is what was hampering us from using that.
The data source class being used in the configuration above is org.springframework.jdbc.datasource.DriverManagerDataSource which according to the javadocs does not pool connections.
NOTE: This class is not an actual connection pool; it does not actually pool Connections. It just serves as simple replacement for a full-blown connection pool, implementing the same standard interface, but creating new Connections on every call.
Apparently this class implements an interface that calls for connection pooling but does not actually pool them.
Aslo,
If you need a “real” connection pool outside of a J2EE container, consider Apache’s Jakarta Commons DBCP or C3P0. Commons DBCP’s BasicDataSource and C3P0′s ComboPooledDataSource are full connection pool beans, supporting the same basic properties as this class plus specific settings (such as minimal/maximal pool size etc).
This makes it clear that you can’t pool connections using DriverManagerDataSource. Instead you use the DataSource class provided by c3p0. But first you must include the c3p0 jar.
1
2
3
4
5
6
7
| <dependency>
<groupId>c3p0</groupId>
<artifactId>c3p0</artifactId>
<version>0.9.1.2</version>
<type>jar</type>
<scope>compile</scope>
</dependency> |
The data source configuration changes to be something like this
1
2
3
4
5
6
7
8
9
10
11
12
| <bean id="dataSource" class = "com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${jdbc.driverClassName}" />
<property name="jdbcUrl" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<!-- these are C3P0 properties -->
<property name="acquireIncrement" value="${c3p0.acquireIncrement}" />
<property name="minPoolSize" value="${c3p0.minPoolSize}" />
<property name="maxPoolSize" value="${c3p0.maxPoolSize}" />
<property name="maxIdleTime" value="${c3p0.maxIdleTime}" />
</bean> |
The data source class being used is com.mchange.v2.c3p0.ComboPooledDataSource which is a c3p0 pooled data source. The other difference from before is that pool properties are set at the Data source level as opposed to the Hibernate session factory. The session factory bean simply becomes:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
| <bean id="sessionFactory"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource">
<ref local="dataSource" />
</property>
<property name="configurationClass">
<value>org.hibernate.cfg.AnnotationConfiguration</value>
</property>
<property name="configLocation">
<value>classpath:hibernate.cfg.xml</value>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.show_sql">${hibernate.show_sql}</prop>
<prop key="hibernate.dialect">${hibernate.dialect}</prop>
<prop key="hibernate.format_sql">${hibernate.format_sql}</prop>
<prop key="hibernate.use_sql_comments">${hibernate.use_sql_comments}</prop>
<prop key="hibernate.auto_close_session">${hibernate.auto_close_session}</prop>
</props>
</property>
</bean> |
Now it clearly tells me in the log that c3p0 is initialized.
INFO 2012-01-05 00:46:42,487: Initializing c3p0-0.9.1.2 [built 21-May-2007 15:04:56; debug? true; trace: 10]
There are a bunch of other c3p0 related logs that tell me further details above the connection pooling. Checking the process list on mysql shows me 5 connections (my minimum pool size).
My team and I are creating a services platform based on a set of RESTful JSON services where each service contributes to the platform by providing distinct feature(s) and/or data. With logs being generated all over the place, we thought it was a good idea to centralize logging and perhaps also provide a rudimentary log viewer that allowed us to view, filter, sort and search our logs. We also wanted our logging to be asynchronous as we didn’t want our services to be held up while trying to write logs say maybe directly to a database.
The strategy for achieving this was straight forward.
- Setup ActiveMQ
- Create a log4j appender that writes logs to the queue (log4j ships with one such appender but lets write our own.
- Write a message listener that reads logs from a JMS queue setup on an MQ server and persists them
Let’s take a look one by one.
Setup ActiveMQ
Setting up an external ActiveMQ server is simple enough. A great tutorial is available at http://servicebus.blogspot.com/2011/02/installing-apache-active-mq-on-ubuntu.html to set it up on Ubuntu. You can also choose to embed a message broker within your application. Spring makes this easy. We will see how later.
Creating a Lo4j JMS appender
First, we create a log4j JMS appender. log4j ships with one such appender (that writes to a JMS topic instead of a queue)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
| import javax.jms.DeliveryMode;
import javax.jms.Destination;
import javax.jms.MessageProducer;
import javax.jms.ObjectMessage;
import javax.jms.Session;
import org.apache.activemq.ActiveMQConnectionFactory;
import org.apache.log4j.Appender;
import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.Logger;
import org.apache.log4j.PatternLayout;
import org.apache.log4j.spi.LoggingEvent;
/**
* JMSQueue appender is a log4j appender that writes LoggingEvent to a queue.
* @author faheem
*
*/
public class JMSQueueAppender extends AppenderSkeleton implements Appender{
private static Logger logger = Logger.getLogger("JMSQueueAppender");
private String brokerUri;
private String queueName;
@Override
public void close() {
}
@Override
public boolean requiresLayout() {
return false;
}
@Override
protected synchronized void append(LoggingEvent event) {
try {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(
this.brokerUri);
// Create a Connection
javax.jms.Connection connection = connectionFactory.createConnection();
connection.start();
// Create a Session
Session session = connection.createSession(false,Session.AUTO_ACKNOWLEDGE);
// Create the destination (Topic or Queue)
Destination destination = session.createQueue(this.queueName);
// Create a MessageProducer from the Session to the Topic or Queue
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(DeliveryMode.NON_PERSISTENT);
ObjectMessage message = session.createObjectMessage(new LoggingEventWrapper(event));
// Tell the producer to send the message
producer.send(message);
// Clean up
session.close();
connection.close();
} catch (Exception e) {
e.printStackTrace();
}
}
public void setBrokerUri(String brokerUri) {
this.brokerUri = brokerUri;
}
public String getBrokerUri() {
return brokerUri;
}
public void setQueueName(String queueName) {
this.queueName = queueName;
}
public String getQueueName() {
return queueName;
}
} |
Lets see whats happening here.
Line 19: We implement the Log4J appender interface that asks us to implement three methods. requiresLayout, close and append. We will keep things simple for the moment and implement the append method which gets called whenever a method call to the logger is made.
Line 37: log4j calls the append method and passes a LoggingEvent object as a parameter which represents a call to a logger. A LoggingEvent object encapsulates all information about every log item.
Line 41 & 42: Create a new connection factory by providing it with a uri of a JMS, in our case activemq, server
Line 45, 46 and 49: We establish a connection and a session to the JMS server. A Session can be opened in several modes. An Auto_Acknowledge session is one in which the acknowledgment of message happens automatically. Other modes include Client_Acknowledge in which a client has to explicitly acknowledge receipt and/or processing of a message and two other modes. For details, refer to the docs at http://download.oracle.com/javaee/1.4/api/javax/jms/Session.html
Line 52: Create a queue. Send the queue name to connect to as a parameter.
Line 56: We set the delivery mode to Non_Persistent. The other option is Persistent where the message is persisted to a persistent store. Persistent mode slows down but adds reliability to the message transfer.
Line 58: We are doing multiple things. First of all I am wrapping the LoggingEvent object into a LoggingEventWrapper. This is because there are some properties within the LoggingEvent object that are not serializeable and also because I want to capture some additional information such as IP address and host name. Next, using the JMS session object, I prepare an object (the wrapper) for transport.
Line 61: I send the object to the queue.
Below is the code for the wrapper.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
| import java.io.Serializable;
import java.net.InetAddress;
import java.net.UnknownHostException;
import org.apache.log4j.EnhancedPatternLayout;
import org.apache.log4j.spi.LoggingEvent;
/**
* Logging Event Wraps a log4j LoggingEvent object. Wrapping is required by some information is lost
* when the LoggingEvent is serialized. The idea is to extract all information required from the LoggingEvent
* object, place it in the wrapper and then serialize the LoggingEventWrapper. This way all required data remains
* available to us.
* @author faheem
*
*/
public class LoggingEventWrapper implements Serializable{
private static final String ENHANCED_PATTERN_LAYOUT = "%throwable";
private static final long serialVersionUID = 3281981073249085474L;
private LoggingEvent loggingEvent;
private Long timeStamp;
private String level;
private String logger;
private String message;
private String detail;
private String ipAddress;
private String hostName;
public LoggingEventWrapper(LoggingEvent loggingEvent){
this.loggingEvent = loggingEvent;
//Format event and set detail field
EnhancedPatternLayout layout = new EnhancedPatternLayout();
layout.setConversionPattern(ENHANCED_PATTERN_LAYOUT);
this.detail = layout.format(this.loggingEvent);
}
public Long getTimeStamp() {
return this.loggingEvent.timeStamp;
}
public String getLevel() {
return this.loggingEvent.getLevel().toString();
}
public String getLogger() {
return this.loggingEvent.getLoggerName();
}
public String getMessage() {
return this.loggingEvent.getRenderedMessage();
}
public String getDetail() {
return this.detail;
}
public LoggingEvent getLoggingEvent() {
return loggingEvent;
}
public String getIpAddress() {
try {
return InetAddress.getLocalHost().getHostAddress();
} catch (UnknownHostException e) {
return "Could not determine IP";
}
}
public String getHostName() {
try {
return InetAddress.getLocalHost().getHostName();
} catch (UnknownHostException e) {
return "Could not determine Host Name";
}
}
} |
The Message Listener
The message listener “listens” to the queue (or topic). Whenever a new message is added to the queue, the onMessage method is called.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
| import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageListener;
import javax.jms.ObjectMessage;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
@Component
public class LogQueueListener implements MessageListener
{
public static Logger logger = Logger.getLogger(LogQueueListener.class);
@Autowired
private ILoggingService loggingService;
public void onMessage( final Message message )
{
if ( message instanceof ObjectMessage )
{
try{
final LoggingEventWrapper loggingEventWrapper = (LoggingEventWrapper)((ObjectMessage) message).getObject();
loggingService.saveLog(loggingEventWrapper);
}
catch (final JMSException e)
{
logger.error(e.getMessage(), e);
} catch (Exception e) {
logger.error(e.getMessage(),e);
}
}
}
} |
Line 23: Checking if the object being picked off the queue is an instance of ObjectMessage
Line 26: Extracting LoggingEventWrapper from the Message
Line 27: Call a service method to persist the log
Wiring up in spring
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
| <?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:jms="http://www.springframework.org/schema/jms"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:aop="http://www.springframework.org/schema/aop"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-3.0.xsd
http://www.springframework.org/schema/jms
http://www.springframework.org/schema/jms/spring-jms-3.0.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core-5.5.0.xsd"
>
<!-- lets create an embedded ActiveMQ Broker -->
<!-- uncomment the tag below only if you need to create an embedded broker -->
<!-- amq:broker useJmx="false" persistent="false">
<amq:transportConnectors>
<amq:transportConnector uri="tcp://localhost:61616" />
</amq:transportConnectors>
</amq:broker-->
<!-- ActiveMQ destinations to use -->
<amq:queue id="destination" physicalName="logQueue" />
<!-- JMS ConnectionFactory to use, configuring the embedded broker using XML -->
<amq:connectionFactory id="jmsFactory" brokerURL="tcp://localhost:61616" />
<bean id="connectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<constructor-arg ref="jmsFactory" />
<property name="exceptionListener" ref="JMSExceptionListener" />
<property name="sessionCacheSize" value="100" />
</bean>
<!-- Spring JMS Template -->
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<constructor-arg ref="connectionFactory" />
</bean>
<!-- listener container definition using the jms namespace, concurrency
is the max number of concurrent listeners that can be started -->
<jms:listener-container concurrency="10">
<jms:listener id="QueueListener" destination="logQueue" ref="logQueueListener" />
</jms:listener-container>
</beans> |
Lines 23-27: Use the broker tag to setup an embedded message broker. Since I am using an external one, I don’t need it.
Line 30: Mention the name of the queue you want to connect to.
Line 33: URI of the Broker Server.
Line 35-45: Connection Factory setup
Line 49-51: Message Listener Setup where we specify the number of concurrent threads that can consume messages off the queue.
Of course, the above example will not work out of the box. You still have to include all JMS dependencies and implement the service that persists logs. But I hope it gives you a decent idea.
I am writing this post to resolve any confusions about ways to set up Spring Application Contexts.
In Spring Web Applications, there are two types of container, each of which is configured and initialized differently. One is the “Application Context” and the other is the “Web Application Context”. Lets first talk about the “Application Context”.
Application Context is the container initialized by a ContextLoaderListener or ContextLoaderServlet defined in the web.xml and the configuration would look something like this:
1
2
3
4
5
6
7
8
9
|
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>classpath:*-context.xml</param-value>
</context-param> |
In the above configuration, I am asking spring to load all files from the classpath that match *-context.xml and create an Application Context from it. This context might, for instance, contain components such as middle-tier transactional services, data access objects, or other objects that you might want to use (and re-use) across the application. There will be one application context per application.
The other context is the “WebApplicationContext” which is the child context of the application context. Each DispatcherServlet defined in a Spring web application will have an associated WebApplicationContext. The initialization of the WebApplicationContext happens like this:
1
2
3
4
5
6
7
8
9
| <servlet>
<servlet-name>platform-services</servlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
<init-param>
<param-name>contextConfigLocation</param-name>
<param-value>classpath:platform-services-servlet.xml</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet> |
You provide the name of the spring configuration file as a servlet initialization parameter. What is important to remember here is that the name of the XML must be of the form <servlet name>-servlet. xml. In our example, the name of the servlet is platform-services therefore the name of our XML must be platform-service-servlet.xml.
Whatever beans are available in the ApplicationContext can be referred to from each WebApplicationContext. It is a best practice to keep a clear separation between middle-tier services such as business logic components and data access classes (that are typically defined in the ApplicationContext) and web- related components such as controllers and view resolvers (that are defined in the WebApplicationContext per Dispatcher Servlet).
In this first part of a multi-part post, we will learn how to create a very simple socket server and socket client pair in Java. Gradually, in subsequent posts, we will add additional functionality to our server and client. Before we head in to the code, lets go over some basics.
Client-Server Model
The client–server model of computing is a distributed application that partitions tasks or workloads between the providers of a resource or service, called servers, and service requesters, called clients.Often clients and servers communicate over a computer network on separate hardware, but both client and server may reside in the same system. A server machine is a host that is running one or more server programs which share their resources with clients. A client does not share any of its resources, but requests a server’s content or service function. Clients therefore initiate communication sessions with servers which await incoming requests.[1]
Socket Programming
A socket is one of the most fundamental technologies of computer networking. Sockets allow applications to communicate using standard mechanisms built into network hardware and operating systems. Although network software may seem to be a relatively new “Web” phenomenon, socket technology actually has been employed for roughly two decades.
A socket represents a single connection between exactly two pieces of software. More than two pieces of software can communicate in client/server or distributed systems (for example, many Web browsers can simultaneously communicate with a single Web server) but multiple sockets are required to do this. Sockets are bidirectional, meaning that either side of the connection is capable of both sending and receiving data. Libraries implementing sockets for Internet Protocol use TCP for streams, UDP for datagrams, and IP itself for raw sockets. [2]
Writing a Server
In this post, the server is simply going to respond to a newly connected client with the following string. “Hello. You are connected to a Simple Socket Server. What is your name?”. As mentioned earlier, we will gradually increase the capability of the server in subsequent parts of the series. Let’s look at the code.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
| package pk.edu.ucp.web.socketServer;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
/**
* A simple socket server
* @author faheem
*
*/
public class SocketServer {
private ServerSocket serverSocket;
private int port;
public SocketServer(int port) {
this.port = port;
}
public void start() throws IOException {
System.out.println("Starting the socket server at port:" + port);
serverSocket = new ServerSocket(port);
//Listen for clients. Block till one connects
System.out.println("Waiting for clients...");
Socket client = serverSocket.accept();
//A client has connected to this server. Send welcome message
sendWelcomeMessage(client);
}
private void sendWelcomeMessage(Socket client) throws IOException {
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(client.getOutputStream()));
writer.write("Hello. You are connected to a Simple Socket Server. What is your name?");
writer.flush();
}
/**
* Creates a SocketServer object and starts the server.
*
* @param args
*/
public static void main(String[] args) {
// Setting a default port number.
int portNumber = 9990;
try {
// initializing the Socket Server
SocketServer socketServer = new SocketServer(portNumber);
socketServer.start();
} catch (IOException e) {
e.printStackTrace();
}
}
} |
Server Code Description
- In lines 56 and 57 we create a new object of our SocketServer class and call the start() method.
- In line 28, we an object of ServerSocket which is a Java class used to create socket servers.
- In line 33, we ask our server to wait for client connections. The serverSocket.accept() blocks the thread until a connection is made. Once a connection is made, a Socket object representing the connected client will be created and assigned to the variable “client”.
- Line 36, will only be executed after a client has connected to our server. In this we call a private method sendWelcomeMessage(client);
- Lines 39 to 43 is the implementation of the sendWelcomeMessage function. It gets the outputstream of the connected client, creates a BufferedWriter which is then used to write the message for the client.
Writing a Client
This client will connect to the server that we have created, and as soon as the connection is created, it will ask the server for any information it needs to send to the client. The client will then write that information to the console.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
| package pk.edu.ucp.web.socketClient;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.Socket;
import java.net.UnknownHostException;
/**
* A Simple Socket client that connects to our socket server
* @author faheem
*
*/
public class SocketClient {
private String hostname;
private int port;
Socket socketClient;
public SocketClient(String hostname, int port){
this.hostname = hostname;
this.port = port;
}
public void connect() throws UnknownHostException, IOException{
System.out.println("Attempting to connect to "+hostname+":"+port);
socketClient = new Socket(hostname,port);
System.out.println("Connection Established");
}
public void readResponse() throws IOException{
String userInput;
BufferedReader stdIn = new BufferedReader(new InputStreamReader(socketClient.getInputStream()));
System.out.println("Response from server:");
while ((userInput = stdIn.readLine()) != null) {
System.out.println(userInput);
}
}
public static void main(String arg[]){
//Creating a SocketClient object
SocketClient client = new SocketClient ("localhost",9990);
try {
//trying to establish connection to the server
client.connect();
//if successful, read response from server
client.readResponse();
} catch (UnknownHostException e) {
System.err.println("Host unknown. Cannot establish connection");
} catch (IOException e) {
System.err.println("Cannot establish connection. Server may not be up."+e.getMessage());
}
}
} |
Client Code Description
- In Line 45 we create an object of our class and pass the hostname and port parameters
- In Line 48 we call the connect method of our client class which will establish the connection to the server
- In Line 50 we call a method that we have written which reads data sent from the client.
- Line 27 to 31 is the connect method. At Line 29 we create a new Socket( a java class that represents a socket connection) that is used to establish connection to our server.
- Line 33 to 41 is our readResponse method, in which we get an inputstream of the socket (representing the connection between the server and the client) and read a line of text from it.
Running the program
You will first run the Server program, like you run any other Java Program. You should get an output like
Starting the socket server at port:9990
Waiting for clients…
Then you run the client after which you get the following output
Attempting to connect to localhost:9990
Connection Established
Response from server:
Hello. You are connected to a Simple Socket Server. What is your name?
Both programs will terminate.
Improvements
You will note that the server we have written is only good for one connection. It terminates after a client has connected with it and information has been exchanged. Also, the information exchange in this example is single directional (from server to the client). Two immediate improvements can be made.
- Having the ability to serve multiple clients
- Bi-directional exchange of information
I will address these two in subsequent posts.
References
1. http://en.wikipedia.org/wiki/Client%E2%80%93server_model
2. http://compnetworking.about.com/od/itinformationtechnology/l/aa083100a.htm
3. http://download.oracle.com/javase/tutorial/networking/sockets/index.html
Adding basic http authentication to a set of Spring MVC based services is very simple. This tutorial teaches how to quickly setup authentication by keeping a set of usernames, passwords and roles in a spring configuration file and associating URL patterns with these for the framework to determine if a user is authorized or not. For more robust authentication and authorization where you might be using a Directory server and LDAP, additional configurations are required which are not being discussed in this post.
In this tutorial, I will work with the assumption that you already have a running application built with Spring MVC. If you do, there really are only two steps involved. Let’s start:
1. Adding Dependencies
The first thing you need to do is get the relevant spring security modules. Add these dependencies to your pom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
| <dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-web</artifactId>
<version>${org.springframework.version}</version>
<type>jar</type>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.springframework.security</groupId>
<artifactId>spring-security-config</artifactId>
<version>${org.springframework.version}</version>
<type>jar</type>
<scope>compile</scope>
</dependency> |
2. Modifying the web.xml
The first thing we do is to add a DelegatingFilterProxy filter. This is a standard servlet filter that delegates responsibility to a Spring bean. In other words, you can provide spring support for servlet filters via the DelegatingFilterProxy. In our example, we will use the DelegatingFilterProxy to delegate authentication (and eventually authorization) responsibilities to a spring bean. Add the following XML to your web.xml.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
| <filter>
<filter-name>springSecurityFilterChain</filter-name>
<filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
</filter>
<filter-mapping>
<filter-name>springSecurityFilterChain</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<listener>
<listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>classpath:security-context.xml</param-value>
</context-param> |
3. Creating the security context xml and placing it in the classpath
In line 16 of the web.xml snippet above, I refer to a file security-context.xml. This gets loaded into the spring context at application startup and consists of our security configurations. This is what this file looks like:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
| <?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:security="http://www.springframework.org/schema/security"
xmlns:context="http://www.springframework.org/schema/context"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/security
http://www.springframework.org/schema/security/spring-security-3.0.xsd">
<security:http auto-config='true'>
<security:intercept-url pattern="/logs/**" access="ROLE_USER, ROLE_ADMIN"/>
<security:intercept-url pattern="/logviewer.html" access="ROLE_USER, ROLE_ADMIN"/>
<security:intercept-url pattern="/qr/**" access="ROLE_USER, ROLE_ADMIN"/>
<!-- security:intercept-url pattern="/sms/**" access="ROLE_USER, ROLE_ADMIN"/-->
<security:intercept-url pattern="/**" access="IS_AUTHENTICATED_ANONYMOUSLY"/>
<security:http-basic />
</security:http>
<security:authentication-manager>
<security:authentication-provider>
<security:user-service>
<security:user name="username1" password="password1" authorities="ROLE_USER" />
<security:user name="username2" password="password2" authorities="ROLE_USER, ROLE_ADMIN" />
</security:user-service>
</security:authentication-provider>
</security:authentication-manager>
</beans> |
Remember to add namespace and xsd declarations for security (lines 4 and 9). From lines 13 to 17, we provide mappings of URLs against roles. The ROLE_ settings are the roles within the application where these could be roles defined in a database or groups in active directory etc. IS_AUTHENTICATED_ANONYMOUSLY is used to specify that you can get in anonymously. Therefore the above settings allow all requests to be served anonymously without authentication except the 3 mentioned that require authentication. The intercepts are evaluated top-down which means that if <security:intercept-url pattern=”/**” access=”IS_AUTHENTICATED_ANONYMOUSLY”/> was first in the list, all requests would go through.
4. Calling the service
Let’s see what happens when we call a URL that is configured to be authenticated. We get a pop up that asks for username and password.

In case you provide an invalid user name and/or password, you get a HTTP 415 error

Lets look at the HTTP request header when we have a successful login

You will notice in the image above that there is a header attribute Authorization with value Basic dXNlcm5hbWUxOnBhc3N3b3JkMQ== If we decode this using any Base64 decoder, we get username1:password1. You can see that HTTP basic authentication simply takes your username and password, concatenates them as username:password and then encodes them in Base64 and sends it over the wire.
The bottom line is that HTTP basic auth is very easy to implement and better than no security at all but is vulnerable and should be replaced with more robust mechanism like Digest Authentication which I may discuss in a future post.
Following is a tentative course outline for a course I would be teaching titled “Web Engineering”
This technology intensive course teaches students various practices, tools and techniques to build modern web applications. Many of the concepts taught in this course have implementations in various development platforms however this course takes a Java centric approach hence assumes that students have an understanding of the Java language. The course also teaches students various factors to consider while making decisions related to technology selection.
Week 1
- Introduction to the course
- Understanding how the internet works
- Overview of the OSI layer
- Network protocols especially HTTP, SMTP and the FTP protocols
Week 2
- Socket Programming in Java
- Assignment 1
Week 3
- Writing RPC applications in Java using RMI and XML-RPC
Week 4
- Understanding Message based systems.
- Integrating systems using JMS and ActiveMQ
- Assignment 2
Week 5
- Web Application Architecture
- Build Tools (maven 2)
- Quiz 1
Week 6
- Java Server-side scripting using Servlets
- Pipes and Filters Architectural Pattern
- Servlet Filters
- Assignment 3
Week 7
Week 8
- Client side scripting using Javascript
- AJAX
- Introduction to JQuery
- Midterm
Week 9
- Webservices
- RESTful architectural style
- Assignment 4
Week 10
- MVC architectural pattern
- Struts 2
Week 11
- Inversion of Control Pattern
- Spring framework
- Spring MVC
- Quiz 2
Week 12
- XML and JSON parsing
- Memcache for performance
- Sending Email
Week 13
Week 14
- Final Project Evaluation
- Final Exam
Grade Breakup
Assignments – 20%
Quizzes – 10%
Project – 10%
Midterm – 25%
Final – 35%
Ground rules
- Late assignments will not be accepted. No exceptions.
- Copied assignments will result in zero marks.
These are a bunch of books that I have bought/ordered recently. Actually, come to think of it, these are quite a lot. I got a bit greedy as someone was coming from the US and I wanted to avoid Amazon shipping costs so I stacked up . I just hope that over the next few months I am able to read all or at least most of them. These book represent areas that I wish to learn more about. I believe any self-respecting solution architect/developer must know about these technologies.
1. Hadoop: The definitive guide
 Hadoop: The definitive guide
There is a lot of talk about Big data and NoSQL that relates to management of huge amounts of data. I bought this book because I wanted to understand how applications like facebook (30 PB of data in its Hadoop cluster as of June 2011), twitter, ebay among many others manage the large amounts of data they have. You never know when you get into a situation where your usual RDBMs do not suffice.
2. Programming Android
 Programming Android
No denying that android has over taken (or is at least at par with) iOS as the next desired mobile phone/tablet platform hence my desire to build apps and games is strong. I remember trying to learn J2ME back in the day. I am glad I didn’t.
3. Physics for Game Developers
 Physics for Game Developers
I don’t think have the right temperament for game development (I don’t think I am creative enough for that) however my curiosity got the better of me. I have been wondering for a while what it would take to build a game like angry birds for example, simulating things like trajectories, collisions etc. Seems interesting.
4. Algorithms in a Nutshell
 Algorithms in a nutshell
I am not sure why I bought this book. To be honest, Cormen is a bit too mathematical and theoretical for me. Wanted to keep this in my bookshelf as a handy reference.
5. Software Architecture: Perspectives on an Emerging Discipline
 Software Archicture
This book is a little outdated but I am sure there is something to learn from Software Architecture’s grand daddy.
|
|