Google App Engine JAX-RS REST service

In this article you will learn how to create a REST service using JAX-RS reference implementation (Jersey) and deploy it on Google AppEngine.

Prerequisites 

For this tutorial you will need:
  • a Google AppEngine account
  • Eclipse Galileo (3.5.x)
  • Google App Engine SDK for Java
    • Install the Google Plugin for Eclipse as documented here (Check that you are using the release 1.3.1 of the GAE Java SDK, if not download it and configure the plugin to use it)
    • it is also useful to have the AppEngine documentation locally, you can download it from here.
  • JAX-RS Reference Implementation, be sure you take the Jersey 1.1.5 release. You can download it from here
    • Unzip the file in a directory that we will call $JERSEY_HOME
  • JAXB 2.2 Implementation to simplify the marshalling/unmarshalling of the XML, and also facilitate the JSON support. Download it from here
    • Install it using thejava -jar JAXB2_20091104.jar command. The installation directory of JAXB will be called $JAXB_HOME



Creating new application

To create a new App Engine project in Eclipse:

  1. Click on the "New Web Application Project" button in the toolbar . It is also possible to do it using the menu File > Web Application Project
  2. The "Create a Web Application Project" wizard opens:
  • Project Name: EmployeeService
  • Package : com.grallandco.employee.service
  • Uncheck "Use Google Web Toolkit"
  • Check that the SDK version your are using is "App Engine 1.3.0"; if not configure the project to use it.
  • The screen should look like the following screen :
  • Click Finish
  • The project should look like the following screen :

Running the application

The App Egine SDK, installed with the Eclipse plugin contains a Web server (based on Jetty), that could be used for testing and debugging. To test that your application has been created correctly select the menu Run > Run As > Web Application. I personnaly most of the time run my server using the debug command Run > DebugAs > Web Application. In debug mode you can change source code and test is without restarting the server.

The web server is starting automatically, you should see the following message in the Eclipse console

The server is running at http://localhost:8080/

You can access the application, and the sample servlet that has been created using the URL: http://localhost:8080/employeeservice

To stop the server, click on the terminate button
in the Eclipse console.

Configuring the REST support in the application

To be able to create and run REST services in your application you need to:

  • Add the JAX-RS, JAXB Jars in your project and application
  • Configure the web application (web.xml) to handle REST requests

Add JAX-RS, JAXB to your project

  1. Right click on the project and select menu entry Build Path > Configure Build Path…
  2. Click on the Add External JARs button
  3. Select all the JARs located in $JERSEY_HOME/lib and $JAXB_HOME/lib folders. You can for better visibility and reuse create a user library with all these JARs
  4. You also need to copy the JARs in the web-inf/lib directory of your application, this step is mandatory to be sure that the JARs are included in the application when deployed to App Engine.
    Note: I do not like this step. I would prefer to do that by configuration of the build path, to automatically add the JARs to the WEB-INF/lib directory when executing/deploying the application. Unfortunately I did not find the way to do it, so if you know it, feel free to post a comment and I will update the article.

Configure the web application

In this step you will register a new URI to handle REST requests. To do that you need to register a new servlet that is using the Jersey API and configure it to a specific URI (eg: /ressources and/or /rest) and configure what are the Java packages that contain the REST implementation classes. So you need to modify the web.xml of your application with the following entries:

  <servlet>
    <servlet-name>Jersey Web Application</servlet-name>
     <servlet-class>com.sun.jersey.spi.container.servlet.ServletContainer</servlet-class>
     <init-param>
     <param-name>com.sun.jersey.config.property.packages</param-name>
     <param-value>com.grallandco.employee.service.rest.impl</param-value>
     </init-param>
     <load-on-startup>1</load-on-startup>
  </servlet>
  <servlet-mapping>
    <servlet-name>Jersey Web Application</servlet-name>
    <url-pattern>/resources/*</url-pattern>
  </servlet-mapping>
   <servlet-mapping>
    <servlet-name>Jersey Web Application</servlet-name>
    <url-pattern>/rest/*</url-pattern>
  </servlet-mapping>
This servlet that will answer to the /resources/ and /rest/ URL. The configuration parameter com.sun.jersey.config.property.packages is used by Jersey to list the packages where REST services implementation are located.Note that you can put as many package as you need to, you just need to separate the package names by a ; 


Creating a simple REST Service to test the environment

The project is now ready to contain REST service. It is time to create one.Create for example the class com.grallandco.employee.service.rest.impl.HelloWorldResource, be sure to use the package name that you have configured in the web.xml for the Jersey servlet, based on the configuration we have made in previous step the package is com.grallandco.employee.service.rest.impl

Here a sample class with the JAX-RS annotations:

package com.grallandco.employee.service.rest.impl;
import javax.ws.rs.Path;
import javax.ws.rs.GET;
import javax.ws.rs.Produces;
@Path("/hr/")
public class EmployeeResource {
 
 @GET
 @Produces("text/plain")
 @Path("/employee") 
 public String getEmployee() {
        return "Hello World!";
    }
}

You should be able to test it, stop the server and run it again, enter the following URL in your browser:

http://localhost:8080/resources/hr/employee

or

http://localhost:8080/rest/hr/employee

Deploying the application to Google App Engine

Before deploying the application you need to register a new application in Google App Engine using the Administartion Console, see the documentation
here. In my example I have used " tugdual" as Application ID.

You can easily now deploy the application to Google App Engine by clicking on the "Deploy App Engine Project" button available in the Eclipse toolbar.

To be able to deploy your application to Google App Engine, you need to check that your application can be registered, the application ID is stored in the WEB-INF/lib/appengine-web.xml.

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
 <application>[your-application-id]</application>   
 <version>1</version>    
 <!-- Configure java.util.logging -->
  <system-properties>
   <property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
 </system-properties>    
</appengine-web-app>

The App Engine deploy button prompts you for multiple informations: username (your Google account) and password.

When the deployment is complete you can access your application using the following URL:

http://%5Byour-application-id%5D.appspot.com/resources/hr/employee

or

http://%5Byour-application-id%5D.appspot.com/rest/hr/employee

Ading XML and JSON support to the service

Let’s now add new method to manipulate an "Employee" object using the service, and the data format should be based on JSON and XML. This is where JAXB is useful, since it allows easily to transform marshall/unmarshall Java objects in XML -obviously- and JSON (cool isn’t!)

Creating an Employee Class

Start with the creation of a new class to manipulate Employee data, this is a very simple Java class that could look like the following code:

package com.grallandco.employee.service.model;
import java.util.Date;

public class Employee {
    private String firstName;
    private String lastName;
    private Date hireDate;
    private String email;   
    public Employee(String firstName, String lastName, Date hireDate, String email) {
        this.firstName = firstName;
        this.lastName = lastName;
        this.hireDate = hireDate;
        this.email = email;
    }
    public Employee() {}
    public String getFirstName() {
 return firstName;
 }
    public void setFirstName(String firstName) {
 this.firstName = firstName;
 }
    public String getLastName() {
 return lastName;
 }
    public void setLastName(String lastName) {
     this.lastName = lastName;
 }
    public Date getHireDate() {
 return hireDate;
 }
    public void setHireDate(Date hireDate) {
 this.hireDate = hireDate;
 }
    public String getEmail() {
        return email;
 }
    public void setEmail(String email) {
        this.email = email;
   }
    public String toString() {
        StringBuffer sb = new StringBuffer();
        sb.append("First: ").append(getFirstName());
        sb.append(" - Last: ").append(getLastName());
        sb.append(" - Date: ").append(getHireDate());
        sb.append(" - Email: ").append(getEmail());
        return sb.toString();
    }
}

When implementing your "real" application with some persistence layer this POJO is the one as JDO/JPA entity.

Create a Converter class for your entity

I usually encapsulate all the transformation in some converter class, like that I do not directly couple my business class to the serialisation mechanism. (So I do that for classes and lists of classes). So instead of adding the JAXB annotations to the Employee class itself, let’s create an EmployeeConverter class that will be responsible of the transformation and used by your REST service.

package com.grallandco.employee.service.converter;

import java.util.Date;
import javax.xml.bind.annotation.XmlElement;
import javax.xml.bind.annotation.XmlRootElement;
import com.grallandco.employee.service.model.Employee;

@XmlRootElement(name = "employee")
public class EmployeeConverter {
 private Employee entity = null;
 public EmployeeConverter() {
 entity = new Employee();
 }

 public EmployeeConverter(Employee entity) {
 this.entity = entity;
 }

 @XmlElement
 public String getFirstName() {
 return entity.getFirstName();
 }

 @XmlElement
 public String getLastName() {
 return entity.getLastName();
 }

 @XmlElement
 public Date getHireDate() {
 return entity.getHireDate();
 }

 @XmlElement
 public String getEmail() {
 return entity.getEmail();
 }

 public Employee getEmployee() {
 return entity;
 }

 public void setFirstName(String firstName) {
 entity.setFirstName(firstName);
 }

 public void setHireDate(Date hireDate) {
 entity.setHireDate(hireDate);
 }

 public void setLastName(String email) {
 entity.setEmail(email);
 }

 public void setEmail(String lastName) {
 entity.setLastName(lastName);
 }
}

You can now update your service to use this utility/converter class to return XML or JSON ojbect based on the content type of the request.

Add support to JSON and XML to your REST service

You need to change the EmployeeRessource class, to change the signature and add new annotations of the getEmployee() method.

The annotation you are adding:

  • @Produces({"application/xml", "application/json"}) : indicates which type of content will be produced by the service. Based on the type of the request.
  • @Path("/employee/{employeeEmail}/") : change the Path to indicate a Path parameter, here for example the URL can accept an email in the URI – not the best example, but you get the point…
  • public EmployeeConverter getEmployee( @PathParam ("employeeEmail") String email) : change the type returned by the method and take a parameter as String that match the Path param defined in the @Path annotation

Here the complete class code:

package com.grallandco.employee.service.rest.impl;

import javax.ws.rs.Path;
import javax.ws.rs.GET;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import com.grallandco.employee.service.converter.EmployeeConverter;
import com.grallandco.employee.service.model.Employee;

@Path("/hr/")
public class EmployeeRessource {

 
 @GET
 @Produces({"application/xml", "application/json"})
 @Path("/employee/{employeeEmail}/") 
 public EmployeeConverter getEmployee( @PathParam ("employeeEmail") String email) {
 //dummy code
 Employee emp = new Employee();
 emp.setEmail(email);
 emp.setFirstName("John");
 emp.setLastName("Doe");
 EmployeeConverter converter = new EmployeeConverter(emp);
 return converter;
 } 
}

Test the service

You can now run the server locally and test the service

http://localhost:8080/resources/hr/employee/tug@grallandco.com

This will return an XML document.

If you want to test the JSON call you have multiple choice:

  • Using following command
tgrall$ curl -H "Accept: application/json" http://localhost:8080/resources/hr/employee/tug@grallandco.com
{"email":"tug@grallandco.com","firstName":"John","lastName":"Doe"}
  • Using an HTTP client that allows your to configure/set the HTTP request completely, I am using the Poster Firefox Plugin
  • Using some Javascript code in an application

You can repeat the test on your deployed application on Google App Engine.

Conclusion

In this article you have learned how to create and deploy a new REST Service on Google App Engine. This service has been created with the JAX-RS Reference Implementation the Jersey project. In the next article you will learn how to add persistence and create a CRUD Rest service on Google App Engine.


Reference:
Create and Deploy a JAX-RS REST service on Google App Engine from our
JCG partner Tugdual Grall at the
Tug’s Blog blog.

Source : http://www.javacodegeeks.com/2012/05/google-app-engine-jax-rs-rest-service.html

Advertisements

Java High CPU troubleshooting guide – part 1

This article is part 1 of a series that will provide you with a comprehensive guide on how you can troubleshoot and identify root cause of Java high CPU problems.

This guide is also applicable for standalone Java programs but designed to help individuals involved in day to day Java EE enterprise production support. It will also include the list of the most common high CPU problems along with high level resolution strategies.


Production problems resolution mindset review

Before we go any further, it is important to review your production problem resolution mindset. One of the most common “reflexes” that I have seen in my experience with Java EE production support teams is that Java VM / middleware restart is often the first recovery action that is performed. While a premature restart can quickly eliminate the business impact, it can also prevent you to get all the technical facts; reducing your capability to identify the root cause and exposing the platform to future re-occurrences of the problem.

Before pulling the trigger and shutdown your Java VM process, ask yourself the following question: do I have all the data available to perform a root cause analysis post restart? If the answer is
no then my recommendation to you is to review and improve your current platform monitoring and / or troubleshooting approaches. Proper performance data gathering before and during a high CPU problem is critical.

Java high CPU – what is it exactly?

Now back to our original topic, a high CPU problem is defined by an observation of one or many Java VM processes consuming excessive CPU utilization from your physical host(s). Excessive CPU can also be described by an abnormal high CPU utilization vs. a known & established
baseline. Ex: if the average CPU utilization of your Java VM under peak load condition is 40% then excessive CPU threshold can be set around 80%.

A typical Java VM process contains several Java Threads, some waiting to do work and others currently executing tasks. The # of Threads can be very low in the event of a single Java program and very high for Java EE enterprise platforms processing heavy concurrent transactions.

In order to understand and identify the source of high CPU of one or many of your Java processes, you will need to understand and perform a full breakdown of all Threads of your Java VM so you can pinpoint the biggest contributors. This analysis exercise can be visualized as per below diagram.

Understand your average CPU utilization

As I mentioned in the earlier section, it is very important that you understand your current average CPU utilization, what I refer as a baseline. This is crucial data that needs to be monitored on a regular basis as part of a comprehensive and ongoing platform capacity planning strategy.

Proper understanding and tracking of your average and “healthy” CPU utilization observed from your Java VM processes will allow you to quickly detect abnormal CPU surge scenarios and correlate with possible root causes (problem introduced by a project, unexpected load increase etc.). Finally, this will provide you with proper threshold values necessary to configure pro-active CPU related alerts using monitoring tools of your choice.

Understand your production environment and available tools

As the middleware and / or application support prime, you really need to understand your production environment, including the out-of-the-box tools available for you to perform low level troubleshooting tasks. This may be trivial for some individuals but if you just recently started to work on a new Java or Java EE platform for a new client, my recommendation is that you should spend enough time understand your client’s environment specifications & business situation as per below:

  • Physical & virtual host configuration and capacity (total # of assigned CPU cores, RAM etc.)
  • OS vendor, version and patch level
  • Middleware vendor, versions and patch level
  • Java vendor & versions (including 32-bit vs. 64-bit); including patch level
  • Third party API’s used within the Java or Java EE applications
  • Existing monitoring tools that you can leverage for historical data and trend analysis
  • History of the environment, known issues, resources utilization etc.
  • Business traffic breakdown per application along with average & peak traffic level of the platform; including peak business periods

Collecting all proper facts as per above will definitely help your root cause analysis process going forward; including for high CPU related problems.

Your homework before jumping to the part 2

Before we jump into the part 2 of this CPU troubleshooting guide, I highly recommend that you study and understand each the following articles below. Focus on the ones applicable for your environment. Each of these articles will provide you with a technical step by step on how to breakdown CPU per Thread for a Java VM; a key troubleshooting skill to acquire in order to investigate Java CPU related problems. This technique is common on some aspects and specific depending of the OS.

# CPU per Thread analysis on Solaris


http://javaeesupportpatterns.blogspot.com/2011/12/prstat-solaris-pinpoint-high-cpu-java.html

# CPU per Thread analysis on Linux


http://javaeesupportpatterns.blogspot.com/2012/02/prstat-linux-how-to-pinpoint-high-cpu.html

# CPU per Thread analysis on AIX


http://javaeesupportpatterns.blogspot.com/2011/12/prstat-aix-how-to-pinpoint-high-cpu.html

# CPU per Thread analysis on Windows


http://javaeesupportpatterns.blogspot.com/2012/04/java-thread-cpu-analysis-on-windows.html

I hope this article has provided with a good starting point on Java CPU problems. The part 2 will provide you with the troubleshooting guide which will include flow diagrams allowing you to choose the right investigation path depending of your problem case.


Reference:
Java High CPU troubleshooting guide – part 1 from our
JCG partner Pierre-Hugues Charbonneau at the
Java EE Support Patterns & Java Tutorial blog.

Source : http://www.javacodegeeks.com/2012/05/java-high-cpu-troubleshooting-guide.html

How to use RestTemplate with Basic Authentication in Spring

This article shows how to use Springs
RestTemplate to
consume a RESTful Service secured with Basic Authentication; the article is part of an in depth
series on REST with Spring.

The REST with Spring series:


Setting up the RestTemplate in Spring

Bootstraping the
RestTemplate into the Spring context can be done by simply declaring a bean for it; however, setting up the
RestTemplate with
Basic Authentication will require manual intervention, so instead of declaring the bean directly, a Spring
FactoryBean will be used for more flexibility. This factory will create and configure the template on initialization:

@Component
public class RestTemplateFactory implements
    FactoryBean< RestTemplate >, InitializingBean{
   private RestTemplate restTemplate;

   public RestTemplate getObject(){
      return restTemplate;
   }
   public Class< RestTemplate > getObjectType(){
      return RestTemplate.class;
   }
   public boolean isSingleton(){
      return true;
   }
   public void afterPropertiesSet(){
      restTemplate = new RestTemplate(
       new HttpComponentsClientHttpRequestFactoryBasicAuth
        ( new HttpHost(host, port, protocol) );
   }
}

The
host and
port values should be dependent on the environment – allowing the client the flexibility to define one set of values for integration testing and another for production use. The values can be managed by the
first class Spring support for properties files.

Basic Authentication for the RestTemplate

Once Basic Authentication is set up for the template, each request will be sent
preemptively containing the full credentials necessary to perform the authentication process. The credentials will be encoded and will use the
Authorization HTTP Header, in accordance with the specs of the Basic Authentication scheme. An example would look like this:

Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==


Manual management of the Authorization HTTP header

The process of creating the
Authorization header is relatively straightforward for Basic Authentication, so it can pretty much be done manually with a few lines of code:

HttpHeaders createHeaders( String username, String password ){
   return new HttpHeaders(){
      {
         String auth = username + ":" + password;
         byte[] encodedAuth = Base64.encodeBase64( auth.getBytes() );
         String authHeader = "Basic " + new String( encodedAuth );
         set( "Authorization", authHeader );
      }
   };
}

Then, sending a request becomes just as simple:

restTemplate.exchange
 (uri, HttpMethod.POST, new HttpEntity<T>(createHeaders(username, password)), clazz);


Automatic management of the Authorization HTTP header

Both Spring 3.0 and 3.1 have very good support for the Apache HTTP libraries:

  • Spring 3.0, the CommonsClientHttpRequestFactory integrated with the now end of lifed HttpClient 3.x
  • Spring 3.1 introduced support for the current HttpClient 4.x via HttpComponentsClientHttpRequestFactory (support added in the JIRA SPR-6180)

Let’s start setting things up with HttpClient 4 and the newly introduced support from Spring 3.1. The
RestTemplate accepts the HTTP request factory in it’s constructor – allowing a factory that supports Basic Authentication to be passed into the template – so far, so good. However, using the existing
HttpComponentsClientHttpRequestFactory will prove to be difficult, as the architecture of
RestTemplate was designed
without good support for
HttpContext – an instrumental piece of the puzzle.

Fortunately this is not without a solution, but first, some details on how to do Basic Authentication with the HttpClient library itself. Doing preemptive Basic Authentication with HttpClient 4.x is
a bit of a burden: the authentication info is cached and the process of setting up this authentication cache is very manual and unintuitive:

private HttpContext createHttpContext() {
    // Create AuthCache instance
    AuthCache authCache = new BasicAuthCache();
    // Generate BASIC scheme object and add it to the local auth cache
    BasicScheme basicAuth = new BasicScheme();
    authCache.put(targetHost, basicAuth);
    // Add AuthCache to the execution context
    BasicHttpContext localcontext = new BasicHttpContext();
    localcontext.setAttribute(ClientContext.AUTH_CACHE, authCache);
    return localcontext;
}

HttpResponse response = httpclient.execute
 (targetHost, new HttpGet("/"), createHttpContext());

Now, this
HttpContext needs to be hooked up to the
RestTemplate, so that requests created via the template can benefit from the Basic Authentication support. As mentioned before, this is not an easy as it should be – the entire
HttpComponentsClientHttpRequestFactory needs to be
subclassed and the
createHttpContext method
overridden:

public class HttpComponentsClientHttpRequestFactoryBasicAuth
 extends HttpComponentsClientHttpRequestFactory {
   HttpHost host;

   public HttpComponentsClientHttpRequestFactoryBasicAuth(HttpHost host) {
      super();
      this.host= host;
   }
   protected HttpContext createHttpContext(HttpMethod httpMethod, URI uri) {
      return createHttpContext();
   }
   private HttpContext createHttpContext() {
      ...
   }
}

And with that, everything is in place – the
RestTemplate will now be able to support the Basic Authentication scheme; a simple usage patter would be:

HttpComponentsClientHttpRequestFactory requestFactory =
 (HttpComponentsClientHttpRequestFactory) restTemplate.getRequestFactory();
DefaultHttpClient httpClient =
 (DefaultHttpClient) requestFactory.getHttpClient();
httpClient.getCredentialsProvider().setCredentials(
 new AuthScope(host, port, AuthScope.ANY_REALM),
  new UsernamePasswordCredentials("name", "pass"));

For an in depth discussion on how to secure the RESTful Service itself,
check out this article.

Maven dependencies

The following Maven dependencies are required for the
RestTemplate itself and for the HttpClient library:

<dependency>
   <groupId>org.springframework</groupId>
   <artifactId>spring-webmvc</artifactId>
   <version>3.1.1.RELEASE</version>
</dependency>

<dependency>
   <groupId>org.apache.httpcomponents</groupId>
   <artifactId>httpclient</artifactId>
   <version>4.1.3</version>
</dependency>

Optionally, if the HTTP
Authorization header is constructed manually, then an additional library is required for the encoding support:

<dependency>
   <groupId>commons-codec</groupId>
   <artifactId>commons-codec</artifactId>
   <version>1.6</version>
</dependency>


Conclusion

Although the 3.x branch of development for Apache HttpClient has reached end of life for a while now, and the Spring support for that version has been fully deprecated, much of the information that can be found on
RestTemplate and security still doesn’t account for
the current HttpClient 4.x releases. This article is an attempt to change that through a detailed, step by step discussion on how to set up Basic Authentication with the
RestTemplate and how to use it to consume a secured RESTful Service.

For the next article, the focus will be on setting up
Digest Authentication in the same way. To go beyond the code samples in the article with a production ready implementation of both the consuming side, examined here, but also the actual RESTful service, check out the
REST github project.


Reference:
How to use RestTemplate with Basic Authentication in Spring from our
JCG partner Eugen Paraschiv at the
baeldung blog.

Source : http://www.javacodegeeks.com/2012/05/how-to-use-resttemplate-with-basic.html

Java Memcached on Mac OS X

Introduction

In this article I will explain how you can:

  1. Install and Configure Memcached on Mac OS X
  2. Use Memcached in your Java Application

I won’t go in too much detail about the benefits of using a distributed cache in your applications, but let’s at least provide some use cases for applications that are running in the context of an enterprise portal, eXo Platform in my case –
surprising isn’t? And I will show this in another post.

We have many reasons to use a cache (distributed or not), in the context of enterprise portal, let’s take a look to some of these reasons:

  • A portal is used to aggregate data in a single page. These data could come from different sources : Web Services, Database, ERP, ….. and accessing the data in real time could be costly. So it will be quite interesting to cache the result of the call when possible.
  • If the portal is used to aggregate many data from many sources, it is sometime necessary to jump into another application to continue some operation. A distributed and shared cache could be used to manage some context between different applications running in different processes (JVM or even technologies)

These are two example where a shared cache could be interesting for your portal based applications, we can find many other reason.

Note that the Portlet API (JSR-286) contains already a cache mechanism that cache the HTML fragment, and that eXo Platform also provide a
low level cache, based on
JBoss Cache.

Installation and Configuration

Installing Memcached from sources

You can find some information about Memcached installation on the Memcached
Wiki. The following steps are the steps that I have used on my environment.

As far as I know, Memached is not available as package for Mac OS X. I am still on Snow Leopard (10.6.8), and I have installed XCode and all development tools. I have use the article "Installing memcached 1.4.1 on Mac OS X 10.6 Snow Leopard" from
wincent.com. For simplicity reason I have duplicate the content and updated to the latest releases.

1. Create a working directory :

$ mkdir memcachedbuild
$ cd memcachebuild

2.Install
libevent that is mandatory for memcached

$ curl -O http://www.monkey.org/~provos/libevent-1.4.14-stable.tar.gz
$ tar xzvf libevent-1.4.14-stable.tar.gz
$ cd libevent-1.4.14-stable
$ ./configure
$ make
$ make verify
$ sudo make install 

3. Install memcached

Go back to your install directory (
memcachedbuild)

$ curl -O http://memcached.googlecode.com/files/memcached-1.4.10.tar.gz
$ tar xzvf memcached-1.4.10.tar.gz
$ cd memcached-1.4.10
$ ./configure
$ make
$ make test
$ sudo make install 

You are now ready to use memcached that is available at /usr/local/bin/memcached

This allows you to avoid changing to the pre-installed memcached located in /usr/bin, if you want to replace it instead of having you own install, just run the configure command with the following parameter: ./configure –prefix=/usr

Starting and testing Memcached

Start the memcached server, using the following command line:

$ /usr/local/bin/memcached -d -p 11211

This command starts the memcached server as demon (-d parameter), on the TCP port 11211 (this is the default value). You can find more about the memcached command using man memcached.

It is possible to connect and test your server using a telnet connection. Once connected you can set and get object in the cache, take a look to the following paragraph.

$ telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to tgrall-server.
Escape character is '^]'.
set KEY 0 600 16
This is my value
STORED
get KEY
VALUE KEY 0 16
This is my value
END

The set command allows you to put a new value in the cache using the following syntax:

set <key> <flags> <expiration_time> <number_of_bytes> [noreply] \n\n

<value>
  • key : the key used to store the data in the cache
  • flags : a 32 bits unsigned integer that memcached stored with the data
  • expiration_time : expiration time in seconds, if you put 0 this means no delay
  • number_if_bytes : number of bytes in the data block
  • noreply : option to tell the server to not return any value
  • value : the value to store and associate to the key.

This is a short view of the documentation located in your source directory /memcachedbuild/memcached-1.4.10/doc/protocol.txt .

The get command allows you to access the value that is associated with the key.

You can check the version of memcahed you are running by calling the stats command in your telnet session.

Your memcached server is up and running, you can now start to use it inside your applications.

Simple Java Application with Memcached

The easiest way to use memcached from your Java applications is to use a client library. You can find many
client libraries. In this example I am using
spymemcached developped by the people from
Couchbase.

1. Adding SpyMemcached to your Maven project

Add the repository to you pom.xml (or you setting.xml)

<repository>
    <id>spy</id>
    <name>Spy Repository</name>
    <layout>default</layout>
    <url>http://files.couchbase.com/maven2/</url>
</repository>

then the dependency to your pom.xml

<dependency>
    <groupid>spy</groupid>
    <artifactid>spymemcached</artifactid>
    <version>2.7.3</version>
</dependency>

2. Use SpyMemcache client in your application

The following code is a simple Java class that allows you to enter the key and the value and set it in the cache.

package com.grallandco.blog;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.Console;
import java.io.InputStreamReader;
import java.util.Date;
import java.util.logging.Level;
import java.util.logging.Logger;
import net.spy.memcached.AddrUtil;
import net.spy.memcached.MemcachedClient;

public class Test {

    public static void main(String[] args) {
       try {
           
           System.out.print("Enter the new key : ");
           BufferedReader reader = new BufferedReader( new InputStreamReader(System.in));
           String key = null;
           key = reader.readLine();
           
           System.out.print("Enter the new value : ");
           String value = null;
           value = reader.readLine();
           
            MemcachedClient cache = new MemcachedClient(AddrUtil.getAddresses("127.0.0.1:11211"));
            
            // read the object from memory
            System.out.println("Get Object before set :"+ cache.get(key)  );

            // set a new object            
            cache.set(key, 0, value );

            System.out.println("Get Object after set :"+ cache.get(key)  );
            

        } catch (IOException ex) {
            Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
            System.exit(0);
        }

       
        System.exit(0);
       
    }
}

So when executing the application you will see something like :

Enter the new key : CITY
Enter the new value : Paris, France
2011-11-16 15:22:09.928 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=/127.0.0.1:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2011-11-16 15:22:09.932 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@5b40c281
Get Object before set :null
Get Object after set :Paris, France

You can also access the object from a Telnet session:

get CITY
VALUE CITY 0 13
Paris, France
END

You can use any Java class in your application, the only thing to do is to make this class serializable.

This is it for the first post about memcached and Java, I am currently working on a small example integrating Web Services call, Portlets and memcached.


Reference:
Installing Memcached on Mac OS X and using it in Java from our
JCG partner Tugdual Grall at the
Tug’s Blog blog.

Source : http://www.javacodegeeks.com/2012/05/java-memcached-on-mac-os-x.html

Spring 3 and Java EE 6 – An unfair and incomplete comparison

The first draft of this small article had the title ‘Spring & Java EE – Comparing Apples and Oranges’. During writing this, I learned that it is possible to compare Spring Framework and Java EE, but it is always an unfair and incomplete work.

The evolution of Java for Enterprise and Spring Framework are strongly connected to each other. Both leaned from each other and partly copied good concepts. In Table 1 you can see a simplified timeline with some key milestones of Java Platform for Enterprise and Spring Framework.

This table explains, that it makes only sense to compare Java EE v6 with Spring v3.0. Earlier versions of Spring Framework and J2EE are outdated and Java EE v7 is still not published (actually delayed to
Q2 2013).

Table 1: Timeline of Java Platforms and Spring Framework

Year Java Platform, Standard Edition Java Platform, Enterprise Edition Spring Framework Key Milestones
2000 J2SE v1.3

J2EE v1.2.1

EJB 2
– difficult deployment descriptors
– difficult testing
– lot of redundantartifacts
2001

J2SE v1.3

2002

2003 J2SE v1.4 J2EE v1.4

2004

Spring v1.0 First Spring Framework
– dependency injection (IoP)
– no Java EE application server
-competitor to J2EE
2005

Spring v1.2

2006 J2SE v5 Java EE v5 Spring v2.0 Java EE 5 (EJB 3.0)
– a lot of functions inspired by Spring
– elimination of component, home and remote interfaces
2007

Spring v2.5

2008

2009 Java SE v6 Java EE v6 Spring v3.0 Java EE 6 (EJB 3.1)
– interfaces are optional
– singleton beans
– cron-like scheduling
– embeddable containers
Spring 3.0
– Spring Expression Language
– MVC-Framework with improved REST support
2010

2011 Java SE v 7

Spring v 3.1

So, in table 2 the main building blocks of Spring Framework are listed. The Inversion of Control Container is the core function of Spring Framework. To understand how this works you may also check
A minimal Java application based on Spring framework(sample code uses Spring 2.5.6.A, but the main principles are the same for Spring 3.0).

Table 2: Selected Building Blocks of the Spring v3.0 Framework

Inversion of Control Container – Inversion of Control
– Dependency Injection
– Configuration with XML files and/or annotations (auto wiring)
Model-View-Controller Framework – Domain objects (Models)
-Usually JSP templates (Views)
-DispatcherServlet as front-controller for Controller
Aspect-Oriented Programming Framework – Basic AOP for cross-cutting concerns in aspects
– Interception based and is configured at runtime
– Configuration with XML files and/or annotations
Batch Framework -Processing large volumes of records or tasks.Including: logging, tracing, transactions, job management, resource management, exception converting
Data Access Framework – Support is provided for popular frameworks
– JDBC, iBatis, Hibernate, JDO, JPA, Oracle TopLink, Apache OJB, and Apache Cayenne
Transaction Management Framework – Abstraction mechanism (JTA only supports nested transactions and global transactions, and requires an application server)
Remote Access Framework – Working with various RPC-based technologies available on the Java platform both for client connectivity and exporting objects on servers

Source & Further Reading:
http://en.wikipedia.org/wiki/Spring_Framework

In table 3 the main standards and components of the Java Platform, Enterprise Edition 6 are listed. This table makes clear that Java EE 6 contains of a lot of standards and is not just a framework.

Table 3: Selected Building Blocks of the Java EE 6 Architecture

Source of Picture The Java EE 6Tutorial,p39

Client Machine
– Java Server Faces (JSF 2.0)
– JSP Standard Tag Library (JSTL)
– Ajax with JavaServer Faces Technology
– Facelets (XHTML)
– Support for the Expression Language (EL)
– Templating for components and pages
– Java Servlet Technology (Servlet 3.0)
– Internationalizing and Localizing Web Applications
Java EE Server
– Enterprise JavaBeans (enterprise bean) components
– JAX-RS RESTful web services
– JAX-WS web service endpoints
– Enterprise Beans as POJO (Session and Message-driven)
– Managed Beans as POJO
– Interceptors
– Dependency Injection for Java (JSR 330)
– Contexts and Dependency Injection, CDI (JSR 299)
– Annotations to minimize deployment descriptor needs
– Asynchronous method invocation in session beans
– Declarative and programmatic security
– Java Message Service (JMS) API
– Java Persistence API entities
Persistance
– Java Database Connectivity API (JDBC)
– Java Persistence API 2.0
– Java EE Connector Architecture
– Java Transaction API (JTA)

Source & Further Reading:
http://olex.openlogic.com/wazi/2010/get-started-with-jee6/

Practical Experiences

When I started to learn Spring Framework 3.0 some years ago, I implemented a small web application with a lot of Spring functions. To be honest, more than necessary for this task. This small application had extensive test code for unit and integration test automation. It has 100% line and branch coverage. Some time later, I decided to implement exactly the same application based on Java EE 6 Architecture to compare the two approaches.Both technologies worked well, have almost the same code size and a good maintainability.

One significant difference is the support of testing. Spring Framework has an excellent support for testing andJava EE 6 has here some weaknesses. ForJava EE 6you can useEmbedded Glassfish, but this approach is annoying slow (long start-up time for embedded container) and touchy in the configuration.

A further outstanding feature of Spring Framework is the easy integration of legacy applications. It is easier to renew an old application in a step by step approach, e.g. with JDBC, transaction management and small parts of IoC in the beginning.

Conclusion

  1. Form the architectural point of view the models of Spring and Java EE are competitors. Depending on the strategy and constraints of your organization both show strengths and weaknesses.
  2. A combination of both can’t generally be recommended, because it is either waste of the EJB Container functionality and/or difficult to maintain.In some selected cases, it may make sense to use Building Blocks of Spring Framework in aJava EE 6application, e.g. Spring Batch, Data Access.
  3. Spring Framework is not a standard – it is a product and an implementation of a framework. This means, there is no alternative vendor. Please, keep this always in mind.

Reference:
Lightweight Java Enterprise Architectures – An Unfair and Incomplete Comparison Between Spring 3 Framework and Java EE 6 from our
JCG partner Markus Sprunck at the
Software Engineering Candies blog.

Source : http://www.javacodegeeks.com/2012/05/spring-3-and-java-ee-6-unfair-and.html

Binary websockets with Play 2.0 and Scala

In a recent
article I showed how you can use webrtc, canvas and websockets together to create a face detection application whose frontend runs completely in the browser, without the need for plugins. In that article I used a Jetty based backend to handle the image analysis using OpenCV through the JavaCV wrapper.

When I almost finished the article, I noticed that websockets is also supported from Play 2.0. I really like developping in Play and in Scala so as an experiment I rewrote the backend part from a Jetty/Java/JavaCV stack to a Play2.0/Scala/JavaCV stack. If you want to do this for yourself, make sure you start with the frontend code from
here. Since the frontend code hasn’t changed except the location where the websockets are listening.

Setting up the Play 2.0 environment

I’m not going to talk too much about how to start a Play 2.0/Scala project. you can find the details in some of my other posts should you need more information. What we do need to do, is setup the dependencies for JavaCV so that they can be used from Play 2. I’ve manually added them to my local ivy repository, so that I can reference them from the sbt configuration like any other dependency. For my example I created the following directory layout for the JavaCV libraries:

./play-2.0-RC2/repository/cache/javacv
./play-2.0-RC2/repository/cache/javacv/javacpp
./play-2.0-RC2/repository/cache/javacv/javacpp/ivy-2.3.1.xml
./play-2.0-RC2/repository/cache/javacv/javacpp/ivydata-2.3.1.properties
./play-2.0-RC2/repository/cache/javacv/javacv
./play-2.0-RC2/repository/cache/javacv/javacv/ivy-2.3.1.xml
./play-2.0-RC2/repository/cache/javacv/javacv/ivydata-2.3.1.properties
./play-2.0-RC2/repository/cache/javacv/javacv-macosx-x86_64
./play-2.0-RC2/repository/cache/javacv/javacv-macosx-x86_64/ivy-2.3.1.xml
./play-2.0-RC2/repository/cache/javacv/javacv-macosx-x86_64/ivydata-2.3.1.properties
./play-2.0-RC2/repository/local/javacv
./play-2.0-RC2/repository/local/javacv/javacpp
./play-2.0-RC2/repository/local/javacv/javacpp/2.3.1
./play-2.0-RC2/repository/local/javacv/javacpp/2.3.1/ivys
./play-2.0-RC2/repository/local/javacv/javacpp/2.3.1/ivys/ivy.xml
./play-2.0-RC2/repository/local/javacv/javacpp/2.3.1/jars
./play-2.0-RC2/repository/local/javacv/javacpp/2.3.1/jars/javacpp.jar
./play-2.0-RC2/repository/local/javacv/javacv
./play-2.0-RC2/repository/local/javacv/javacv/2.3.1
./play-2.0-RC2/repository/local/javacv/javacv/2.3.1/ivys
./play-2.0-RC2/repository/local/javacv/javacv/2.3.1/ivys/ivy.xml
./play-2.0-RC2/repository/local/javacv/javacv/2.3.1/jars
./play-2.0-RC2/repository/local/javacv/javacv/2.3.1/jars/javacv.jar
./play-2.0-RC2/repository/local/javacv/javacv-macosx-x86_64
./play-2.0-RC2/repository/local/javacv/javacv-macosx-x86_64/2.3.1
./play-2.0-RC2/repository/local/javacv/javacv-macosx-x86_64/2.3.1/ivys
./play-2.0-RC2/repository/local/javacv/javacv-macosx-x86_64/2.3.1/ivys/ivy.xml
./play-2.0-RC2/repository/local/javacv/javacv-macosx-x86_64/2.3.1/jars
./play-2.0-RC2/repository/local/javacv/javacv-macosx-x86_64/2.3.1/jars/javacv-macosx-x86_64.jar

As you can see from this listing, I just added the three javacv supplied jars to my local repository. I also added a minimal ivy.xml so that they can be used from ivy and sbt. This minimal ivy.xml looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
    <info
        organisation="javacv"
        module="javacv-macosx-x86_64"
        revision="2.3.1"
        status="release">
        </info>

        <publications>
            <artifact type="jar"/>
        </publications>
</ivy-module>

With these files added to my repository I can setup the dependencies for the Play 2.0 project in the Build.scala file.

import sbt._
import Keys._
import PlayProject._

object ApplicationBuild extends Build {

    val appName         = "PlayWebsocketJavaCV"
    val appVersion      = "1.0-SNAPSHOT"

    val appDependencies = Seq(
      "javacv" % "javacv" % "2.3.1",
      "javacv" % "javacpp" % "2.3.1",
      "javacv" % "javacv-macosx-x86_64" % "2.3.1"
    )

    val main = PlayProject(appName, appVersion, appDependencies, mainLang = SCALA).settings(
      // Add your own project settings here      
    )
}

Now run "play update" and "play eclipsify" to update the dependencies and your Eclipse configuation (if you’re using Eclipse that is).

Configure websockets in Play

Using websockets in Play 2.0 is very easy. The first thing you need to do is add the URL to your routes configuration in the conf directory.

GET     /wsrequest                  controllers.Application.wsrequest

And, of course, you need to implement the action this route points to:

/**
 * Simple websocket listener configured in play 2. This uses a synchronous model, where
 * the same channel is used to send the response. For this usecase this is useful, if
 * we want async processing we could have used Akka actors together with play 2.0 async
 * support.
 */
def wsrequest = WebSocket.using[Array[Byte]] { request =>

  // Create the outbound value that is called for each
  val out = Enumerator.imperative[Array[Byte]]();

 val in = Iteratee.foreach[Array[Byte]](content => {
    out.push(FaceDetect.detect(content));
  })

  // tie the in and out values to each other
  (in, out)
}

In this code we configure an input channel (in), and an output channel (out) and connect them to the socket. Whenever the HTML5 client sends a request over the websocket our "in" method is called, and when we want to send something to the client we can use the "out" channel. The "in" channel needs to be defined as an Iteratee (more info see
these Play docs). What this does is, that for each input message we receive we run the specifici method. In this case we run the FaceDetect.detect operation (more on this later) and the result from this operation is pushed back to the client using the "out" channel. This "out" channel itself is defined as an Enumerator (see
these play docs). We can attach different listeners if we want to this enumerator, but in this case we don’t do anything with the message, just pass it along to the client.

Using JavaCV from scala

The last step is the code of the FaceDetect.detect function. The java version, see earlier mentioned
article, is very easily converted to a scala one.

package javacv

import com.googlecode.javacv.cpp.opencv_core._
import com.googlecode.javacv.cpp.opencv_imgproc._
import com.googlecode.javacv.cpp.opencv_highgui._
import com.googlecode.javacv.cpp.opencv_objdetect._
import com.googlecode.javacpp.BytePointer
import java.nio.ByteBuffer
import javax.imageio.ImageIO
import java.io.ByteArrayOutputStream
import scala.tools.nsc.io.VirtualFile

object FaceDetect {

 var CASCADE_FILE =PATH_TO_CASCADE_FILE;
   var minsize = 20;
 var group = 0;
 var scale = 1.1;

  def detect(imageData:Array[Byte]) : Array[Byte] = {

    // we need to wrap the input array, since BytePointer doesn't accept
    // a bytearray as input. It accepts a byte varargs, but Array[Byte]
    // doesn't convert automatically
    var wrappedData = ByteBuffer.wrap(imageData);
    var originalImage = cvDecodeImage(cvMat(1, imageData.length,CV_8UC1, new BytePointer(wrappedData)));

    // convert to grayscale for easy detection
    var grayImage = IplImage.create(originalImage.width(), originalImage.height(), IPL_DEPTH_8U, 1);
    cvCvtColor(originalImage, grayImage, CV_BGR2GRAY);

    // storage is needed to store information during detection
 var storage = CvMemStorage.create();

 // load and run the cascade
 var cascade = new CvHaarClassifierCascade(cvLoad(CASCADE_FILE));
 var faces = cvHaarDetectObjects(grayImage, cascade, storage, scale, group, minsize);

 // draw a rectangle for the detected faces
 for (i <- 0 until faces.total) {
   var r = new CvRect(cvGetSeqElem(faces, i));
   cvRectangle(originalImage, cvPoint(r.x, r.y),
     cvPoint(r.x + r.width(), r.y + r.height),
     CvScalar.YELLOW, 1, CV_AA, 0);
 }

 // convert to bytearray and return
 var bout = new ByteArrayOutputStream();
 var imgb = originalImage.getBufferedImage();
 ImageIO.write(imgb, "png", bout);

    bout.toByteArray()
  }
}

The only issue I ran into was with the BytePointer constructor. One of the signatures accepts a varargs of the type byte. In java this allows me to just supply this constructor with a byte[], in Scala however this doesn’t work. Luckily though, a BytePointer can also be created using a ByteBuffer. For the rest this is a one-to-one conversion of Java to Scala.

Running the code

And that’s almost it. By default play listens on port 9000, in the Jetty based example we had the server running on 9999 and listening to the root context. To work with our Play2/Scala based server we just need to point the browser to the correct websocket server url.

 ws = new WebSocket("ws://127.0.0.1:9000/wsrequest");

And now, when we run it, we use Play 2 as our server and run the JavaCV code using scal. And more importantly it still works:


Reference:
Binary websockets with Play 2.0 and Scala (and a bit op JavaCV/OpenCV) from our
JCG partner Jos Dirksen at the
Smart Java blog.

Source : http://www.javacodegeeks.com/2012/05/binary-websockets-with-play-20-and.html