Configure logging in Google App Engine

Google App Engine for Java is using java.util.logging.Logger to perform the logging. In this tutorial, we show you how to do logging in GAE environment.

1. Logging Example

Example to define a logger in MovieController, and log the messages in different logging levels (info, warning and error)

import java.util.logging.Logger;
//...
 
@Controller
@RequestMapping("/movie")
public class MovieController {
 
private static final Logger log = Logger.getLogger(MovieController.class.getName());
 
@RequestMapping(value="/{name}", method = RequestMethod.GET)
public String getMovie(@PathVariable String name, ModelMap model) {
 
log.info("Information log message.");
 
log.warning("Warning log message.");
 
log.severe("Error log message.");
 
return "page";
 
}
 
}

2. Logging Example

Create a file, logging.properties, set the logging level, and put it in “$project/war/WEB-INF

File : $project/war/WEB-INF/logging.properties

# Set the default logging level for all loggers to WARNING
.level = WARNING

3. appengine-web.xml

Update appengine-web.xml, define system properties tag, and point logging to above “WEB-INF/logging.properties“.

File : $project/war/WEB-INF/appengine-web.xml

<?xml version="1.0" encoding="utf-8"?>
<appengine-web-app xmlns="http://appengine.google.com/ns/1.0">
<application>mkyong-springmvc</application>
<version>1</version>
 
<!-- Configure java.util.logging -->
<system-properties>
<property name="java.util.logging.config.file" value="WEB-INF/logging.properties"/>
</system-properties>
 
</appengine-web-app>

4. Done

In GAE local development environment, all logged message will be display on console. In GAE production environment, you can access the logged messages in your application’s administrator page.

gae java logging gae java logging

Cron job on Google App Engine for Java

In this tutorial, we will show you how to create a scheduler task or cron job on Google App Engine, for Java. In GAE, cron job uses HTTP GET request to call an URL, the duration of the cron job running time is limited to run up to 10 minutes only, if excess, GAE kill your job.

To create a cron job on GAE, just define cron jobs in a file “cron.xml“, and put in the “WEB-INF” folder.

Cron Job Tutorial

Now, we will create a cron job on GAE, and schedule call an URL from Spring MVC REST example.

  1. Google App Engine Java SDK 1.6.3.1
  2. Spring 3.1.1
  3. JDK 1.6
  4. Eclipse 3.7 + Google Plugin for Eclipse

1. Spring Controller

A simple Spring REST controller.

package com.mkyong.controller;
 
import org.springframework.stereotype.Controller;
import org.springframework.ui.ModelMap;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
 
@Controller
@RequestMapping("/cron")
public class CronController {
 
static int total = 0;
 
@RequestMapping(value="/count", method = RequestMethod.GET)
public String getCount(ModelMap model) {
 
model.addAttribute("total", total);
 
return "list";
 
}
 
//cron job running this
@RequestMapping(value="/addCount/{num}", method = RequestMethod.GET)
public String addCount(@PathVariable int num, ModelMap model) {
 
total += num;
model.addAttribute("total", total);
 
return "list";
 
}
 
}

File : list.jsp – Page to display the total counts.

<html>
<body>
<h1>GAE + Spring 3 MVC REST + CRON Example</h1>
 
<h3>Counts : ${total} </h3>
 
</body>
</html>

2. cron.xml

Create a “cron.xml” and put in the “WEB-INF” folder. This cron job will call URL “/cron/addCount/1” every 1 minutes.

File : cron.xml

<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/cron/addCount/1</url>
<description>Add count + 1 every 1 minutes</description>
<schedule>every 1 minutes</schedule>
</cron>
</cronentries>

Note
The cron job schedule format is a simple English-like format. Please read this GAE scheduler format for more detail.

every N (hours|mins|minutes) ["from" (time) "to" (time)]

3. Output

Deployed on GAE, you can view the cron job status on application administrator page.

gae cron example gae cron example

URL : http://mkyong-springmvc-cron.appspot.com/cron/count , result after one minutes

cron job on gae cron job on gae

Download Source Code

Due to large file size, all Spring and GAE jars are excluded.

Download – CronJob-GoogleAppEngine-Example.zip (13 KB)

10 Java Regular Expression Examples You Should Know

Regular expression is an art of the programing, it’s hard to debug , learn and understand, but the powerful features are still attract many developers to code regular expression. Let’s explore the following 10 practical regular expression ~ enjoy 🙂

1. Username Regular Expression Pattern

 ^[a-z0-9_-]{3,15}$

^                    # Start of the line
[a-z0-9_-] # Match characters and symbols in the list, a-z, 0-9 , underscore , hyphen
{3,15} # Length at least 3 characters and maximum length of 15
$ # End of the line

==> See the explanation and example here

2. Password Regular Expression Pattern

((?=.*\d)(?=.*[a-z])(?=.*[A-Z])(?=.*[@#$%]).{6,20})

(			# Start of group
(?=.*\d) # must contains one digit from 0-9
(?=.*[a-z]) # must contains one lowercase characters
(?=.*[A-Z]) # must contains one uppercase characters
(?=.*[@#$%]) # must contains one special symbols in the list "@#$%"
. # match anything with previous condition checking
{6,20} # length at least 6 characters and maximum of 20
) # End of group

==> See the explanation and example here

3. Hexadecimal Color Code Regular Expression Pattern

^#([A-Fa-f0-9]{6}|[A-Fa-f0-9]{3})$

^		 #start of the line
# # must constains a "#" symbols
( # start of group #1
[A-Fa-f0-9]{6} # any strings in the list, with length of 6
| # ..or
[A-Fa-f0-9]{3} # any strings in the list, with length of 3
) # end of group #1
$ #end of the line

==> See the explanation and example here

4. Email Regular Expression Pattern

^[_A-Za-z0-9-]+(\\.[_A-Za-z0-9-]+)*@[A-Za-z0-9]+
(\\.[A-Za-z0-9]+)*(\\.[A-Za-z]{2,})$

^			#start of the line
[_A-Za-z0-9-]+ # must start with string in the bracket [ ], must contains one or more (+)
( # start of group #1
\\.[_A-Za-z0-9-]+ # follow by a dot "." and string in the bracket [ ], must contains one or more (+)
)* # end of group #1, this group is optional (*)
@ # must contains a "@" symbol
[A-Za-z0-9]+ # follow by string in the bracket [ ], must contains one or more (+)
( # start of group #2 - first level TLD checking
\\.[A-Za-z0-9]+ # follow by a dot "." and string in the bracket [ ], must contains one or more (+)
)* # end of group #2, this group is optional (*)
( # start of group #3 - second level TLD checking
\\.[A-Za-z]{2,} # follow by a dot "." and string in the bracket [ ], with minimum length of 2
) # end of group #3
$ #end of the line

==> See the explanation and example here

5. Image File Extension Regular Expression Pattern

([^\s]+(\.(?i)(jpg|png|gif|bmp))$)

(			#Start of the group #1
[^\s]+ # must contains one or more anything (except white space)
( # start of the group #2
\. # follow by a dot "."
(?i) # ignore the case sensitive checking
( # start of the group #3
jpg # contains characters "jpg"
| # ..or
png # contains characters "png"
| # ..or
gif # contains characters "gif"
| # ..or
bmp # contains characters "bmp"
) # end of the group #3
) # end of the group #2
$ # end of the string
) #end of the group #1

==> See the explanation and example here

6. IP Address Regular Expression Pattern

^([01]?\\d\\d?|2[0-4]\\d|25[0-5])\\.([01]?\\d\\d?|2[0-4]\\d|25[0-5])\\.
([01]?\\d\\d?|2[0-4]\\d|25[0-5])\\.([01]?\\d\\d?|2[0-4]\\d|25[0-5])$

^		#start of the line
( # start of group #1
[01]?\\d\\d? # Can be one or two digits. If three digits appear, it must start either 0 or 1
# e.g ([0-9], [0-9][0-9],[0-1][0-9][0-9])
| # ...or
2[0-4]\\d # start with 2, follow by 0-4 and end with any digit (2[0-4][0-9])
| # ...or
25[0-5] # start with 2, follow by 5 and end with 0-5 (25[0-5])
) # end of group #2
\. # follow by a dot "."
.... # repeat with 3 time (3x)
$ #end of the line

==> See the explanation and example here

7. Time Format Regular Expression Pattern

Time in 12-Hour Format Regular Expression Pattern

(1[012]|[1-9]):[0-5][0-9](\\s)?(?i)(am|pm)

(				#start of group #1
1[012] # start with 10, 11, 12
| # or
[1-9] # start with 1,2,...9
) #end of group #1
: # follow by a semi colon (:)
[0-5][0-9] # follow by 0..5 and 0..9, which means 00 to 59
(\\s)? # follow by a white space (optional)
(?i) # next checking is case insensitive
(am|pm) # follow by am or pm

==> See the explanation and example here

Time in 24-Hour Format Regular Expression Pattern

([01]?[0-9]|2[0-3]):[0-5][0-9]

(				#start of group #1
[01]?[0-9] # start with 0-9,1-9,00-09,10-19
| # or
2[0-3] # start with 20-23
) #end of group #1
: # follow by a semi colon (:)
[0-5][0-9] # follow by 0..5 and 0..9, which means 00 to 59

==> See the explanation and example here

8. Date Format (dd/mm/yyyy) Regular Expression Pattern

(0?[1-9]|[12][0-9]|3[01])/(0?[1-9]|1[012])/((19|20)\\d\\d)

(			#start of group #1
0?[1-9] # 01-09 or 1-9
| # ..or
[12][0-9] # 10-19 or 20-29
| # ..or
3[01] # 30, 31
) #end of group #1
/ # follow by a "/"
( # start of group #2
0?[1-9] # 01-09 or 1-9
| # ..or
1[012] # 10,11,12
) # end of group #2
/ # follow by a "/"
( # start of group #3
(19|20)\\d\\d # 19[0-9][0-9] or 20[0-9][0-9]
) # end of group #3

==> See the explanation and example here

9. HTML tag Regular Expression Pattern

<("[^"]*"|'[^']*'|[^'">])*>

<	  	#start with opening tag "<"
( # start of group #1
"[^"]*" # only two double quotes are allow - "string"
| # ..or
'[^']*' # only two single quotes are allow - 'string'
| # ..or
[^'"
>] # cant contains one single quotes, double quotes and ">"
) # end of group #1
* # 0 or more
> #end with closing tag ">"

==> See the explanation and example here

10. HTML links Regular Expression Pattern

HTML A tag Regular Expression Pattern

(?i)<a([^>]+)>(.+?)</a>

(		#start of group #1
?i # all checking are case insensive
) #end of group #1
<a #start with "<a"
( # start of group #2
[^>]+ # anything except (">"), at least one character
) # end of group #2
> # follow by ">"
(.+?) # match anything
</a> # end with "</a>

Extract HTML link Regular Expression Pattern

\s*(?i)href\s*=\s*(\"([^"]*\")|'[^']*'|([^'">\s]+));

\s*			   #can start with whitespace
(?i) # all checking are case insensive
href # follow by "href" word
\s*=\s* # allows spaces on either side of the equal sign,
( # start of group #1
"([^"]*") # only two double quotes are allow - "string"
| # ..or
'[^']*' # only two single quotes are allow - 'string'
| # ..or
([^'"
>]+) # cant contains one single / double quotes and ">"
) # end of group #1

==> See the explanation and example here

Spring Batch Tutorial

spring batch model spring batch model

Photo credit: Spring Source

Spring Batch, is an open source framework for batch processing – execution of a series of jobs. Spring Batch provides classes and APIs to read/write resources, transaction management, job processing statistics, job restart and partitioning techniques to process high-volume of data.

The following Spring Batch tutorials and examples are tested with :

  1. Spring Core 3.2.2.RELEASE
  2. Spring Batch 2.2.0.RELEASE

P.S Spring Batch is part of the Spring Portfolio.

1. Quick Guide

A simple batch job, and run with command line.

  • Spring Batch Hello World Example
    Short description about Spring batch, and create a job to read data from a CSV File, process it, and write it to an XML file (JAXB).
  • Spring Batch TaskletStep example
    Example to cleanup the resources after the batch job has completed.
  • Run Spring Batch Job With CommandLineJobRunner
    Run Spring batch job with command line.

2. ItemReader, ItemProcessor, ItemWriter

Few examples to show the use of Spring batch classes to read/write resources (csv, xml and database).

  • Spring Batch Example – CSV File To MySQL database
    Read data from a CSV file and write it into a MySQL database, job meta is stored in the database.
  • Spring Batch Example – XML File To MongoDB database
    Read data from a XML file (XStream) and write it into a nosql database MongoDB, also unit test the batch job.
  • Spring Batch Example – XML File To CSV File
    Read data from a XML file (JAXB2), process with ItemProcessor and write it into a CSV file. It also shows you how to convert Date and BigDecimal data type to/from object via JAXB2.
  • Spring Batch Example – MySQL Database To XML
    Read data from a MySQL database and write it into an XML file (XStream), also the use of jobParameters.
  • Spring Batch MultiResourceItemReader example
    A ItemReader to read multiple files.

3. Scheduler

Run batch job with scheduler frameworks.

  • Spring Batch + Spring TaskScheduler example
    Spring TaskScheduler to schedule a batch job to run every 5 seconds.
  • Spring Batch + Quartz Scheduler example
    Integrate Quartz to schedule a batch job to run every 10 seconds.

4. Unit Test

  • Spring Batch unit test example
    How to unit test a batch job or individual steps.

5. Advance

  • Spring Batch partitioner example
    Multiple threads to process range of data.
  • Spring Batch listener example
    Examples to intercept the step execution of a batch job.

6. Spring Batch FAQs

Some common questions and answers in Spring Batch.

  • How to convert Date in BeanWrapperFieldSetMapper
  • Spring Batch : A job instance already exists and is complete for parameters={}
  • Spring Batch metadata tables are not created automatically?
  • NoSuchBeanDefinitionException : No qualifying bean of type JobLauncherTestUtils
  • jobParameters cannot be found on object of type BeanExpressionContext
  • Spring Batch official website
  • Spring Batch Reference Documentation
  • Wikipedia : Spring Batch
  • Spring Batch Samples
Next >
Index Spring Batch Hello World Example

GAE : how to output log messages to a file

By default, all logging messages will output to log console. To change the logging settings, find this file – {Google App Engine SDK directory}\google\appengine\tools\dev_appserver_main.py.

File : dev_appserver_main.py – Find following pattern

#...
import getopt
import logging
import os
import signal
import sys
import tempfile
import traceback
 
logging.basicConfig(
level=logging.INFO,
format='%(levelname)-8s %(asctime)s %(filename)s:%(lineno)s] %(message)s')
#...

Output to File

In order to output log messages to a file, we can change the configuration of logging in dev_appserver_main.py like in below:

#...
import getopt
import logging
import os
import signal
import sys
import tempfile
import traceback
 
# default , comment out
#logging.basicConfig(
# level=logging.INFO,
# format='%(levelname)-8s %(asctime)s %(filename)s:%(lineno)s] %(message)s')
 
# new log settings , output to a file
logging.basicConfig(
filename='/Users/lokjack/gae.log',
filemode='a',
level=logging.DEBUG,
format='%(levelname)-8s %(asctime)s %(filename)s:%(lineno)s] %(message)s')
#...

Restart the dev_appserver.py after changed the dev_appserver_main.py.

Now, the log console would not show any log messages, instead all output to a file (In this example, all log messages will be output to “/Users/lokjack/gae.log“).

Note
This hacks only works on local GAE development environment.

Download Source Code

Download it – gae-logging-to-file.zip (11 kb)

How to install Google Plugin for Eclipse

In this tutorials, we will show you how to install “Google Plugin for Eclipse“, in Eclipse 3.7 (Indigo).

1. Installation

In Eclipse 3.7, click “Help” –> “Install New Software…“, copy and paste following URL :

http://dl.google.com/eclipse/plugin/3.7

Note
For other Eclipse version like 3.3, 3.4, 3.5, 3.6, please refer to this GAE Eclipse documentation.

Figure : Select “Google Plugin for Eclipse (required), and Google App Engine SDK for Java.

install google plugin for eclipse 3.7 install google plugin for eclipse 3.7
Note
The Google App Engine Java SDK is optional, you can download it together with the plugin, Or, download manually from GAE website and link it to “Google Plugin for Eclipse” later.

2. Verification

Wait few minutes for the installation progress, when done, Eclipse prompts you to restart, click yes and the Google Plugin for Eclipse is installed.

Figure – A small Google icon is available in the Eclipse toolbar.

instal googe plugin for eclipse 3.7 success instal googe plugin for eclipse 3.7 success
Don’t use JRE 7
At this moment, don’t use JRE 7 runtime in Eclipse, it will caused the “Google Plugin for Eclipse” failed to install, and prompts you many “jar has been tampered” error message. See this thread.

Top 5 open source Q&A systems

List of the open source Q&A implementation or stackoverflow-like website, with “Active” development and promising features and community.

P.S This list is not in any particular order.

1. OSQA

osqa osqa

2. Shapado

shapado shapado

3. Askbot

askbot askbot

4. Question2Answer

question2answer question2answer

5. LampCMS

lampcms lampcms
Best Engine ?
Personally, i will go for OSQA, for their huge community in DZone.