SPRING BATCH

Spring Batch, is an open source framework for batch processing – execution of a series of jobs. Spring Batch provides classes and APIs to read/write resources, transaction management, job processing statistics, job restart and partitioning techniques to process high-volume of data.


Spring Batch is a framework for batch processing – execution of a series of jobs. In Spring Batch, A job consists of many steps and each step consists of a READ-PROCESS-WRITE task or single operation task (tasklet).

1. For “READ-PROCESS-WRITE” process, it means “read” data from the resources (csv, xml or database), “process” it and “write” it to other resources (csv, xml and database). For example, a step may read data from a CSV file, process it and write it into the database. Spring Batch provides many made Classes to read/write CSV, XML and database.
2. For “single” operation task (tasklet), it means doing single task only, like clean up the resources after or before a step is started or completed.
3. And the steps can be chained together to run as a job.

1 Job = Many Steps.
1 Step = 1 READ-PROCESS-WRITE or 1 Tasklet.
Job = {Step 1 -> Step 2 -> Step 3} (Chained together)

Spring Batch Examples

Consider following batch jobs :

1. Step 1 – Read CSV files from folder A, process, write it to folder B. “READ-PROCESS-WRITE”
2. Step 2 – Read CSV files from folder B, process, write it to the database. “READ-PROCESS-WRITE”
3. Step 3 – Delete the CSB files from folder B. “Tasklet”
4. Step 4 – Read data from a database, process and generate statistic report in XML format, write it to folder C. “READ-PROCESS-WRITE”
5. Step 5 – Read the report and send it to manager email. “Tasklet”

In Spring Batch, we can declare like the following :

  <job id="abcJob" xmlns="http://www.springframework.org/schema/batch">
 <step id="step1" next="step2">
   <tasklet>
  <chunk reader="cvsItemReader" writer="cvsItemWriter"  
                    processor="itemProcesser" commit-interval="1" />
   </tasklet>
 </step>
 <step id="step2" next="step3">
   <tasklet>
  <chunk reader="cvsItemReader" writer="databaseItemWriter"  
                    processor="itemProcesser" commit-interval="1" />
   </tasklet>
 </step>
 <step id="step3" next="step4">
   <tasklet ref="fileDeletingTasklet" />
 </step>
 <step id="step4" next="step5">
   <tasklet>
  <chunk reader="databaseItemReader" writer="xmlItemWriter"  
                    processor="itemProcesser" commit-interval="1" />
   </tasklet>
 </step>
 <step id="step5">
  <tasklet ref="sendingEmailTasklet" />
 </step>
  </job>

The entire jobs and steps execution are stored in database, which make the failed step is able to restart at where it was failed, no need start over the entire job.

REST - Representational State Transfer

REST stands for Representational State Transfer. (It is sometimes spelled "ReST".) It relies on a stateless, client-server, cacheable communications protocol -- and in virtually all cases, the HTTP protocol is used.

REST is an architecture style for designing networked applications. The idea is that, rather than using complex mechanisms such as CORBA, RPC or SOAP to connect between machines, simple HTTP is used to make calls between machines.

• In many ways, the World Wide Web itself, based on HTTP, can be viewed as a REST-based architecture.

RESTful applications use HTTP requests to post data (create and/or update), read data (e.g., make queries), and delete data. Thus, REST uses HTTP for all four CRUD (Create/Read/Update/Delete) operations.
REST is a lightweight alternative to mechanisms like RPC (Remote Procedure Calls) and Web Services (SOAP, WSDL, et al.). Later, we will see how much more simple REST is.

• Despite being simple, REST is fully-featured; there's basically nothing you can do in Web Services that can't be done with a RESTful architecture.

REST is not a "standard". There will never be a W3C recommendataion for REST

REST is a lightweight alternative to Web Services and RPC. Much like Web Services, a REST service is:

• Platform-independent (you don't care if the server is Unix, the client is a Mac, or anything else),
• Language-independent (C# can talk to Java, etc.),
• Standards-based (runs on top of HTTP), and
• Can easily be used in the presence of firewalls.

Like Web Services, REST offers no built-in security features, encryption, session management, QoS guarantees, etc. But also as with Web Services, these can be added by building on top of HTTP:

• For security, username/password tokens are often used.
• For encryption, REST can be used on top of HTTPS (secure sockets).

One thing that is not part of a good REST design is cookies: The "ST" in "REST" stands for "State Transfer", and indeed, in a good REST design operations are self-contained, and each request carries with it (transfers) all the information (state) that the server needs in order to complete it.

Some soft guidelines for designing a REST architecture:

1. Do not use "physical" URLs. A physical URL points at something physical -- e.g., an XML file: "http://www.acme.com/inventory/product003.xml". A logical URL does not imply a physical file: "http://www.acme.com/inventory/product/003".
   • Sure, even with the .xml extension, the content could be dynamically generated. But it should be "humanly visible" that the URL is logical and not physical.
2. Queries should not return an overload of data. If needed, provide a paging mechanism. For example, a "product list" GET request should return the first n products (e.g., the first 10), with next/prev links.
3. Even though the REST response can be anything, make sure it's well documented, and do not change the output format lightly (since it will break existing clients).
   • Remember, even if the output is human-readable, your clients aren't human users.
   • If the output is in XML, make sure you document it with a schema or a DTD.
4. Rather than letting clients construct URLs for additional actions, include the actual URLs with REST responses. For example, a "product list" request could return an ID per product, and the specification says that you should use http://www.acme.com/product/PRODUCT_ID to get additional details. That's bad design. Rather, the response should include the actual URL with each item: http://www.acme.com/product/001263, etc.
   • Yes, this means that the output is larger. But it also means that you can easily direct clients to new URLs as needed, without requiring a change in client code.
5. GET access requests should never cause a state change. Anything that changes the server state should be a POST request (or other HTTP verbs, such as DELETE).

Key components of a REST architecture:

• Resources, which are identified by logical URLs. Both state and functionality are represented using resources.
  • The logical URLs imply that the resources are universally addressable by other parts of the system.
  • Resources are the key element of a true RESTful design, as opposed to "methods" or "services" used in RPC and SOAP Web Services, respectively. You do not issue a "getProductName" and then a "getProductPrice" RPC calls in REST; rather, you view the product data as a resource -- and this resource should contain all the required information (or links to it).
• A web of resources, meaning that a single resource should not be overwhelmingly large and contain too fine-grained details. Whenever relevant, a resource should contain links to additional information -- just as in web pages.
• The system has a client-server, but of course one component's server can be another component's client.
• There is no connection state; interaction is stateless (although the servers and resources can of course be stateful). Each new request should carry all the information required to complete it, and must not rely on previous interactions with the same client.
• Resources should be cachable whenever possible (with an expiration date/time). The protocol must allow the server to explicitly specify which resources may be cached, and for how long.
  • Since HTTP is universally used as the REST protocol, the HTTP cache-control headers are used for this purpose.
  • Clients must respect the server's cache specification for each resource.
• Proxy servers can be used as part of the architecture, to improve performance and scalability. Any standard HTTP proxy can be used.

Note that your application can use REST services (as a client) without being a REST architecture by itself; e.g., a single-machine, non-REST program can access 3rd-party REST services.