spring batch 是一個開源的批處理框架.執(zhí)行一系列的任務(wù). 在 spring batch 中 一個job 是由許多 step 組成的。而每一個 step 又是由 read-process-write task或者 單個 task 組成。
1. "read-process-write" 處理，根據(jù)字面意思理解就可以:
- read 就是從資源文件里面讀取數(shù)據(jù)，比如從xml文件,csv文件,數(shù)據(jù)庫中讀取數(shù)據(jù).
- process 就是處理讀取的數(shù)據(jù)
- write 就是將處理過的數(shù)據(jù)寫入到其他資源文件中去，可以是xml,csv,或者數(shù)據(jù)庫.
比如：從csv文件中 讀取數(shù)據(jù)，經(jīng)過處理之后，保存到數(shù)據(jù)庫. spring batch 提供了很多類去處理這方面的東西。
2.單個task, 也就是處理單個任務(wù)。比如在一個step 開始之前或者完成之后清除資源文件等.
3.許多個step 組成在一起，就組成了一個job. 所以他們之間的關(guān)系，就如同下面的描述:
一個 job = 很多steps
一個step = 一個read-process-write 或者 一個task.
同樣一個job = step1 -->step2--step3 這樣鏈表形式的組成.
spring batch 例子
考慮如下一個批處理的例子,看起來有點(diǎn)啰嗦，只是為了說明用途:
1. step1 : 從 a 文件夾中讀取csv 文件，處理之后，寫入到b文件夾中(read-process-write)
2. step2 : 從 b 文件夾中讀取csv文件 ，處理之后， 存儲到數(shù)據(jù)庫中(read-process-write).
3. step3 : 刪除b文件夾下的csv文件。(用到單個task)
4. step4 : 從數(shù)據(jù)庫讀取數(shù)據(jù)，處理之后，生成xml報(bào)表文件(read-process-write).
5. step5 : 讀取xml報(bào)表,并發(fā)送email給管理員(用到單個task)
用spring batch 我們可以如下定義這個job:
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
<job id="abcjob" xmlns="http://www.springframework.org/schema/batch"> <step id="step1" next="step2"> <tasklet> <chunk reader="cvsitemreader" writer="cvsitemwriter" processor="itemprocesser" commit-interval="1" /> </tasklet> </step> <step id="step2" next="step3"> <tasklet> <chunk reader="cvsitemreader" writer="databaseitemwriter" processor="itemprocesser" commit-interval="1" /> </tasklet> </step> <step id="step3" next="step4"> <tasklet ref="filedeletingtasklet" /> </step> <step id="step4" next="step5"> <tasklet> <chunk reader="databaseitemreader" writer="xmlitemwriter" processor="itemprocesser" commit-interval="1" /> </tasklet> </step> <step id="step5"> <tasklet ref="sendingemailtasklet" /> </step> </job> |
整個 job 的執(zhí)行是存儲在數(shù)據(jù)庫中的，所以即使是某一個step出錯失敗，也不需要全部從頭開始執(zhí)行這個job.下面是一個真正的入門教程例子.
采用 jar包如下:
spring-batch-2.2.3 以上版本，但是我在2.2.3版本中發(fā)現(xiàn) org/springframework/batch/core/schema-mysql.sql 里面的的mysql 創(chuàng)建表的語句是有問題的，也就是少了“," 號導(dǎo)致的問題( not null, 后面幾個創(chuàng)建表的語句not null 后面少了逗號)，當(dāng)然你可以自己修改后再執(zhí)行，執(zhí)行完畢后有如下幾個表:
xstream-1.3.jar 必須的。
jettison-1.3.3.jar也是必須的, 否則會出現(xiàn)
java.lang.noclassdeffounderror: org/codehaus/jettison/mapped/mappedxmloutputfactory錯誤。
另外我用的spring 是 3.1 版本的，可以下載相關(guān)jar包，還有apache common 相關(guān)jar包就可以了。
mysql-connect-java-5.1.jar 連接mysql 數(shù)據(jù)庫用的。
假設(shè)要將如下 csv 文件讀取出來處理之后，寫入到一個xml文件之中.
|
1
2
3
|
,"213,100",980,"mkyong", 29/7/2013,"320,200",1080,"staff 1", 30/7/2013,"342,197",1200,"staff 2", 31/7/2013 |
用 flatfileitemreader 去讀取csv 文件, 用 itemprocessor 去處理數(shù)據(jù)，用 staxeventitemwriter 去寫數(shù)據(jù)
job 的定義如下(job-hello-world.xml):
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
|
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:batch="http://www.springframework.org/schema/batch" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://www.springframework.org/schema/batch http://www.springframework.org/schema/batch/spring-batch-2.2.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd "> <import resource="../config/context.xml" /> <import resource="../config/database.xml" /> <bean id="report" class="yihaomen.model.report" scope="prototype" /> <bean id="itemprocessor" class="yihaomen.customitemprocessor" /> <batch:job id="helloworldjob"> <batch:step id="step1"> <batch:tasklet> <batch:chunk reader="cvsfileitemreader" writer="xmlitemwriter" processor="itemprocessor" commit-interval="10"> </batch:chunk> </batch:tasklet> </batch:step> </batch:job> <bean id="cvsfileitemreader" class="org.springframework.batch.item.file.flatfileitemreader"> <property name="resource" value="classpath:cvs/input/report.csv" /> <property name="linemapper"> <bean class="org.springframework.batch.item.file.mapping.defaultlinemapper"> <property name="linetokenizer"> <bean class="org.springframework.batch.item.file.transform.delimitedlinetokenizer"> <property name="names" value="id,sales,qty,staffname,date" /> </bean> </property> <property name="fieldsetmapper"> <bean class="yihaomen.reportfieldsetmapper" /> <!-- if no data type conversion, use beanwrapperfieldsetmapper to map by name <bean class="org.springframework.batch.item.file.mapping.beanwrapperfieldsetmapper"> <property name="prototypebeanname" value="report" /> </bean> --> </property> </bean> </property> </bean> <bean id="xmlitemwriter" class="org.springframework.batch.item.xml.staxeventitemwriter"> <property name="resource" value="file:xml/outputs/report.xml" /> <property name="marshaller" ref="reportmarshaller" /> <property name="roottagname" value="report" /> </bean> <bean id="reportmarshaller" class="org.springframework.oxm.jaxb.jaxb2marshaller"> <property name="classestobebound"> <list> <value>yihaomen.model.report</value> </list> </property> </bean></beans> |
映射csv文件到 report 對象并寫xml文件 (通過 jaxb annotations).
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
|
package yihaomen.model;import java.math.bigdecimal;import java.util.date;import javax.xml.bind.annotation.xmlattribute;import javax.xml.bind.annotation.xmlelement;import javax.xml.bind.annotation.xmlrootelement;@xmlrootelement(name = "record")public class report { private int id; private bigdecimal sales; private int qty; private string staffname; private date date; @xmlattribute(name = "id") public int getid() { return id; } public void setid(int id) { this.id = id; } @xmlelement(name = "sales") public bigdecimal getsales() { return sales; } public void setsales(bigdecimal sales) { this.sales = sales; } @xmlelement(name = "qty") public int getqty() { return qty; } public void setqty(int qty) { this.qty = qty; } @xmlelement(name = "staffname") public string getstaffname() { return staffname; } public void setstaffname(string staffname) { this.staffname = staffname; } public date getdate() { return date; } public void setdate(date date) { this.date = date; } @override public string tostring() { return "report [id=" + id + ", sales=" + sales + ", qty=" + qty + ", staffname=" + staffname + "]"; }} |
為了轉(zhuǎn)換日期,用了自定義的 fieldsetmapper. 如果沒有數(shù)據(jù)需要轉(zhuǎn)換, beanwrapperfieldsetmapper 通過名稱name 去自動映射值。
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
package yihaomen;import java.text.parseexception;import java.text.simpledateformat;import org.springframework.batch.item.file.mapping.fieldsetmapper;import org.springframework.batch.item.file.transform.fieldset;import org.springframework.validation.bindexception;import yihaomen.model.report;public class reportfieldsetmapper implements fieldsetmapper<report> { private simpledateformat dateformat = new simpledateformat("yyyy-mm-dd"); @override public report mapfieldset(fieldset fieldset) throws bindexception { report report = new report(); report.setid(fieldset.readint(0)); report.setsales(fieldset.readbigdecimal(1)); report.setqty(fieldset.readint(2)); report.setstaffname(fieldset.readstring(3)); //default format yyyy-mm-dd //fieldset.readdate(4); string date = fieldset.readstring(4); try { report.setdate(dateformat.parse(date)); } catch (parseexception e) { e.printstacktrace(); } return report; }} |
在寫入數(shù)據(jù)之前調(diào)用itemprocessor 處理數(shù)據(jù)
|
1
2
3
4
5
6
7
8
9
10
11
|
package yihaomen;import org.springframework.batch.item.itemprocessor;import yihaomen.model.report;public class customitemprocessor implements itemprocessor<report, report> { @override public report process(report item) throws exception { system.out.println("processing..." + item); return item; }} |
spring 配置文件和數(shù)據(jù)庫配置文件
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
|
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd"> <!-- stored job-meta in memory --> <!-- <bean id="jobrepository" class="org.springframework.batch.core.repository.support.mapjobrepositoryfactorybean"> <property name="transactionmanager" ref="transactionmanager" /> </bean> --> <!-- stored job-meta in database --> <bean id="jobrepository" class="org.springframework.batch.core.repository.support.jobrepositoryfactorybean"> <property name="datasource" ref="datasource" /> <property name="transactionmanager" ref="transactionmanager" /> <property name="databasetype" value="mysql" /> </bean> <bean id="transactionmanager" class="org.springframework.batch.support.transaction.resourcelesstransactionmanager" /> <bean id="joblauncher" class="org.springframework.batch.core.launch.support.simplejoblauncher"> <property name="jobrepository" ref="jobrepository" /> </bean> </beans> |
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:jdbc="http://www.springframework.org/schema/jdbc" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.2.xsd http://www.springframework.org/schema/jdbc http://www.springframework.org/schema/jdbc/spring-jdbc-3.2.xsd"> <!-- connect to mysql database --> <bean id="datasource" class="org.springframework.jdbc.datasource.drivermanagerdatasource"> <property name="driverclassname" value="com.mysql.jdbc.driver" /> <property name="url" value="jdbc:mysql://localhost:3306/test" /> <property name="username" value="root" /> <property name="password" value="" /> </bean> <bean id="transactionmanager" class="org.springframework.batch.support.transaction.resourcelesstransactionmanager" /> <!-- create job-meta tables automatically --> <jdbc:initialize-database data-source="datasource"> <jdbc:script location="org/springframework/batch/core/schema-drop-mysql.sql" /> <jdbc:script location="org/springframework/batch/core/schema-mysql.sql" /> </jdbc:initialize-database> </beans> |
運(yùn)行程序
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
package yihaomen;import org.springframework.batch.core.job;import org.springframework.batch.core.jobexecution;import org.springframework.batch.core.jobparameters;import org.springframework.batch.core.launch.joblauncher;import org.springframework.context.applicationcontext;import org.springframework.context.support.classpathxmlapplicationcontext;public class app { public static void main(string[] args) { string[] springconfig = { "spring/batch/jobs/job-hello-world.xml" }; applicationcontext context = new classpathxmlapplicationcontext(springconfig); joblauncher joblauncher = (joblauncher) context.getbean("joblauncher"); job job = (job) context.getbean("helloworldjob"); try { jobexecution execution = joblauncher.run(job, new jobparameters()); system.out.println("exit status : " + execution.getstatus()); } catch (exception e) { e.printstacktrace(); } system.out.println("done"); }} |
運(yùn)行結(jié)果 ：
|
1
2
3
4
5
6
7
8
9
10
11
|
十二月 03, 2013 8:56:24 下午 org.springframework.batch.core.launch.support.simplejoblauncher$1 runinfo: job: [flowjob: [name=helloworldjob]] launched with the following parameters: [{}]十二月 03, 2013 8:56:24 下午 org.springframework.batch.core.job.simplestephandler handlestepinfo: executing step: [step1]processing...report [id=1001, sales=213100, qty=980, staffname=yihaomen]processing...report [id=1002, sales=320200, qty=1080, staffname=staff 1]processing...report [id=1003, sales=342197, qty=1200, staffname=staff 2]十二月 03, 2013 8:56:25 下午 org.springframework.batch.core.launch.support.simplejoblauncher$1 runinfo: job: [flowjob: [name=helloworldjob]] completed with the following parameters: [{}] and the following status: [completed]exit status : completeddone |
結(jié)果生成了output.xml 在你工程目錄的 xml 目錄下。
整個源代碼，除去jar包之后下載:spring batch 入門教程下載
以上就是本文的全部內(nèi)容，希望對大家的學(xué)習(xí)有所幫助，也希望大家多多支持服務(wù)器之家。
原文鏈接：http://blog.csdn.net/achuo/article/details/50982298
