Download 2 files Spring Batch Step - file

Question

I am building a Spring Batch Job that has 2 steps for downloading 2 files. The file names are job parameters.
Please find below the steps configuration class (FileDownloader and FileDownloadTasklet are custom classes for the download logic):
#Autowired
private FileDownloader fileDownloader;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Bean
public Step downloadFirstFileStep() {
return stepBuilderFactory.get("downloadFirstFileStep")
.tasklet(firstFileDownloadTasklet(null)).build();
}
#Bean
#StepScope
public FileDownloadTasklet firstFileDownloadTasklet(
#Value("#{jobParameters['firstFile']}") String fileName) {
return new FileDownloadTasklet(fileDownloader, fileName);
}
#Bean
public Step downloadSecondFileStep() {
return stepBuilderFactory.get("downloadSecondFileStep")
.tasklet(secondFileDownloadTasklet(null)).build();
}
#Bean
#StepScope
public FileDownloadTasklet secondFileDownloadTasklet(
#Value("#{jobParameters['secondFile']}") String fileName) {
return new FileDownloadTasklet(fileDownloader, fileName);
}
I'm feeling I'm not doing the right thing by duplicating the Step beans since the only difference is the actual name of the file to be downloaded.
Can you please give me a hint on how can I do it without duplicating the beans?

Answer 1

Yes we can do this in better way using a partitioner. Currently you are downloading 2 files and future you might want to download more number of files.
#Bean(name = "partitionerJob")
public Job partitionerJob() {
return jobs.get("partitioningJob")
.start(partitionStep())
.build();
}
#Bean
#StepScope
public Step partitionStep() {
return steps.get("partitionStep")
.partitioner("slaveStep", partitioner())
.step(downloadFirstFileStep())
.taskExecutor(taskExecutor())
.build();
}
#Bean
public Step downloadFirstFileStep() {
return stepBuilderFactory.get("downloadFirstFileStep")
.tasklet(firstFileDownloadTasklet(null)).build();
}
#Bean
#StepScope
public FileDownloadTasklet firstFileDownloadTasklet(
#Value("#{stepExecutionContext['fileName']}") String fileName) {
return new FileDownloadTasklet(fileDownloader, fileName);
}
#Bean
public YourPartitioner partitioner() {
return new YourPartitioner();
}
public class YourPartitioner implements Partitioner {
#Value("#{jobParameters['fileNames']}") //Pass Comma separated file names as argument
protected String fileNames;
#Override
public Map<String, ExecutionContext> partition(int gridSize) {
Map<String, ExecutionContext> map = new HashMap<>(gridSize);
int i = 0, k = 1;
for (String resource : fileNames.split(",") {
ExecutionContext context = new ExecutionContext();
context.putString("fileName", resource); //This will be fetched as argument to the step job from JobExecutionContext
map.put("PARTITION_KEY " + i, context);
i++;
}
return map;
}
}