Chunk-oriented step callback functions and their lifecycle
Key Takeaway
Optional hooks such as beforeStep and afterStep run only if their attributes (before-step-function, after-step-function) are defined in the step's steptypes.json entry. Unlike read, process and write, the hooks don't fall back to a function with the default name.
A chunk-oriented job step is a custom B2C Commerce job step, written as a CommonJS script module, that works through a list of items in fixed-size batches called chunks. The job framework calls a set of named callback functions in a fixed order. Setup functions run once, read and process run once per item, and write runs once per chunk. Cleanup functions run at the end. The design suits countable data. The design also gives fine-grained progress monitoring, because the count of written items updates after every chunk.
When to use a chunk-oriented step
Choose between a chunk-oriented module and a task-oriented module based on the work the step does.
| Chunk-oriented module | Task-oriented module | |
|---|---|---|
| Fits | Countable data whose boundaries are fixed or quick to calculate, such as exporting a list of orders | A single task whose progress can't be calculated or whose boundaries are hard to calculate |
| Functions | A set of named callback functions that the framework calls in a fixed order | One main function |
| Exit status | Can't be defined, so the step always finishes with either OK or ERROR. | The function can return a dw.system.Status object with a custom code such as FILE_NOT_FOUND. |
A chunk-oriented step must be a CommonJS script module. You can't implement one with a pipeline module.
The work isn't limited to database transactions. A chunk-oriented script can include any series of processing steps, such as reading catalog products and writing them to a file in the IMPEX directory.