- Creating a base steering file using Marlin
- Creating a jobsub steering template from the base file
- Updating steering file content through Marlin
The steering files control which processors with which set of options are executed when passing the file to Marlin. The steering files are written in XML and can be easily edited with any text editor.
As a starting point, EUTelescope provides sets of steering files as example, where every steering file corresponds to a step in the analysis chain. These examples can be found in the jobsub/examples folder - please note the accompanying README files!
Whether you use an example as basis for your own analysis and you would like to add a missing processor, or whether you create the steering templates from scratch, ILCSoft offers tools that make these task significantly easier! The following steps show how you can create a new steering template from scratch for job submission with jobsub using Marlin. Alternatively, you could also use MarlinGUI, a graphical front end for the steering file setup. However, MarlinGUI is not part of the standard EUTelescope installation due to additional dependencies.
Creating a base steering file using Marlin
By running the command
Marlin -x > steering-base.xml
you can produce the steering file steering-base.xml which features the basic structure of a steering file and contains the options of every processor known to Marlin. While all optional settings are commented out, each processor's mandatory options are active and set to their default values as given in the processor source code.
Using the base file you can now prepare a steering template to be used with jobsub.
Creating a jobsub steering template from the base file
First, copy the XML structure including the <marlin>, <execute> and <global> sections (including closing tags in the end of the file!) into a new file used as template for your first analysis step e.g. step1-tmp.xml.
Now, copy the <processor> section of every processor you would like to execute in your analysis and adjust its options according to your needs. Any option that varies from run to run (e.g. the file name, beam energy, thresholds, ...) should be filled with a placeholder variable that can be substituted by jobsub. This placeholder can have any name (capitalization is ignored) but must be enclosed in '@' signs. Make sure that you use the placeholder @RunNumber@ for every place where you want the run number to appear as this is the only variable required by jobsub.
Make sure that all processors are also included in the <execute> section, or they will not be run by Marlin!
Congratulations, you have created your very own steering file! See the pages on job submission for information on how to run your analysis.
Updating comments and available options in steering files through Marlin (new in ILCsoft v01-17!)
Warning: if you have added or modified comments in the steering file to be updated by Marlin, these will be lost; your settings, parameters and template modifications will be left intact, however!
If you would like to use an older steering file for your analysis, which does not contain options introduced in later EUTelescope versions, you can update your steering file using Marlin. This will add all available options for each processor - options you have set to a specific value are kept as they are (but their description will be updated) while previously unset options will appear with their default value but commented out and now-removed options will be commented out completely.
Marlin -u old_steering_file.xml new_steering_file.xml
and the updated file will be stored in new_steering_file.xml (requires ILCsoft v01-17-01 or higher). Warnings issued by Marlin concerning e.g. missing LCIO files you can usually ignore - however, if your template breaks the XML formatting (e.g. like it was often the case with legacy pysub templates which use @Comment@ strings inside the XML structure) the resulting steering file might not be valid. Please verify it by using for example meld or diff and notify us of any issues you encounter!