Extending Sumatra with plug-ins¶
To support the wide diversity of scientific workflows, Sumatra aims to make it easy (or at least possible) to customize the way it works. The following components can be customized:
- launching computations (local execution, distributed using MPI, batch jobs using SLURM, ...)
- parameter file formats (JSON, YAML, ...)
- executable programs/programming languages (Python, MATLAB, R, ...)
- data stores (local disk, network storage, WebDAV, Dropbox, ...)
- record stores (SQLite, PostgreSQL, web-based, ...)
- output formats (plain text, HTML, LaTeX, JSON, ...)
- version control systems (Git, Mercurial, Subversion, ...)
Here is an example of wrapping the PHP command-line interface. (In itself, this gives little or no benefit, as Sumatra can automatically obtain version information from most command-line tools, but (a) it is a prerequisite if we wish to add dependency tracking, (b) it makes a nicely simple example of writing a plug-in!)
from sumatra.core import component
from sumatra.programs import Executable
@component
class PHPcli(Executable):
name = "PHP"
executable_names = ('php',)
file_extensions = ('.php',)
default_executable_name = 'php'
requires_script = True
To use this plug-in in a Sumatra project, it should be saved in a file (e.g. php.py
)
that is on the Python module search path (for example, installed in site-packages
or
added to the PYTHONPATH
environment variable), and then added to the project using:
$ smt configure --add-plugin=php
You can also manually edit .smt/project
and add the module path to the “plugins” list.
To list the installed plug-ins, use smt info
.
To remove a plug-in from a project:
$ smt configure --remove-plugin=php
Component interfaces¶
As shown in the PHP example above, writing a custom component involves writing a new class which inherits from a Sumatra base class, and then registering the class with the component registry.
The available base classes are:
sumatra.datastore.base.DataStore
sumatra.datastore.base.DataItem
sumatra.formatting.Formatter
sumatra.recordstore.base.RecordStore
sumatra.versioncontrol.base.Repository
sumatra.versioncontrol.base.WorkingCopy
sumatra.launch.LaunchMode
sumatra.parameters.ParameterSet
sumatra.programs.Executable
For full details of which methods need to be implemented in your sub-class, see the API reference.
Note
At present, there is no mechanism for plug-ins to change the list of options
for command-line tools such as smt
. This is planned for the future.