Extend VHC

VHC has been designed with extendability in mind: the goal is to allow people to use the available infrastructure but with custom recipes, drivers and reference file parsers. To do this we use entry-points.

Also the builtin recipes, drivers and parsers are distribute through the same infrastructure.

In the following we will describe the entry point groups used by VHC, naming conventions and the interface for each of the groups. We will also provide some example on how to create new entry points and the associated functions.

For the sake of this documentation we assume that the custom code is implemented into a package with the following structure:

vhc_extra
├── setup.py                 # make the package installable
├── vhc_extra                # the actual code
│   ├── __init__.py          # this is a package
│   ├── drivers.py           # module implementing the driver
│   ├── ref_file_parsers.py  # module implementing new file parsers
│   └── recipe.py            # vhc modifications
├── ...
...

The setup.py can look like:

from setuptools import setup, find_packages
# put here anything else you need
# [...]

entry_points = {
# this will be filled as we go
}

setup(
    name="vch_extra",
    version='0.0.0',
    author="My Name",
    author_email='my@email.me',
    description="VHC extension package",
    long_description="This is a long descriptions",
    packages=find_packages(),
    install_requires=[''],  # put here any extra package vhc_extra depends on
    entry_points=entry_points,
    # any other relevant entry
)

Plug new recipes

A recipe tells vhc what kind of files we expect to work on. To a given recipe, we associate a file name template, used to collect files for vhc and to create the recipe file if not available, and a list of default drivers, were a v_driver.txt file not found.

Recipes are advertised under the vhc.recipes group and each should appear as:

name = module:function

where function has the following signature:

libvhc.recipes.recipe_protopype(name)

Signature for a function used to register a recipe.

Parameters:

name : string

name under which the recipe is advertised

Returns:

string

file name template to use to collect all the relevant file or to guess the recipe name from the files; e.g. for the flat recipe [0-9]*flt.fits is returned

list of strings

list of default drivers to run for the recipe, if no v_driver.txt file is found; e.g. ['common:n_files', 'common:saturation']

Let’s say that you want to add a new recipe type virus_ancillary, you need to do the following:

1. implement a function according to the prototype (in vhc_extra/vhc_extra/recipe.py)

   def virus_ancillary_recipe(name):
       """File names and drivers available for the virus_ancillary recipe"""
       recipe_match = "*[0-9]*sci.fits"
       default_drivers = common_drivers + ["vhc_extra:do_something",
                                           "do_something_else"]
       return recipe_match, default_drivers
   

Once you have a recipe, you can register it adding the following entry in the entry_points dictionary:

entry_points = {
    'vhc.recipes':
    ['virus_ancillary = vhc_extra.recipe:virus_ancillary_recipe']
}

Now, after installing vhc_extra your recipe will be available to VHC.

Note

If you want to use the machinery provided by the Factory functions to implement new drivers as described in Create and use a drivers you must add a section in the configuration file with the name of your new recipe and the number of expected exposure/dithers:

[virus_ancillary]
# expected number of exposures
n_exposures = 1

Plug new drivers

In Create and use a drivers we have seen some way to write new drivers. Here we illustrate how to advertise new drivers, so that vhc can pick them up.

Drivers are looked for under the vhc.drivers entry point group and, similarly to what seen for the recipes, each entry must be have name of the driver and the position of the callable implementing it. So for the recipe described in Plug new recipes, we need to create two functions to implement the them: lets say that we write two functions vhc_extra/drivers.py:

def do_something(vcheck, path, argv):
    "implement a driver that does something"
    pass

def an_other_driver(vcheck, path, argv):
    """This driver does something else and it also shows that the driver
    and its function names don't have to be the same: how cunning...
    """

After this is done, we tell python where to find them adding to setup.py:

entry_points = {
    'vhc.drivers':
    ['vhc_extra:do_something = vhc_extra.drivers:do_something',
     'do_something_else = vhc_extra.drivers:an_other_driver'
    ]
}

Now whoever installs the vhc_extra package can use the virus_ancillary recipe and the two new drivers just described.

Plug new reference file parsers

Some of the drivers require some reference values or files to run. We store those values or file names in tables in the vhc_config repository and we use a common framework to parse and get values from them. The framework is described and implemented in libvhc.reference_file_parser.

Every file parser is derived from a common ancestor, _BaseParser, already implementing most of the methods necessary. The only method that needs to be implemented to have a working parser is filename().

Now, let’s say that your do_something_else driver need some reference value from a some_reference file somewhere. To avoid hard coding it in the python implementation you can add it in the configuration file:

[virus_ancillary]
# any other configuration
some_reference = /path/to/some_reference.dat

Then in vhc_extra/ref_file_parsers.py we can create the parser:

import libvhc.reference_file_parser as vhcrfp

class MyParser(vhcrfp._BaseParser):
    def filename(self):
        """returns the file name from the configuration"""
        return self.conf.get("vhc_extra", "some_reference")

    def parse_value(self, str_value):
        """parse the extra columns at need"""
        return str_value.split()

Now we need to advertise the parser using the vhc.file_parsers entry point:

entry_points = {
    'vhc.file_parsers':
    ['some_reference = vhc_extra.ref_file_parsers:MyParser']
    }

Once the parser is advertised, it’s possible to get it in your driver with the picker() function:

import libvhc.reference_file_parser as vhcrfp
[...]
parser = vhcrfp.picker('some_reference')
values = parser.get_value(001, 'L', 'U')

---

Bonus

If you want to do more you can, probably.

The vhc executable simply calls libvhc.vhc.main(). If you want/need to run some code before and/or after vhc, you can wrap the above function and create a console script. E.g. in a entry_point.py module you can define:

import libvhc.vhc as vhc

def main(argv=None):

    # do the setup that you need
    [...]
    # call vhc
    vhc.main(argv=argv)
    # do any cleanup if needed
    [...]

Then add the entry point in your setup.py file to create a new vhc_extra executable when installing your package

entry_points={'console_scripts':
              ['vhc_extra = vhc_extra.entry_point:main']}