reference_file_parser – Parse files storing reference values

The module provides the tools to parse and retrieve information from reference files with the following structure

# id channel amplifier some_number [some_number2 [..]]
001       L       L         1      [    1        [..]]
001       L       U         1      [    1        [..]]
001       R       L         1      [    1        [..]]
001       R       U         1      [    1        [..]]
...

It is also possible to use wild cards similarly to:

# id channel amplifier some_number [some_number2 [..]]
 *        *       *         1      [    1        [..]]
001       *       U         1      [    1        [..]]
001       R       *         1      [    1        [..]]
 *        R       U         1      [    1        [..]]

When using wild cards, the resolution order highlighted in _key_wildcards() is used.

New in version 0.3.0.

Each reference file is represented by a class derived from _BaseParser. The method _BaseParser.filename() returns the name of the file to parse and must be overridden in every derived class. Each class implementing a parser can be registered advertising it in the vhc.file_parsers entry point group

# in mylib/parser.py
class _AReferenceParser(_BaseParser):
    def filename(self):
        return "/path/to/reference/file.dat"

# in setup.py
entry_points = {'vhc.file_parsers':
                ['reference_parser = mylib.parser:_AReferenceParser']
                }

See _BaseParser for a list and description of all the available methods.

The registered classes are then initialised by init().

After initialization, the classes are accessible via the picker() function. Then one can call get_value method.

parser = picker("reference_parser")
parser.get_value(4, "L", "L")
libvhc.reference_file_parser.init(conf, log)[source]

Initialise the registered classes.

This function assumes that all the classes are initialized as _BaseParser

Parameters:
conf : pyhetdex.tools.configuration.ConfigParser instance

configuration object: should contain any info necessary to use the instances

log : logging.LoggerAdapter or logging.Logger instance

logger
libvhc.reference_file_parser.picker(name)[source]

Pick the instance registered under ‘name’. Should be called after init().

Parameters:
name : string

name under which the class or instance must be registered
Returns:
_BaseParser child instance
Raises:
KeyError

if the name is not registered
class libvhc.reference_file_parser._BaseParser(conf, log)[source]

Bases: object

Base class for the parsers.

All derived classes must overload filename()

Parameters:
conf : pyhetdex.tools.configuration.ConfigParser instance

configuration object: should contain any info necessary to use the instances

log : logging.LoggerAdapter or logging.Logger instance

logger
Attributes:
conf, log : as above
_key(self, id_, channel, amplifier)[source]

Return the dictionary key

Parameters:
id_, channel, amplifier : string

they should match any of the first columns in the file
Returns:
string

dictionary key
_key_wildcards(self, id_, channel, amplifier)[source]

Return the first key of the list below that exist:

id_ channel amplifier
id_ channel *
id_ * amplifier
id_ * *
* channel amplifier
* channel *
* * amplifier
* * *

If none is matched, returns the first one.

Parameters:
id_, channel, amplifier : string

they should match any of the first columns in the file
Returns:
string

dictionary key
get_value(self, id_, channel, amplifier)[source]

Returns the value(s) for the id_ fiber bundle or spectrograph (or whatever), for the given channel/amplifier pair.

Before returning, the values are parsed through parse_value()

This is the only method that should be used.

Parameters:
id_, channel, amplifier : string

they should match any of the first columns in the file
Returns:
whatever parse_value() returns
Raises:
VHCReferenceKeyError

if the combination of id, channel and amplifier does not exist in the reference file
filename(self)[source]

Abstract method: name of the reference file. Either hard-coded or from the configuration object.

Returns:
filename : string
_parse_file(self)[source]

Parse the input file and save it into a dictionary

The key of the dictionary is returned by _key() and the value are the columns from the 4th on.

parse_value(self, str_value)[source]

Parse the input string. The return value of get_value() is passed through this function.

This implementation just return the input value

Parameters:
str_value : string

string to be parsed
Returns:
string

parsed string
expand_file_name(self, fname)[source]

If fname is not absolute, expand the fname relative to the directory containing the reference file name returned by filename()

_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._Overscan(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file with the information for the overscan checks

filename(self)[source]

Name from the overscan_reference section of the [common] configuration section

parse_value(self, str_value)[source]

Parse the input string. The return value of get_value() is passed through this function.

This implementation just return the input value

Parameters:
str_value : string

string to be parsed
Returns:
mean, stddev, max_ovc_dev_mean, max_ovc_dev_rms : float

expected mean and standard deviation of the overscan
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._Bias(conf, log)[source]

Bases: libvhc.reference_file_parser._Overscan

Parse the file with the information for the bias comparison checks

filename(self)[source]

Name from the bias_reference section of the [bias] configuration section

_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._NFibers(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file with the information for the n_fibers checks

filename(self)[source]

Name from the nfibers_reference section of the [flat] configuration section

parse_value(self, str_value)[source]

Parse the remaining columns of the n_fibers reference file

Parameters:
str_value : string

string to be parsed
Returns:
n_fibers, tolerance : int

expected number of fibers and maximum error
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._MinFlux(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file with the information for the min_flux checks

filename(self)[source]

Name from the min_flux_reference section of the [flat] configuration section

parse_value(self, str_value)[source]

Parse the remaining column of the min_flux reference file

Parameters:
str_value : string

string to be parsed
Returns:
n_fibers : float

minimum allowed flux
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._Saturation(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file with the information for the saturation checks

filename(self)[source]

Name from the saturation_reference section of the [common] configuration section

parse_value(self, str_value)[source]

Parse the remaining column of the saturation reference file

Parameters:
str_value : string

string to be parsed
Returns:
n_fibers : float

maximum allowed flux
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._Distortion(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file containing the list of reference distortion files

filename(self)[source]

Name from the distortion_reference section of the [common] configuration section

parse_value(self, str_value)[source]

Parse the name of the reference distortion file

Parameters:
str_value : string

string to be parsed
Returns:
dist_file : string

name of the distortion file
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._Arcs(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file containing the list of reference line list files and maximum deviation of the arcs peaks from the expected position

filename(self)[source]

Name from the arcs_reference section of the [arc] configuration section

parse_value(self, str_value)[source]

Parse the name of the reference distortion file and the number of pixel tolerance

Parameters:
str_value : string

string to be parsed
Returns:
line_list : string

name of the line list file

peak_tolerance : int

maximum tolerance for arc peak shift
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._Dettemp(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file containing the maximum detector temperature and the maximum allowed difference between the commanded and the actual temperature

filename(self)[source]

Name from the dettemp_reference section of the [common] configuration section

parse_value(self, str_value)[source]

Parse the value and returns two floats

Parameters:
str_value : string

string to be parsed
Returns:
floats

temperature and delta temperature
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._Nullpixel(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file containing the maximum number of null pixels

filename(self)[source]

Name from the nullpixel_reference section of the [common] configuration section

parse_value(self, str_value)[source]

Parse the value and returns one float

Parameters:
str_value : string

string to be parsed
Returns:
floats

temperature and delta temperature
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._nHist(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file with the information for the row_cte checks

filename(self)[source]

Name from the row_cte_reference section of the [hetdex_dithers] configuration section

parse_value(self, str_value)[source]

Parse the remaining column of the row_cte reference file

Parameters:
str_value : string

string to be parsed
Returns:
n_hist : float

histogram selection parameter
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>
class libvhc.reference_file_parser._SkyLevel(conf, log)[source]

Bases: libvhc.reference_file_parser._BaseParser

Parse the file with the information for the sky level check

filename(self)[source]

Name from the sky_level_reference section of the [hetdex_dithers] configuration section

parse_value(self, str_value)[source]

Parse the remaining column of the sky_level reference file

Parameters:
str_value : string

string to be parsed
Returns:
sky_level : float

counts over the median overscan value
_abc_cache = <_weakrefset.WeakSet object>
_abc_negative_cache = <_weakrefset.WeakSet object>
_abc_negative_cache_version = 50
_abc_registry = <_weakrefset.WeakSet object>