VHC Drivers ########### .. contents:: :local: :depth: 2 Introduction ============ By default every check prints a log message to STDOUT. E.g. for a passed test:: Running exposure time validation test on 20190425T215026.6_032RU_zro.fits and for a failed test:: 20190425T215026.6_032RU_zro.fits failed exptime check, EXPTIME == 0 for a non-bias observation! If the tests are run with the ``-j`` flag, additionally a dicitionary in JSON format is printed to STDOUT as well. To make parsing easier, every line of the JSON output is prefixed with ``+++VHCJSON+++`` e.g. for a failed test:: +++VHCJSON+++ {"filename": "20190425T215026.6_032RU_zro.fits", "testname": "exposure time validation", "checks":[{"subtestname":"nonzro_exptime", "expected":"0.001", "tolerance":0, "actual":"5.5467e-05", "result":"FAILED"}], "result":"FAILED", "error": "failed exptime check, EXPTIME == 0 for a non-bias observation!"} The keys of the dictionary are: filename The name of the checked file testname The name of the test run checks A list of all run subtests subtestname The name of the run subtest epxected Expected result of the subtest tolerance Allowed tolerance for the subtest actual The actually measured value result The result of the subtest, one of ``PASSED``, ``FAILED`` or ``NOTRUN`` error An error message explaining the reason why the test failed. General checks ============== .. contents:: :local: .. _dettemp_txt: Reference file: ``dettemp.txt`` ------------------------------- The default values for the common detector temperature check are stored in the ``dettemp.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of specid, channel and amplifier, it specifies these columns: +-----------------+---------------------------------------------------+ | max_dettemp | Maxiumum detector temperature | +-----------------+---------------------------------------------------+ | delta_setppoint | Maximum difference between the commanded | | | temperature and the measured detector temperature | +-----------------+---------------------------------------------------+ .. _nullpixel_txt: Reference file: ``nullpixel.txt`` --------------------------------- The default values for the common nullpixel check are stored in the ``nullpixel.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of specid, channel and amplifier, it specifies these columns: +-----------------+---------------------------------------------------+ | n_null_pix | Maximum number of null pixels | +-----------------+---------------------------------------------------+ .. _saturation_txt: Reference file: ``saturation.txt`` ---------------------------------- The default values for the common saturation check are stored in the ``saturation.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of specid, channel and amplifier, it specifies these columns: +-----------------+---------------------------------------------------+ | saturation | Saturation level | +-----------------+---------------------------------------------------+ | max_sat_pixels | Maximum allowed number of pixels above the | | | saturation level | +-----------------+---------------------------------------------------+ .. _overscan_txt: Reference file: ``overscan.txt`` -------------------------------- The default values for the common overcan check are stored in the ``overscan.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of specid, channel and amplifier, it specifies these columns: +----------------+----------------------------------------------------+ | mean | Expected mean bias value ignored if specified in | | | the fits header keyword BIASTRGT) | +----------------+----------------------------------------------------+ | stddev | Expected standard deviation | +----------------+----------------------------------------------------+ | max_dev_mean | Maximum allowed deviation for the overscan mean | +----------------+----------------------------------------------------+ | max_dev_stddev | Maximum allowed deviation for the overscan stddev | +----------------+----------------------------------------------------+ | line_stddev | Maximum allowed sttdev for line overscan check | +----------------+----------------------------------------------------+ .. _n_files: ``common:n_files`` - Correct number of files -------------------------------------------- **Header Key:** ``VHCNFLILE`` Check the number of exposures and, for each exposure, the number of files. Subtest: ``n_exp`` - Number of exposures """""""""""""""""""""""""""""""""""""""" Checks that the number of exposure subdirectories found in the shot directory matches the number expected for the given recipe. The expected number is read from the ``n_exposures`` field under the recipe section in the config file. The format of the exposure directory names is defined by the ``exposure_dir`` field unter the common section in the config file. The test fails if not all exposure directories are found. **Expected value:** Expected number of exposure directories **Tolerance:** None **Actual value:** Found number of exposure directories Subtest: ``n_files_exp??`` - Number of files per exposure """"""""""""""""""""""""""""""""""""""""""""""""""""""""" Checks that the number of files in one exposure subdirectory. This subtest is repeated for each exposure directory. The number of files is calculated from the number of active IFUs. The filenames have to match the filename regexp defined by the recipe. The test fails if not all expected files are found. **Expected value:** Expected number of files in the exposure directory **Tolerance:** None **Actual value:** Found number of files in the exposure directory .. _n_pixels: ``common:n_pixels`` - Correct image size ---------------------------------------- **Header Key:** ``VHCNPIX`` Check the image size to be correct. Subtest: ``size_naxis1`` - Size along NAXIS1 """""""""""""""""""""""""""""""""""""""""""" Checks that the number of pixels along NAXIS1 is 1064 pixels (2064 / 2 + 32 pixels overscan). The test fails if the number of pixels in the image does not match the expected number. **Expected value:** 1064 pixels **Tolerance:** None **Actual value:** Number of pixels along NAXIS1 Subtest: ``size_naxis2`` - Size along NAXIS2 """""""""""""""""""""""""""""""""""""""""""" Checks that the number of pixels along NAXIS2 is 1032 pixels (2064 / 2). The test fails if the number of pixels in the image does not match the expected number. **Expected value:** 1032 pixels **Tolerance:** None **Actual value:** Number of pixels along NAXIS2 .. _exptime: ``common:exptime`` - Correct exposure time ------------------------------------------ **Header Key:** ``VHCEXPTM`` Check that the exposure time matches the recipe (zero for bias frames, non-zero otherwise). Subtest: ``zro_exptime`` - Bias frame exposure time """"""""""""""""""""""""""""""""""""""""""""""""""" This test is only run with the bias frame recipe. Checks that the exposure time of the image is zero seconds. The test fails if the exposure time is larger than 0.001 seconds. **Expected value:** Exposure time smaller than 0.001 seconds. **Tolerance:** None **Actual value:** Recorded exposure time Subtest: ``nonzro_exptime`` - Non Bias frame exposure time """""""""""""""""""""""""""""""""""""""""""""""""""""""""" This test is run for all non bias recipes. Checks that the exposure time of the image is not zero seconds. The test fails if the exposure time is smaller than 0.001 seconds. **Expected value:** Exposure time larger than 0.001 seconds **Tolerance:** None **Actual value:** Recorded exposure time .. _overscan: ``common:check_overscan``- Check overscan value ------------------------------------------------ **Header Key:** ``VHCOVSCN`` Check the mean and standard deviation of the overscan against expected values. Subtest: ``overscan_mean`` - Mean overscan value """""""""""""""""""""""""""""""""""""""""""""""" Compares the five sigma clipped mean of the of the overscan region (the size of the region is taken from the BIASSEC header keyword) against the expected mean value taken from the mean parameter of the :ref:`overscan.txt ` reference file, or if present from the BIASTRGT header keyword. The test fails if the difference between the measured overscan mean and the expected value is larger then max_dev_mean times the expected value. **Expected value:** Expected mean overscan value **Tolerance:** Maximum fractional deviation from the expected mean **Actual value:** Measured mean overscan value Subtest: ``overscan_stddev`` - Overscan standard deviation """""""""""""""""""""""""""""""""""""""""""""""""""""""""" Compares the five sigma clipped standard deviaton of the overscan region against the expected standard deviation value taken from the stdddev parameter of the :ref:`overscan.txt ` reference file. The test fails if the difference between the measured overscan standard deviation and the expected value is larger then max_dev_stddev times the expected value. **Expected value:** Expected standard deviation of the overscan **Tolerance:** Maximum fractional deviation from the expected stddev **Actual value:** Measured standard deviation of the overscan .. _line_overscan: ``common:check_line_overscan`` - Overscan line features ------------------------------------------------------- **Header Key:** VHCOLSTD Check the overscan for line structure in the files. Calculates the five sigma clipped mean of of each line of the overscan region, and checks the standard deviation of the resulting 1-D arry. The test fails if the the measured standard deviationand exceeds the value specified in the corresponding line_stddev column in the :ref:`overscan.txt ` reference file. **Expected value:** Maximum allowed standard deviation **Tolerance:** None **Actual value:** Measured standard deviation of 1-D line array .. _dettemp: ``common:dettemp`` - Detector temperature checks ------------------------------------------------- **Header Key:** VHCTEMP Check the detector temperature. Subtest: ``abs_dettemp`` - Detector temperature """"""""""""""""""""""""""""""""""""""""""""""" Check that the detector does not exceed the expected temperature. The maximum allowed temperature is read from the max_dettemp column in the :ref:`dettemp.txt ` reference file. The detector temperature is read from the ``DETTEMP`` header keyword. The test fails if the detector temperature exceeds the maximum allowed value. **Expected value:** Maximum allowed detector temperature **Tolerance:** None **Actual value:** Detector temperature read from ``DETTEMP`` Subtest: ``set_dettemp`` - Detector temperature """"""""""""""""""""""""""""""""""""""""""""""" Check the difference between the detector temperature read from the ``DETTEMP`` header keyword and the temperature setpoint read from the ``SETPOINT`` header. The maximum allowed difference is read from the delta_setpoint column in the :ref:`dettemp.txt ` reference file. The test fails if the difference exceeds the maximum allowed difference. **Expected value:** Setpoint read from the ``SETPOINT`` header keyword **Tolerance:** The maximum allowed difference **Actual value:** Read detector temperature .. _headerkeys: ``common:headerkeys`` - Header keyword checks --------------------------------------------- **Header Key:** VHCHEAD Checks presence of the following header keywords: +--------------------------------------------------------------------------+ | Integer keywords | +--------------+--------------+--------------+--------------+--------------+ | ``NAXIS1`` | ``NAXIS2`` | ``SPECID`` | ``IFUSLOT`` | ``DETECTOR`` | +--------------+--------------+--------------+--------------+--------------+ | ``OBSID`` | | | | | +--------------+--------------+--------------+--------------+--------------+ | Float keywords | +--------------+--------------+--------------+--------------+--------------+ | ``DETTEMP`` | ``SETPOINT`` | ``MJD`` | ``GAIN`` | ``RDNOISE`` | +--------------+--------------+--------------+--------------+--------------+ | ``AIRMASS`` | ``EXPTIME`` | ``DARKTIME`` | ``STRUCTAZ`` | ``READTIME`` | +--------------+--------------+--------------+--------------+--------------+ | String keywords | +--------------+--------------+--------------+--------------+--------------+ | ``BIASSEC`` | ``TRIMSEC`` | ``TELESCOP`` | ``INSTRUME`` | ``CONTID`` | +--------------+--------------+--------------+--------------+--------------+ | ``IFUID`` | ``CCDPOS`` | ``CCDHALF`` | ``DATE-OBS`` | ``DATE`` | +--------------+--------------+--------------+--------------+--------------+ | ``UT`` | ``ST`` | ``IMAGETYP`` | ``OBJECT`` | | +--------------+--------------+--------------+--------------+--------------+ if ``IMAGETYP`` is ``sci``, in addtition these keywords are checked: +--------------------------------------------------------------------------+ | Integer keywords | +--------------+--------------+--------------+--------------+--------------+ | ``DITHER`` | | | | | +--------------+--------------+--------------+--------------+--------------+ | Float keywords | +--------------+--------------+--------------+--------------+--------------+ | ``TRAJCRA`` | ``TRAJCDEC`` | ``TRAJCEQ`` | ``TELEQNOX`` | ``ZD`` | +--------------+--------------+--------------+--------------+--------------+ | ``STRUCTEL`` | ``PARANGLE`` | | | | +--------------+--------------+--------------+--------------+--------------+ | String keywords | +--------------+--------------+--------------+--------------+--------------+ | ``TELRA`` | ``TELDEC`` | ``HA`` | | | +--------------+--------------+--------------+--------------+--------------+ If the keyword is missing, or unset, for this the test checks if the the integet value or float value is smaller than -999990, or the string value is ``999999``. The test fails if the keyword is missing or unset. Each keyword check is treated as an individual subtest. **Expected value:** ``SET`` keyword **Tolerance:** None **Actual value:** ``SET`` if the keyword is present and set, ``UNSET`` otherwise. .. _bitstat: ``common:bitstat`` - Check for stuck bits ----------------------------------------- **Header Key:** VHCBITS Check for a controller error that causes stuck bits. The tests create a histogram of the distribution of ones and zeros over the whole image array. Subtest: ``even_bits`` - Check for only even or odd bits populated """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Check for a controller error that causes only even or odd bits to be populated. The test fails if the lowest bit is filled with only ones or zeroes. **Expected value:** 0 **Tolerance:** None **Actual value:** 1 if the test failed, 0 if the test succeeded. Subtest: ``all_bits_one`` - Check for bits all one """""""""""""""""""""""""""""""""""""""""""""""""" Check for a controller error that causes bits to be all set to one. The test fails if the bits for values not around the image average are all set to one. **Expected value:** 0 **Tolerance:** None **Actual value:** Number of failed bits Subtest: ``significant_bits`` - Check for a balanced least significant bit """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Check for a controller error that causes the least significant bit is unbalanced. The test fails if less than one percent the least significant bit is filled with ones or zeroes. **Expected value:** 0 **Tolerance:** None **Actual value:** 1 if the test failed, 0 if the test succeeded. .. _nullpix: ``common:nullpix`` - Check for null pixels ------------------------------------------ **Header Key:** VHCPIXEL Check for an exceeding number of null pixels in the image. The maximum allowed number of null pixels is read from the n_null_pix column in the :ref:`nullpixel.txt ` reference file. The test fails if the number of null pixels exceeds the maximum allowed value. **Expected value:** Maximum allowed number of null pixels **Tolerance:** None **Actual value:** Actual number of null pixels .. _repeat: ``common:repeat`` - Check for the ``REPEAT`` keyword ---------------------------------------------------- **Header Key:** VHCREPT Check if the ``REPEAT`` keyword is set in the header. This keyword flags an error condition, causing the same data being written to the image data section during following observations. The test fails if the ``REPEAT`` keyword is present and set to ``TRUE``. **Expected value:** ``FALSE`` **Tolerance:** None **Actual value:** The actual value of the ``REPEAT`` keyword. .. _saturation: ``common:saturation`` - Check for saturated pixels -------------------------------------------------- **Header Key:** VHCSATUR Check for an exceeding number of saturated pixels in the image. The value above which a pixel is considered saturated is read from the saturation column, The maximum allowed number of saturated pixels is read from the max_sat_pixels column in the :ref:`nullpixel.txt ` reference file. The test fails if the number of saturated pixels exceeds the maximum allowed number. **Expected value:** Maximum allowed number of saturated pixels **Tolerance:** None **Actual value:** Actual number of saturated pixels Bias Frame Checks ================= .. contents:: :local: .. _bias_txt: Reference file: ``bias.txt`` ---------------------------- The default values for the bias frame checks are stored in the ``bias.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of specid, channel and amplifier, it specifies the columns: +--------------------+----------------------------------------------------+ | ``mean`` | Expected mean bias value ignored if specified in | | | the fits header keyword ``BIASTRGT``) | +--------------------+----------------------------------------------------+ | ``stddev`` | Expected standard deviation | +--------------------+----------------------------------------------------+ | ``max_dev_mean`` | Maximum allowed deviation for mean bias | +--------------------+----------------------------------------------------+ | ``max_dev_stddev`` | Maximum allowed deviation for mean stddev | +--------------------+----------------------------------------------------+ | ``max_diff`` | Maximum allowed deviation in mean flatness test | +--------------------+----------------------------------------------------+ | ``max_diff_rms`` | Maximum allowed deviation in stddev flatness test | +--------------------+----------------------------------------------------+ | ``line_stddev`` | Maximum allowed deviation in stddev for line check | +--------------------+----------------------------------------------------+ | ``max_fft_signal`` | Maximum signal if fft check | +--------------------+----------------------------------------------------+ .. _bias_compare: ``bias:compare`` - Reference Bias --------------------------------- **Header Key:** VHCBCMP Check the mean and the standard deviation of the bias frame against expected values. Subtest: ``bias_average`` - Mean bias value """"""""""""""""""""""""""""""""""""""""""" Compares the five sigma clipped mean of the bias frame against the expected mean value taken from the ``mean`` parameter in the :ref:`bias.txt ` reference file, or if present from the ``BIASTRGT`` header keyword. The test fails if the difference between the measured bias mean and the expected value is larger then ``max_dev_mean`` times the expected value. **Expected value:** Expected mean bias value **Tolerance:** Maximum fractional deviation from the expected mean **Actual value:** Measured mean bias value Subtest: ``bias_stddev`` - Bias standard deviation """""""""""""""""""""""""""""""""""""""""""""""""" Compares the five sigma clipped standard deviaton of the bias frame against the expected standard deviation value taken from the ``stdddev`` parameter in the :ref:`bias.txt ` reference file. The test fails if the difference between the measured bias standard deviation and the expected value is larger then ``max_dev_stddev`` times the expected value. **Expected value:** Expected standard deviation of the bias **Tolerance:** Maximum fractional deviation from the expected stddev **Actual value:** Measured bias standard deviation .. _bias_flat: ``bias:flat`` - Bias Flatness ----------------------------------- **Header Key:** VHCBFLAT Check the flatness of the bias frame by measuring mean and standard deviation in four different areas of the image: * ``[125:375,125:375]`` * ``[125:375,625:875]`` * ``[625:875,125:375]`` * ``[625:875,625:875]`` Subtest: ``flat_bias_avg`` - Flatness of mean """"""""""""""""""""""""""""""""""""""""""""" Check the flatness of the bias mean by calculating the five sigma clipped mean of the bias in four separate areas. The test fails if the difference between the measured bias mean and the expected value differ by more than ``max_diff`` for any of the regions. **Expected value:** Expected mean bias value **Tolerance:** Maximum difference between expected and measured value **Actual value:** Measured mean bias value and measured area. Subtest: ``flat_ovsc_avg`` - Flatness of mean against overscan """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Check the flatness of the bias mean against the overscan, by calculating the five sigma clipped mean of the bias in four separate areas. The test fails if the difference between the measured bias mean and the average of the overscan area is more than ``max_diff`` for any of the regions. **Expected value:** Expected mean bias value **Tolerance:** Maximum difference between overscan and measured value **Actual value:** Measured mean bias value and measured area. Subtest: ``flat-bias_stddev`` - Flatness of standard deviation """""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Check the flatness of the bias standard deviation by calculating the five sigma clipped standard deviation of the bias in four separate areas. The test fails if the difference between the measured bias mean and the expected value is larget than ``max_diff_rms`` times the expected value for any of the areas. **Expected value:** Expected bias standard deviation **Tolerance:** Maximum fractional difference from the expected stddev **Actual value:** Measured bias stddev and measured area. Subtest: ``flat_ovsc_stddev`` - Flatness of standard deviation against overscan """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Check the flatness of the bias standard deviation against the standard deviation of the overscan, by calculating the five sigma clipped standard deviation of the bias in four separate areas. The test fails if the difference between the measured bias standard deviation and the standard deviation of the overscan area is more than ``max_diff_rms`` for any of the regions. **Expected value:** Expected standard deviation of the bias **Tolerance:** Maximum fractional difference between overscan and bias stddev **Actual value:** Measured bias stddev and measured area. .. _bias_linestddev: ``bias:linestddev`` - Bias line features ---------------------------------------- **Header Key:** VHCBLSTD Check the bias for line structure in the files. Calculates the five sigma clipped mean of of each line of the bias frame, and checks the standard deviation of the resulting 1-D arry. The test fails if the the measured standard deviationand exceeds the value specified in the corresponding ``line_stddev`` column. **Expected value:** Maximum allowed standard deviation **Tolerance:** None **Actual value:** Measured standard deviation of 1-D line array .. _bias_fft: ``bias:fft`` - Bias wave features --------------------------------- **Header Key:** VHCWAVE Test for wave like structures in the bias frame, by checking for a signal in a line based fast fourier transformation. The fourier transformation is calculated for 100 lines separated by 10 lines starting at line one. For each of these lines it calculates the fourier transformation, trims and normalizes the signal array, and keeps the maximum signal. The test fails if the three sigma clipped mean of the maximum signals exceeds the value specified in the corresponding ``max_fft_signal`` column. **Expected value:** Maximum allowed fft signal **Tolerance:** None **Actual value:** Measured fft signal Arc Frame checks ================ .. contents:: :local: .. _arcs_txt: Reference file: ``arcs.txt`` ---------------------------- The default values for the arc frame checks are stored in the ``arcs.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of specid, channel and amplifier, it specifies these columns: +-------------+----------------------------------------------------+ | linefile | Filename of file with arc line positions | +-------------+----------------------------------------------------+ | peakshift | Maximum allowed shift of the lines in wl direction | +-------------+----------------------------------------------------+ | start_y | Starting row in y direction for search | +-------------+----------------------------------------------------+ .. _arc_peaks: ``arc:find_peaks`` - Number of peaks ------------------------------------ **Header key:** ``VHCPEAKS`` Check for the arc lines. The test checks, that the arc lines are found in the positions specified in ``linefile``. These positions are allowed to shift by a maximum of ``peakshift`` pixels with respect to the nominal position. All arc lines must be shifted by the same amount. The search for the lines will start at ``start_y`` in the y direction of the image. The test fails if not all lines are found. **Expected value:** Number of lines in ``linefile`` **Tolerance:** None **Actual value:** Number of lines found a the specified positions Flat Frame checks ================= .. contents:: :local: .. _n_fibers_txt: Reference file: ``n_fibers.txt`` -------------------------------- The default values for the number of fibers flat frame check are stored in the ``n_fibers.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of ifu bundle id, channel and amplifier, it specifies these columns: +-------------+----------------------------------------------------+ | n_fibers | Expected number of fibers | +-------------+----------------------------------------------------+ | tolerance | Maximum allowed deviation from expected number of | | | fibers | +-------------+----------------------------------------------------+ .. _min_flux_txt: Reference file: ``min_flux.txt`` -------------------------------- The default values for the minimum flux flat frame check are stored in the ``min_flux.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of ifu bundle id, channel and amplifier, it specifies these columns: +-------------+----------------------------------------------------+ | min_flux | Minimum allowed flux in the flat spectrum | +-------------+----------------------------------------------------+ .. _flat_n_fibers: ``flat:n_fibers`` - Number of fibers ------------------------------------ **Header key:** ``VHCNFIB`` Check for the correct number of fibers in the image. The expected number of fibers and the tolerance are read from the ``n_fibers`` and ``tolerance`` colums in the :ref:`n_fibers.txt ` reference file. To find the fibers it collapses the central 40 pixels, and tries to find the first fiber by fitting a Gaussian to a moving window, till the gaussian matches. Once the first fiber is found, it moves the search box down in steps equal to the fiber separation. If a fiber is missing, it starts seach again with the moving search box. The test fails if the number of missing fibers exceeds the tolerance limit. **Expected value:** Number of fibers **Tolerance:** Allowed tolerance **Actual value:** Actual number of fibers found .. _flat_min_flux: ``flat:min_flux`` - Minimum flux in fibers ------------------------------------------ **Header key:** ``VHCMFLUX`` Check that the flux in all fibers is above the required minimum flux, Searches for the fiber traces in the images and for each found spectrum checks that the flux at the positions in x ``100, 300, 512, 724, 924`` is above the flux read from the ``min_flux`` column in the :ref:`min_flux.txt ` reference file. The expected number of fibers and the tolerance are read from the ``n_fibers`` and ``tolerance`` colums in the :ref:`n_fibers.txt ` reference file. The test fails, if too few fibers are found with flux above the expected minimum. Each x position is treated as a separate subtest **Expected value:** Expected number of fibers minus tolerance **Tolerance:** None **Actual value:** Actual number of fibers with sufficient flux found .. _flat_col_cte: ``flat:col_cte`` - Column shift CTE issues ------------------------------------------ See :ref:`col_cte` for details Science Frame checks ==================== .. contents:: :local: .. _row_cte_txt: Reference file: ``row_cte.txt`` ------------------------------- The default values for the row shift cte science frame check stored in the ``row_cte.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of ifu bundle id, channel and amplifier, it specifies these columns: +-------------+----------------------------------------------------+ | n_hist | Histogram cutoff value | +-------------+----------------------------------------------------+ .. _sky_level_txt: Reference file: ``sky_level.txt`` --------------------------------- The default values for the minimum sky level science frame check are stored in the ``min_flux.txt`` reference file in the ``reference_files`` directory of the ``vhc_settings`` repository. For any combination of ifu bundle id, channel and amplifier, it specifies these columns: +-------------+----------------------------------------------------+ | sky_level | Minimum expected counts above median of overscan | +-------------+----------------------------------------------------+ .. _n_dithers: ``hetdex_dithers:n_dithers`` - Number of dithered exposures ----------------------------------------------------------- **Header key:** ``VHCNDITH`` Check the number of dithers for a shot. Subtest: ``n_dither_nexp`` - Correct number of exposure directories """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Check for the correct number of exposure directories for a dithered hedex operation. The expected number is taken from the ``n_exposures`` option of the ``hetdex_dithers`` recipe and compared with the number of exposure directories found. The pattern for the exposure directory is taken from the ``exposure_dir`` option in the ``[common]`` section. The test fails if a wrong number of exposure subdirectories is found. **Expected value:** Expected number of exposure directories **Tolerance:** None **Actual value:** Found number of exposure directories Subtest: ``n_dithers_keyword`` - Correct dither keywords """""""""""""""""""""""""""""""""""""""""""""""""""""""" Check that all files in each exposure directory have the same value set for the ``DITHER`` keyword. The test fails if not all files in each exposure directory have the same value set for the ``DITHER`` keyword. **Expected value:** Expected number of exposure directories **Tolerance:** None **Actual value:** Number of dither keys found Subtest: ``n_dithers_probe`` - Correct number dither keywords """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""" Check that the values found in the ``DITHER`` keywords match the dithering strategy. If dithering is performed using the dithering mechanism, there must be in total as many values of ``DITHER`` as there are exposure directories, otherwise the values of ``DITHER`` must be the same across all the exposures. The ``dither_with_probe`` option of the ``[hetdex_dithers]`` configuration section instructs the driver whether the dithering is done offsetting the fiducial position in the drive probes (``true``) or using the dithering mechanism (``false``). The test fails if the values of the ``DITHER`` keywords do not match the dithering strategy. **Expected value:** Expected number of ditherent ``DITHER`` keys **Tolerance:** None **Actual value:** Number of dither keys found .. _row_cte: ``hetdex_dithers:row_cte`` - Check for row shift CTE problems ------------------------------------------------------------- **Header key:** ``VHCRSCTE`` Check the problems with the charge transfer efficienct (CTE) during the row shift during readout. This issue causes charges to be smeared along the readout direction. This causes cosmics to be elongated along the row shift direction. To find these issues, ``yoda`` is run to detect cosmics in the image. Then it creates a histogram of the position angles of the detected sources. The test fails if the count of the 90 degree bins of the position angle histogram exceed ``n_hist`` times the mean of the remaining histogram. The ``n_hist`` value is read from the :ref:`row_cte.txt ` reference file. If ``yoda`` is not found, the driver is marked as ``NOTRUN``. **Expected value:** Average position angle **Tolerance:** Standard deviation of the histogram **Actual value:** Number of detections in the 90 degree bin of the histogram .. _col_cte: ``hetdex_dithers:col_cte`` - Check for column shift CTE problems ---------------------------------------------------------------- **Header key:** ``VHCCSCTE`` Check the problems with the charge transfer efficienct (CTE) during the column shift during readout. This issue causes charges to be smeared along the column readout direction. This causes charges from the image region to be dragged over into the overscan region leading to a drop off at the beginning of the overscan region. To find these issues, each line of the two columns of the overscan region closest to the image region are averaged. Every line of last 20 columns of the image region is averaged, and the five sigma clipped mean of the remaining overscan region calculated. If less than 50 lines of the cte region of the overscan have sufficient flux (more than the mean of the overscan plus half the standard deviation of the overscan) the check is aborted and marked as ``NOTRUN``. The test fails if the flux in the lines in the cte region is closer to the flux in the image region than the overscan region. **Expected value:** Average difference between overscan and cte region **Tolerance:** None **Actual value:** Average difference between image and cte region .. _sky_level: ``hetdex_dithers:sky_level`` - Check for sufficiont level --------------------------------------------------------- **Header key:** ``VHCSKYLV`` Check for sufficient sky level in the indivdual spectra. It collapses the central 40 pixels, calculates the five sigma clipped mean of this region, and subtracts the mean overscan. The test fails if the resulting sky level is below the expected sky level read from the ``sky_level`` column in the :ref:`sky_level,txt ` reference file. **Expected value:** Expected sky level **Tolerance:** None **Actual value:** Measured sky level region