PRIMUS Data Reduction

From NYU CCPP Wiki

This is a page about the PRIMUS project.

Contents

[edit]

Required packages

The PRIMUS data reduction piepline is written in IDL and requires several IDL packages fo run successfully. These include:

  1. primus -- the location of this should be set to $PRIMUS_DIR in your .cshrc or similar file
  2. idlutils -- cvs version (current at least through June 2006)
  3. idlspec2d -- v5_0_0 (if you are running on Mac OS X, you will want the cvs version)
  4. kcorrect
  5. ds9
  6. xpaset -- This package allows images to be displayed to the command line or from IDL scripts
[edit]

Required Data

Along with the Raw data from IMACS, there are several data products that you will need to reduce PRIMUS data.

  1. The _slits_ file for each mask you want to reduce. The $PRIMUS_DIR/bin/get_slitfiles.pl will download these to the $PRIMUS_DIR/masks/ directory
  2. The slit_tags.dat should be updated to reflect any new masks names before running the pipeline. This file specifies which tags should be used for the magnitude of each source and other useful defintions.
  3. The $PRIMUS_DIR/etc/primus_batch.par and $PRIMUS_DIR/etc/primus_flatbatch.par files should be updated with the IDL code queue_batch in order to run in batch mode. This code allows the user to specify which arc and flat files to use when reducing each night of data.


[edit]

Running the 2-d pipeline

Only one command is needed to run the full 2-d extraction of PRIMUS data.

IDL> run_batch, rerun=rerun, flatlook=

This command will first open a ds9 window and load one image for each mask and prompt the user to select one trace with a given rectangular region. This is used to provide a robust first guess to the slit locations when locating each trace. This takes a few minutes of user manipulation.

Once the models have been created for each mask, the flat fields are constructed for any night with flat field observations. If the user set the /flatlook option when running run_batch, the code will stop after this step to allow the user to examine each flat generated to be sure that no artifacts (star trails, etc) are present in the final stack.

After the flats are combined, each of the arcfiles specifiy by the primus_batch.par file will be processed and the wavelength scale term will be calculated and written to disk. Now that all of the initial calibrations are processed, the full extraction process will begin. The output will be placed in the $PRIMUS_REDUX directory according to the PRIMUS data model.

[edit]

Overview of 2d Pipeline

TODO: Add names to tasks that still need attention/creation.

  • Preprocessing per night/run: Create normalized combined flat-fields

(P1) Super-flats should be named flat_utYYMMDD.fits and be applied to the data from properly matched nights.

  • Per Science Mask:
  1. If multiple science frames, coadd first and solve for halo image. If not multiple frames, than default to quick-look pipeline?
    • coadd_images_agg [Done]
  2. Find apertures and create aperture structure file
    • Use "hogg_find_slits".
  3. Analyze arcframe associated with science mask.
    • fit_helium_arc [Done, needs testing]
  4. Run optimal extraction on coadded frame, and fit coadded sky profile in each good aperture.
    • extract_newopt [Done, but need to save/write all necessary data]
  5. Do a global fit of sky profile parameters and flag bad sky extractions in coadded slits
    • Save the best slit position and IMACS psf from the global fit, this will be used as input for each of the individual extractions below. This is NOT done, and will need serious work with robust fitting.
  6. Loop over each individual frame and perform optimal extraction
    • The extraction is written (extract_newopt) with these features to be added:
    1. Need to use the global sky fit parameters as the first guess for slit centers and IMACS psf [Scott needs to implement this]
    2. Need to iterate sky model subtraction and use a prior that residual sky = 0 to successfully extract severely beblended slits. Difficult implementation and only helps the worst 5% of the slits.
    • Note here: We can store the coadded A + B spectra in each frame. This is useful for quick look but should not be used in the final science analysis.
  7. Coadd A spectra, Coadd B spectra. Output a single .extract structure with coadded A spectra, coadded B spectra and if needed the sum of (coadd A + coadd B).
    • The code exists for this step (coadd_spectra) just need to make sure all tags necessary for 1d fitting are passed along.
Retrieved from "http://howdy.physics.nyu.edu/index.php/PRIMUS_Data_Reduction"