Hectospec Data Pipeline Status as of 2003-01-30

I. Preparation

1. Get positions to better than 0.25 arcseconds
   • Catalogs
     You should use only a large catalog with good sky coverage and good astrometry, such as the GSC II, 2MASS PSC, USNO-B1.0, and the smaller Tycho-2. The catalog from which you select positions should all program objects plus guide stars.
     WCSTools SCAT should be able to convert anything reasonable into an acceptable input for the next step.
     Astrometric cluster catalogs are probably not OK because there usually will not be available guide stars on the same system.
     We await the release of the complete 2MASS Point Source Catalog and would like to have a local copy of the GSC II as the STScI server is often down on weekends and at other awkward times.
   • Images
     Use procedure outline on the "Getting Good Coordinates" web page to make catalogs for each image using WCSTools software using one of the above catalogs as the reference catalog. The large catalog can then provide guide star candidates.
     Software is still needed to merge catalogs into single project catalog.
2. Find fiber assignments and select guide stars
   Use John Roll's fitfibs or xfitfibs to create fiber assignment tables to be taken to the telescope.
   This step needs further testing and documentation before being unleashed on the world.
   It is not clear how guide stars are selected, and we need to know the magnitude limits for guide stars. We should be able to use existing software, such as scat to prepare the input for this stage.

II. Observation

• Observing protocols
  1. Make sure you have prepared your target list so that coordinates are good to 0.25".
     DO NOT COME TO THE TELESCOPE WITHOUT PREPARED AND CHECKED COORDINATE LISTS.
  2. Before arriving at the telescope, run the fiber positioning program; check to make sure some fibers are on sky and the rest are on the target objects.
  3. Take at least 10 biases each night.
  4. Take 10 domeflats each night.
  5. Each night take 10 twilight sky flats varying the exposure time from 2 seconds to 50 seconds.
  6. Take at least 5 darks at least once during your run. Dark exposure times should be the same as your longest object exposures.
  7. Take a mininum of three 5-minute comparison exposures each night.
  8. For each target pointing, take at least 3 identically timed exposures so we can automatically remove cosmic rays.
  9. Check with quick look after each new target exposure to make sure you have an adequate integration time and to check your positions: image and skys.
  10. Velocity standards?
      There should be standard fields for standards.
  Definitions
  Target List: Observer's list of objects to be observed.
  Field: Observer defined region of target list
  Pointing: Fiber positioning of field; may be multiple pointings of a field.
  Exposure/Integration: One of three or more images taken at a specific pointing.

• Quick Look
  Needed at multiple times:
  1. After first exposure to check brightness. Display counts above sky in each fiber.
     Maureen is onto an IRAF task which will give total flux along a trace, so this look would be a combination of viewing the image and checking the flux for any or all fibers.
  2. After second exposure to check whether sources are showing up.
     View images of each chip after running ccdproc and combining amplifiers using ds9 or SAOimage.
     This can be done now.
  3. After third or final exposure to make sure data is usable.
     Reduce data using dohecto and sumspec, viewing the resulting 2-d stacked spectrum files (one per chip) with image viewer.
     This is the pipeline we have developed. It would be fairly easy in either SAOimage or ds9 to add a task to view any specified fiber as a graph.

III. Reduction

1. Remove bias and combine amplifiers
   Use mscred ccdproc in IRAF to remove the bias and dark levels from the frames and combine amplifiers into a two images, one per detector, each containing up to 150 fiber targets.
   Existing IRAF code should work perfectly, but needs to be tested.
   We need raw images from the separate chips for testing and long exposure test data to see if the dark current is significant.
2. Remove Cosmic Rays
   Use imcombine to combine the multiple exposures at a single pointing, removing cosmic rays in the process.
   Existing IRAF code should work perfectly, but needs to be tested.
3. Extract Spectra
   dohspec is the main processing program. Before running, check the parameter settings by typing epar dofibers and typing params. Run dofibers on each of the images created in step 1. The output will be a multispec file in which each of the fibers has been normalized, perhaps scattered light corrected, traced, extracted and wavelength calibrated.
   This task will have to be rewritten or the wavelength dispersion correction done as a separate step afterward if we have to correct for shifts in spectrum position. The existing form of dofibers can be used for quick-look reduction. How will sky flats be taken
4. Recalibrate Spectra
   Find and repair any shift which occured to the spectra.
   Use an IRAF task (to be written) similar to reref which will be used, once it is debugged, to analyze and fix calibration shifts.
5. Rebin Spectra for sky subtraction
   Use rvsao.sumspec to rebin each of the 300 extracted spectra from the multispec file into a stacked-spectrum 2-D file so they all have the same number of pixels and cover the same wavelength range.
   This task works. This is where quick look for all of the spectra could stop.
6. Create individual spectrum files
   Use the hectospec.hsplit task to split the two files of stacked spectra into individual multispec files, putting these newly created files into a new directory.
   This task works.
7. Remove Background/Sky
   • In this newly created master subdirectory, and outside of IRAF, run skyproc which finds, sums, and then subtracts skies from the object spectra. The output files are multispec files with 3 vectors: sky-subtracted object, sky + object, and sky.
     This program works.
   • Or use singular value decomposition to remove skies from the stacked spectrum file after rebinning and before creating the individual spectrum files.
     There is a script in the IRAF svdfit package which will run the multiple passes necessary to do this.
   • Produce a total count summary and compare with input magnitudes.
     A script using IRAF's imstat task (or Doug's more scriptable variation istat) would do the trick.
8. Compute redshifts
   Use IRAF rvsao.xcsao to find the redshifts of all of the object spectra.
   We will have to make or find good templates for Hectospec.
9. Quality Control
   Methods are yet to be determined.

IV. Distribution

V. Archive