Optical and Infrared Division

M E M O R A N D U M

August 2, 1990

To: Doug Mink, Charles Maxson, John Huchra, Susan Tokarz

Subject: Yet Again REVISED Archive format for Reticon reduced files

From: Bill Wyatt

This documents the archive format for Reticon reduced files on the Telescope Data Center system. The formats for the analysis-related record are set out below.

- HEADER (variable length) ASCII

This record consists of ASCII lines in the form Keyword = Parameter. It is similar to FITS except that the lines are not any particular length and are terminated with a newline (a linefeed on Unix systems). The minimal set of parameters are below. Other parameters, comments, and history lines may be added. As in FITS, the RA, DEC and other strings are enclosed in single quotes.Minimum parameters:

keyword format what RFN = integer Reduced File Number OBJECT = 'name' File name RA = 'hh:mm:ss.s' right ascension DEC = 'dd:mm:ss.s' declination EPOCH = float epoch of RA, Dec in years (e.g. 1950.0) EXPTIME = float integration time, seconds JDN = float Julian Day Number at end of exposure, topocentric GJDN = float Julian Day Number at middle of exposure, topocentric HJDN = float Julian Day Number at middle of exposure, heliocentric UT = 'hh:mm:ss' Universal Time at middle of exposure DATE-OBS = 'dd/mm/yy' Date of observation PI = 'name' Principal Investigator PROGRAM = 'string' Program name or number OBSERVER = 'name' observer at telescope FILTER = 'string' Filter name END Header ending line

- Note 1: Only GJDN and HJDN of the Julian Day parameters are generally given; if the middle of the exposure cannot be computed, only the JDN will be given. Older data from the Nova systems will be JDN, while the new data system will have GJDN and HJDN if the object coordinates were given and only GJDN otherwise.
- Note 2: The UT time given above is always the estimated middle of the exposure, computed by subtracting off half the exposure time from the exposure ending time and doing an average weighted by the exposure times of each integration. The UT time is topocentric.

- REDUCESUMMARY 96 binary

This contains various parameters, including some duplicated in the HEADER record, above.format what int*4 Reduced File Number float*4 RA, radians float*4 Dec, radians float*4 Epoch of coordinates, years (e.g. 1950.0) float*8 Julian Day Number (but see note to HEADER section) float*4 Integration time, seconds float*4 Heliocentric Velocity correction, km/sec int*2 telescope number: 0 = unknown 1 = FLWO 61cm, 2 = FLWO 1.5m 3 = MMT 4 = ORO 1.5m 5 = FLWO 1.2m (48-inch) 6 = MMT 6.5m upgraded int*2 Grating number (600 lines/mm, etc.) int*2 Image tube number int*2 Object Category float*4 observatory longitude, radians float*4 observatory latitude, radians int*4 Number of left-slit files minus number of right-slit files (i.e., should be zero) float*4*4 Ncheck RMS, 2-pattern, 4-pattern, 8-pattern float*4 Beginning-end comparison shift, pixels float*4 Left-right comparison shift, pixels float*4 Mean comparison line width float*4 Mean sky line width float*4 Hour angle, radians float*4 Sidereal time, radians float*4 air mass

- REDUCESUMMARY2 120 binary

This contains various parameters, including some duplicated in the HEADER record, above.format what int*4 Reduced File Number float*4 RA, radians float*4 Dec, radians float*4 Epoch of coordinates, years (e.g. 1950.0) float*8 HJDN = Heliocentric Julian Day Number (but see note to HEADER section) float*4 Integration time, seconds float*4 HCV = Heliocentric Velocity correction, km/sec int*2 telescope number: 0 = unknown 1 = FLWO 61cm, 2 = FLWO 1.5m 3 = MMT 4 = ORO 1.5m 5 = FLWO 1.2m (48-inch) 6 = MMT 6.5m upgraded int*2 Grating number (600 lines/mm, etc.) int*2 Image tube number int*2 Object Category float*4 observatory longitude, radians float*4 observatory latitude, radians int*4 Number of left-slit files minus number of right-slit files (i.e., should be zero) float*4*4 Ncheck RMS, 2-pattern, 4-pattern, 8-pattern float*4 Beginning-end comparison shift, pixels float*4 Left-right comparison shift, pixels float*4 Mean comparison line width float*4 Mean sky line width float*4 Hour angle, radians float*4 Sidereal time, radians float*4 air mass float*8 GJDN = Topocentric Julian Day Number (but see note to HEADER section) float*4 BCV = Barycentric Velocity correction, km/sec float*4 altitude of observatory, meters. char*8 spare bytes

- SKELETON (variable length) ASCII

Like the HEADER record, the SKELETON record consists of lines of the form Keyword = Parameter. The keywords control the reduction of the raw data, and can be quite extensive. Fortunately, most of the reduction parameters are repeated time after time, so these are gathered up into a so-called `template' file. This skeleton record then refers to a template and contains only the unique information, such as file names, that need be specified for this reduced file.The full documentation of the keywords and their effects on reduction will be in another document. Meanwhile, a minimal set is documented below:

keyword format what rfn = integer Reduced File Number template = 'pathname' Template file for defaults r5w = 'pathname' File to read preliminary 5th-order wavelength solution from rawdir = 'pathname' Directory containing raw data files telescope = integer Telescope number (used only if no TELESCOP keyword in FITS header) grating = integer Grating number itube = integer Image tube number objtype = integer Object type (e.g., star = 32, Redshift object = 2) incandN = 'pathname' Incandescent normalization file N, where N = 1, 2, etc., relative to rrrraaaawwwwddddiiiirrrr compN = 'pathname' Comparison file N (N = 1, 2, ...), relative to rrrraaaawwwwddddiiiirrrr objectN = 'pathname' Object data file N (N = 1, 2, ...), relative to rrrraaaawwwwddddiiiirrrr END End marker for skeleton record

- COMMENTS (variable length) ASCII

Each line beginning with `COMMENT' in the raw data file FITS header is copied to this record (without the leading `COMMENT'). Leading and trailing blanks are removed and each line is terminated with a newline. The comments end with an `END' line. - DISTORTION (variable length) binary

This record, if present, contains the right-to-left distortion removal polynomial determined during the reduction and the basic fitting results.format what int *4 Number of lines in fit (i.e. matched on both sides) float*4 RMS residual of fit int*4 dimension of polynomial pointer*4 Pointer to coefficients in the in-core C poly structure (i.e. irrelevant to archive file) float*8 Midpoint of independent variable (subtracted off first when evaluating the polynomial) float*8 Scaling factor for independent variable (divided through after midpoint subtraction when evaluating the polynomial; zero means no scaling done) float*8*N N coefficients of the polynomial, lowest order term first

- COARSEWAVER (variable length) binary

This contains the polynomial used during the wavelength-table-matching phase of the reductions. By default this is a 5th-order polynomial (dimension 6) for a record length of 164 bytes, but if a lower order fit was selected in the reduction, then that will be shorter. The format is similar to that for the distortion polynomial, except the wavelength limits and RMS of the fit are also in the record.format what float*4 bluest wavelength line identified float*4 reddest wavelength line identified float*4 RMS of wavelength fit, Angstroms float*4 RMS of wavelength fit, pixels int*4 Number of lines in fit

**Pixel -> Wavelength polynomial (WAVER)**int*4 dimension of WAVER polynomial (nominally 6) pointer*4 Pointer to coefficients in the in-core C poly structure (i.e. irrelevant to archive file) float*8 Midpoint of independent variable (subtracted off first when evaluating the polynomial), pixels float*8 Scaling factor for independent variable (divided through after midpoint subtraction when evaluating the polynomial; zero means no scaling done) float*8*N N coefficients of the WAVER polynomial (nominally 6), lowest order term first**Wavelength -> Pixel polynomial (IWAVER)**int*4 dimension of IWAVER polynomial (nominally 6) pointer*4 Pointer to coefficients in the in-core C poly structure (i.e. irrelevant to archive file) float*8 Midpoint of independent variable (subtracted off first when evaluating the polynomial), Angstroms. float*8 Scaling factor for independent variable (divided through after midpoint subtraction when evaluating the polynomial; zero means no scaling done) float*8*N N coefficients of the IWAVER polynomial (nominally 6), lowest order term first - FINEWAVER 196 binary

This contains the polynomial used for the final fitting and line-rejection pass of the reduction phase. The default is a 7th-order polynomial (dimension 8) for a length of 196 bytes, but if a lower order fit was selected in the reduction, then the length given above will be shorter. The format is similar to that for the COARSEWAVER polynomial, except the wavelength limits are for the reddest and bluest lines actually used in the fit.format what float*4 bluest wavelength line used float*4 reddest wavelength line used float*4 RMS of wavelength fit, Angstroms float*4 RMS of wavelength fit, pixels int*4 Number of lines in fit

**Pixel -> Wavelength polynomial (WAVER)**int*4 dimension of WAVER polynomial (nominally 8) pointer*4 Pointer to coefficients in the in-core C poly structure (i.e. irrelevant to archive file) float*8 Midpoint of independent variable (subtracted off first when evaluating the polynomial), pixels float*8 Scaling factor for independent variable (divided through after midpoint subtraction when evaluating the polynomial; zero means no scaling done) float*8*N N coefficients of the WAVER polynomial (nominally 8), lowest order term first**Wavelength -> Pixel polynomial (IWAVER)**int*4 dimension of IWAVER polynomial (nominally 8) pointer*4 Pointer to coefficients in the in-core C poly structure (i.e. irrelevant to archive file) float*8 Midpoint of independent variable (subtracted off first when evaluating the polynomial), Angstroms. float*8 Scaling factor for independent variable (divided through after midpoint subtraction when evaluating the polynomial; zero means no scaling done) float*8*N N coefficients of the IWAVER polynomial (nominally 8), lowest order term first - COMPLINES (variable length) binary

This record contains all the comparison and sky lines found by the reduction program, even those not matched to a line in the wavelength table. (N.B.: archive files converted from Nova files are limited to 64 lines, and only include those lines that matched a table line.) The record consists of some summary information and then the parameters for each line:format what int*4 Total number of lines in table, N int*4 Total number of lines matched (comparison + sky) int*4 Number of skyline matched float*4 Mean comparison line width, pixels float*4 Mean sky fit residual, Angstroms line*28*N N lines, structure as below Line records: format what float*4 Center of line, pixels. If negative, this was rejected by the 7th order fit (see also the rejection flag), and the absolute value is the true line center. float*4 Table Wavelength of line, Angstroms. If zero, no wavelength match was made. If negative, this was rejected by the 7th order fit (see also the rejection flag). float*4 Height of line, counts float*4 Width of line, pixels float*4 Continuum level of line, counts float*4 Slope of continuum at line center, counts int*4 Rejection flag: >= 0 : line used in 7th order fit (i.e. not rejected) = -1 : line never matched to a wavelength; never used <= -2 : line rejected by wavelength window criterion in the 5th order fit, or, if the line center is negative, by the 7th order fit.

- SPECTRUM (variable length) BITS nbits type DIM naxes xlen ylen ...

This record contains the spectral or image data. The BITS parameter given in the label gives the number of bits in the data, as with FITS. The type parameter is either IIII for integer or FFFF for IEEE floating point (both network byte order). If the value is 8 or 16, the data must be integer.The values following the DIM keyword give the number of data axes and their sizes, as in FITS, with the xlen coordinate incrementing fastest.

- ANALYSISSUMMARY 40 binary

Summarizes the overall object velocity, including the separately determined emission and correlation velocities. Both are not always calculated.format what int*4 Quality control status: 0 - Not reviewed yet 1 - Inconclusive velocity determination 2 - Insufficient wavelength coverage 3 - Incorrect redshift velocity 4 - Correct redshift velocity float*4*3 Overall CZ8, error, confidence float*4*3 Correlation CZ8, error, R value float*4*3 Emission CZ8, error, scatter x82

- CORRELATION (variable length) binary

Correlations against various templates.format what float*4*2 LW1, LW2: Blue and Red ends of template8|- float*4 RMS of prepared object spectrum int*2 Nt, Number of templates following int*2 Nc, Number of emission lines chopped int*2*2*Nc ln(L) of line center, +/- range in pixels chopped. corr*44*Nt Nt template correlations, where the structure is as below Template correlation structure: format what char*16 Template name, space padded float*4 Center of correlation peak, lnL, where z = exp(c/AA)-1 float*4 Height of correlation peak float*4 width (fwhm) of correlation peak float*4 Antisymmetric RMS => Rvalue = height/(2 * Arms) float*4 Correlation RMS float*4 Template shift, km/sec. float*4 AA = pixels per lnL = (N - 1)/(ln LW2 - ln LW1)

|- Note that the LW1 and LW2 wavelength limits are integers on the original Nova files, and are meaningful only for the Z-Machine data, where all templates were specified to have the same limits. The limits are not saved for the echelle templates, but the AA value - CORRVELOCITYDISPERSION 32 binary

Summarizes the dispersion velocity calculated from the stellar template correlation width. This record is not present if the dispersion is not calculated.format what float*4*3 Velocity dispersion, +68%, -68% confidence errors float*4 Velocity width RMS of stellar template float*4*4 Correlation width to dispersion conversion coefficients, 3rd order polynomial

- EMISSIONLINES (variable length) binary

Gives the emission line data found by the analysis program. This record is not present if emission lines are not fitted.format what int*2 Number of emission lines found. int*2 Number of emission lines contributing to emission CZ8. line*76*N N emission lines, structure as below Emission line records: format what float*4 Rest wavelength, Angstroms. float*4 Pixel center float*4 Line height float*4 Line width float*4 Continuum level float*4 Continuum slope float*4*2 Line center, +/- error from Gaussian fit float*4*2 Line height, +/- error from Gaussian fit float*4*2 Line width, +/- error from Gaussian fit float*4*3 Parabolic fit to continuum (during Gaussian fit) float*4*2 Equivalent width, error (Angstroms) float*4 Chi^2 int*2 Degrees of freedom int*2 Weight in average (zero means not used in emission velocity average).

contains most of the necessary information. - EQUIVALENTWIDTH (variable length) MMMMEEEETTTTHHHHOOOODDDD type binary

Equivalent widths of various lines. If the MMMMEEEETTTTHHHHOOOODDDD is type `NOVA', these are copied from the Nova reduced files, and the equivalent widths are not calculated in a standard way.format what line*16*N N equivalent widths, structure as below Line records: format what char*8 Line name (e.g. MgH, CaK, etc.), blank padded float*4*2 Equivalent width, +/- error, milliAngstoms

The changes from previous formats are:- A new record type, named REDUCESUMMARY2, is created. This has extra fields compared to the original REDUCESUMMARY record. All archive files should be rewritten with this extra info, but until that hap- pens, a distinction has to be made between the types.
- The HEADER record and the REDUCESUMMARY2 records con- tain both the GJDN and HJDN keywords and values, if possible. This eliminates the need to recalculate topocentric time if it is needed.
- Both the barycentric (BCV) and heliocentric (HCV) velocity corrections are given in the REDUCESUMMARY2 record.
- The observatory altitude has been added to the REDUCESUMMARY2 record.
- Terminology change: what was called `geocentric' before (e.g. GJDN) will now be called `topocentric'.

Every record starts with an ASCII 48-character descriptive label giving at minimum a name and the length in free-field format, followed by the data record. This record is not a priori in any particular format; that is determined strictly on a record-by-record basis. The length given in the 48-character label is used by routines wishing to skip the data without further ado.

Note that a given archive file need not have the records in the same order as given here, and that applica- tion or instrument specific records can be added at any time. All programs accessing these files should simply skip or copy unknown records as appropriate.