The SAO Telescope Data Center in Cambridge, Massachusetts, supports

Sloan Digital Sky Survey at the CFA Data Release 6

Photometric Data

The PhotoPrim.db starbase file is a subset of the SDSS PhotoPrimary view of the entire photometry database. It has been sequentially indexed (i.e. physically sorted) by declination, (the dec column), into 1/10 degree bins and right ascension-sorted (ra column) within each declination bin. The database is located in the directory: /data/astrocat/SDSS-dr6.

One important improvement in DR6 is the improved photometric calibration, known as UBERCAL. See UBERCAL for more information on the algorithm. The Ubercal-calibrated magnitudes and colors are in new columns, whose names are prefixed with ub_. Thus, any queries using the old names will return the old values and data sets, but new queries presumably should use the new forms. Note also that the model mags have not had ub_ versions included here.

There are 229,003,861 rows in the database, out of over 387,000,000 in the full PhotObjAll set. Using the search program from the Starbase package (see http://cfa-www.harvard.edu/~john/starbase/starbase.html) makes obtaining region-delimited objects quick even for this size database of about 261 GB.

The selected columns are:

objID fieldID specObjID skyVersion rerun run camcol field obj flags flags2 type status ra dec probPSF u g r i z Err_u Err_g Err_r Err_i Err_z type_u type_g type_r type_i type_z flags_u flags_g flags_r flags_i flags_z flags2_u flags2_g flags2_r flags2_i flags2_z psfMag_u psfMag_g psfMag_r psfMag_i psfMag_z psfMagErr_u psfMagErr_g psfMagErr_r psfMagErr_i psfMagErr_z psfMag_u_g psfMag_g_r psfMag_r_i psfMag_i_z fiberMag_u fiberMags_g fiberMag_r fiberMag_i fiberMag_z fiberMagErr_u fiberMagErr_g fiberMagErr_r fiberMagErr_i fiberMagErr_z fiberMag_u_g fiberMag_g_r fiberMag_r_i fiberMag_i_z petroMag_u petroMag_g petroMag_r petroMag_i petroMag_z petroMagErr_u petroMagErr_g petroMagErr_r petroMagErr_i petroMagErr_z petroMag_u_g petroMag_g_r petroMag_r_i petroMag_i_z extinction_u extinction_g extinction_r extinction_i extinction_z sky_u sky_g sky_r sky_i sky_z skyErr_u skyErr_g skyErr_r skyErr_i skyErr_z isoA_u isoA_g isoA_r isoA_i isoA_z isoAErr_u isoAErr_g isoAErr_r isoAErr_i isoAErr_z isoB_u isoB_g isoB_r isoB_i isoB_z isoBErr_u isoBErr_g isoBErr_r isoBErr_i isoBErr_z isoPhi_u isoPhi_g isoPhi_r isoPhi_i isoPhi_z isoPhiErr_u isoPhiErr_g isoPhiErr_r isoPhiErr_i isoPhiErr_z conv_u conv_g conv_r conv_i conv_z ub_psfMag_u ub_psfMag_g ub_psfMag_r ub_psfMag_i ub_psfMag_z ub_psfMag_u_g ub_psfMag_g_r ub_psfMag_r_i ub_psfMag_i_z ub_petroMag_u ub_petroMag_g ub_petroMag_r ub_petroMag_i ub_petroMag_z ub_petroMag_u_g ub_petroMag_g_r ub_petroMag_r_i ub_petroMag_i_z ub_fiberMag_u ub_fiberMag_g ub_fiberMag_r ub_fiberMag_i ub_fiberMag_z ub_fiberMag_u_g ub_fiberMag_g_r ub_fiberMag_r_i ub_fiberMag_i_z

To see this for yourself, use the Starbase progran headline and execute:

  headline </data/astrocat/SDSS-dr6/PhotoPrim.db

Full documentation on these columns is available at: http://cas.sdss.org/dr6/en/help/browser/browser.asp?n=PhotoPrimary&t=U and href="http://www.sdss.org/dr6/algorithms/fluxcal.html#ubercal"

Note that the u, g, r, i, z magnitudes are the modelMag magnitudes from the SDSS view, and to get the dereddened magnitude, you need to subtract the corresponding extinction_ value. The ub_petroMag_x magnitudes are recommended for use with galaxies and the ub_psfMag_x magnitudes are recommended for stars.

Also, note that the colors, e.g. ub_petroMag_u_g, are not in the original table view from Sloan, but were computed including the extinction corrections, when extracting the values to put in this database, e.g.:

  ub_petroMag_u_g = (petroMag_u - extinction_u + conv_u) - (petroMag_g - extinction_g + conv_g)

and the original, pre-Ubercal:

  petroMag_u_g = (petroMag_u - extinction_u) - (petroMag_g - extinction_g)

These columns have been indexed for use by search, where all but the ra & dec are secondary indices:

ra dec type specObjID objID petroMag_u petroMag_g petroMag_r petroMag_i petroMag_z petroMag_u_g petroMag_g_r petroMag_r_i petroMag_i_z psfMag_u psfMag_g psfMag_r psfMag_i psfMag_z psfMag_u_g psfMag_g_r psfMag_r_i psfMag_i_z fiberMag_u fiberMag_g fiberMag_r fiberMag_i fiberMag_z fiberMag_u_g fiberMag_g_r fiberMag_r_i fiberMag_i_z ub_petroMag_u ub_petroMag_g ub_petroMag_r ub_petroMag_i ub_petroMag_z ub_petroMag_u_g ub_petroMag_g_r ub_petroMag_r_i ub_petroMag_i_z ub_psfMag_u ub_psfMag_g ub_psfMag_r ub_psfMag_i ub_psfMag_z ub_psfMag_u_g ub_psfMag_g_r ub_psfMag_r_i ub_psfMag_i_z ub_fiberMag_u ub_fiberMag_g ub_fiberMag_r ub_fiberMag_i ub_fiberMag_z ub_fiberMag_u_g ub_fiberMag_g_r ub_fiberMag_r_i ub_fiberMag_i_z

The primary index being on RA and Dec means the database file itself is sorted by RA and Dec. In this case the sorting is optimized into into bins of 0.1 degrees in Dec and by RA within each Dec bin, so that nearby objects are likely to be in the same disk block or at least nearby. Each secondary index consists of a sorted file of pointers to the indexed column. The search program uses these indices, if available, for fast access. See also the section below Searching the database.

Validating objects in the database

Objects must be validated before you can be sure what you're getting.

For example, if the type field is 3, the object is (supposed to be) a galaxy, type == 6 is a star and type == 8 is a sky exposure. The full type table, reproduced from the SDSS web site:

Class Name Code Unknown UNK 0 Cosmic Ray CR 1 Defect DEFECT 2 Galaxy GALAXY 3 Ghost GHOST 4 Known object KNOWNOBJ 5 Star STAR 6 Star trail TRAIL 7 Sky SKY 8

Also, each object has associated flag bits from the reduction stage to allow selection or rejection on various citeria. These are 64-bit quantities in the original SDSS database. Since Starbase is an awk front end, it cannot handle these as numbers, so I have split them into two 32-bit fields, calling the lower-order one flags and the high order one flags2. Then, one or more bits can be tested using the bit-field testing functions, and(), or(), not(), and xor().

For example, the 0x4 bit signifies an object too close to the edge of the frame, the 0x40000 bit means the object has one or more saturated pixels, while the 0x400000 bit means the sky level was bad. To test and reject objects with any of these conditions, the following pipeline fragment can be used:

  [...input... ] | row 'and(flags, 0x440004U) == 0' | [...output...]

NB: The U suffix forces the starbase programs to interpret the number as unsigned, which is what you need for bit logical operations on all 32 bits.

Since the 'stationary' bit is 0x1000000000, it is in the upper 32 bits, so the bit-field and test would be

and(flags2,0x10U) == 0x10U

and has to be tested separately from the bits in flags.

The full set of 64 flag bits is documented at the cas.sdss.org site.

Searching the database

The search and row commands from the Starbase package can be used to search for rows with specific objects or values. The URL for the full Starbase documentation is:

http://cfa-www.harvard.edu/~john/starbase/starbase.html

Briefly, search uses index files, if available, to quickly seek to matching rows in the database, while row is a front-end for the awk variant mawk, for statements where the column names substitute for $1, $2, etc., so the user does not have to know the column order of the database.

To find all the objects with photometry near one or more coordinates, a small Starbase tab-separated table containing the RA and Dec is used containing the center(s) of the space to be searched. The example below is for one coordinate, but more are possible (the SDSS coordinates are all in degrees).

There are multiple ways to create a Starbase table. Two examples:

echo "190. 50." | fldtotable ra dec >/tmp/table

printf "ra\tdec\n-\t-\n190.\t50.\n" >/tmp/table

Both of these result in a table that looks like:

  ra      dec
  --      ---
  190.    50.

where the whitespace between the columns is a single tab character. Then, using this table /tmp/table, the command to search around the coordinate with a radius of 0.05 degrees is:

  search PhotoPrim.db -S2ddd ra dec 0.05 </tmp/table

Note that unless you are in the /data/astrocat/SDSS-dr6 subdirectory, you need to give the full path to PhotoPrim.db ; I am typing only the file name for clarity in this document.

A simple table cam be made on the fly, piping it directly into search:

  printf "ra\tdec\n-\t-\n190.\t50.\n" | search PhotoPrim.db -S2ddd ra dec 0.05

This returns a short table with 165 rows in just a fraction of a second. Searching linearly through the 261 GB would take approximately 11 hours, depending on the speed of your network link and machine.

To find all the objects with observed spectra near one or more coordinates, the output of a search such as the above is checked for rows where the column specObjID is non-zero. It happens that the 165 returned objects from the above search contains just a single object with a corresponding spectrum. It can be isolated using row on the above output:

  printf "ra\tdec\n-\t-\n190.\t50.\n"    |
  search PhotoPrim.db -S2ddd ra dec 0.05 |
  row 'specObjID != 0'                   |
  project specObjID

[N.B.: the above is a single command broken across lines for readability]
Here, the project program outputs only the columns given as its arguments, in this case giving

  specObjID
  -
  359951516078964736

Using this result for a query of the SpecPhoto.db database, the command:

  row <SpecPhoto.db 'specObjID == "359951516078964736"' |
  justify DR class subclass ra%.4f dec%.4f z z_err zwarning

Note that the awk statement inside the row command is using a string comparison, not a numeric one. The specObjID and objID columns are 64-bit numbers which are too big for starbase's underlying implementation of awk.

The command returns (where justify projects and formats a subset of the columns to display):

  DR   class   subclass     ra        dec      z         z_err     zwarning
  ---  ------  -----------  --------  -------  --------  --------  --------
  DR3  GALAXY  STARFORMING  189.9365  50.0143  0.083877  0.000009      0x0U

Note that the two databases may have slightly different values of RA and Dec for the object. In the above case, for example, the positions differ radially by 0.076 arc-seconds.

To print the RA and Dec in the conventional sexagismal format instead of degrees, use the Starbase compute command as a filter to convert the RA to hours (i.e., divide by 15), then use the "@" format specifier with justify on the ra and dec columns (and see the column name change at the end):

  row <SpecPhoto.db 'specObjID == "359951516078964736"' |
  compute 'ra /= 15.'                                   |
  justify DR progName class subclass ra%.1@ dec%@ z%.3f z_err%.4f zwarning=zwarn

and the output is:

  DR   class   subclass     ra          dec       z      z_err  zwarn
  ---  ------  -----------  ----------  --------  -----  -----  -----
  DR3  GALAXY  STARFORMING  12:39:44.6  50:00:51  0.084  0.000   0x0U

Getting Images of PhotoPrim Objects

getobjid is a tool to query the SDSS SkyServer for a JPEG image of the area around an object or coordinate, putting the image into a directory designated by the user.

for example, the command

   getobjid -d $HOME -h 200 -w 400 -e GL -q 'GA(15,18)'
            -b 1. -f sdss.jpg -v -o 587731511532060708

(line broken into two or clarity) delivers this image whose objid = 587731511532060708, where there is a label, a grid marking the center, and triangles marking galaxies with magnitudes between 15 and 18 in any of its bands.

For those interested in the details, see the documentation on the getobjid tool, below, and the full set of options is documented at the SDSS help page: http://cas.sdss.org/dr6/en/tools/chart/chart.asp

[Introduction] [Photometric Data] [Spectral Data] [Spectral Line Data] [Tools & Examples]