IntoductionToThePool

Back to Pool page: PoolObserving

The Bolometer Pool Observation Database System (ODS) at the IRAM 30m Telescope

Pool Observations

Most cutting edge scientific programs carried out at the IRAM observatories require excellent weather conditions. While the service observing mode at the Plateau de Bure interferometer allows a flexible scheduling of demanding programs, fixed scheduled projects at the 30m telescope often suffer from limitations due to inadequate weather conditions.

To optimize the observing efficiency given the often quickly changing weather conditions, IRAM offers the pool observing mode at the 30m telescope. In this mode all bolometer and some high demanding spectroscopy programs share the allocated observing time together with some less weather demanding spectroscopy programs.

Projects are observed according to the current weather conditions, their ranking from the program committee and the source visibility. This way the chance of all demanding projects have drastically improved: the average success rate of A ranked projects requiring good or excellent weather conditions in last winter pool was 80%, compared to 20% of good weather conditions during the allocated pool observing time (see Fig below).

The pool observations are conducted mostly by the astronomers who's programs are included in the pool. They are aided by the IRAM astronomers and telescope operators. There are typically three guest astronomers, one IRAM astronomer, and the telescope operators at the telescope. Given the flexible use of different instruments and the large variety of scientific programs, pooled observations are part of IRAM's training efforts for students.

Pool observations were first tested at the IRAM 30m telescope during the winter 2000/2001, and have now expanded to 12 weeks in the winter semester, and about 4 weeks in the summer. The winter 2003/4 pool includes almost 50 different programs with more than 500 different targets. To manage such a pool requires an efficient organizational structure, so that at any time the status of a program, target priorities and weather/technical requirements can be assessed. This motivated the observational database system described here.

The PHP/MySQL software (http://pool.iram.es) system was first conceived by Alexandre Beelen (IAS Paris) and Frank Bertoldi (MPIfR Bonn) in December 2001, and subsequently being developed by Axel Weiss. Stephane Leon has been the pool coordinator for many years. In October/November 2008, Carsten Kramer is the interim bolo pool manager. All data reduction tools have been developed and are maintained by Robert Zylka (IRAM).

Technical Summary

The IRAM pool observation database system (ODS) is based on PHP scripts executed from an apache web-server, which access a MySQL database. The database contains detailed informations on all projects including technical and metrological requirements. Observational information is read from the fits data headers of each scan. This observation information is easily associated with observations projects, and thereby allows bookkeeping, planning, data quality control, and in connection with an external data reduction software an automated pipeline data reduction. The system also permits data archiving and the easy access/download of data for the users. The users interactively enter and modify their target lists, observation instructions, and they are able to check the state of the project and the data quality.

The software used in the development of ODS are part of all Linux distributions. Its main constituents are PHP and MySQL. PHP (recursive acronym for "PHP: Hypertext Preprocessor") is a widely-used Open Source general-purpose C-based scripting language that is especially suited for Web development and can be embedded into HTML (see e.g. Welling & Thomson 2003). The PHP package contains functions which allow to retrieve and store information in a MySQL database. MySQL is the most popular Open Source SQL (Structured Query Language) database management system. It provides an easy way to store and sort large amounts of information and to give limited access to different users on the MySQL server (e.g. DuBois 2000). MySQL in conjunction with PHP is becoming the most commonly used platform for web-based data access and processing applications. The database system at the IRAM 30m currently runs on PHP version 4.1 and MySQL 5.0 on Debian Linux distribution version 4.0.

The main body of ODS are a collection of PHP scripts (the PHP engine) which perform different kinds of operations. Information can be received and/or stored in the MySQL database (e.g. the amount of observing time spent on a specific project). Other PHP scripts generate files associated with the observations such as the observing log-files, the 30m source catalogs or Xephem files for astronomical visualization. PHP also allows to call external programs. This way the PHP engine can parse a list of scans to an external data reduction software, e.g. the IRAM continuum data processing software MOPSIC, and store data information such as the achieved rms in the MySQL database. All PHP scripts are executed by an Apache web-server. Thereby all information can be accessed through the Internet. Access is restricted by PHP and by MySQL security applications as well as the Apache web-server configuration. This allows to setup different user accounts in ODS with limited access to specific information.

ODS User Accounts

ODS currently knows five different kinds of users. The two user types which have general purpose administration privileges are the superuser and the scheduler account. Both accounts allow to enter e.g. new projects to the pool, update the allocated observing time and to set the observing priority within the pool. These settings reflect the recommendations of the IRAM program committee. The superuser account also allows to manipulate entries in ODS such as wrong or misleading entries in the observing instructions. The IRAM user account provides an overview on all projects and there technical specifications. The observer account allows to access the status of each project and details on how to carry out the observations. It also provides tools for a detailed documentation of individual observations. The project account itself provides access to a specific project only. It allows to enter/modify the source list and the observing instructions including technical and meteorological requirements. It also allows to download the project data including the log-files and a detailed listing of all observations. All accounts can be accessed remotely by Internet. This structure allows to include pilot projects and directors time proposals into the pool on short

Project Setup

Each observational project is identified by a project number (e.g. 123_08) which corresponds to a password-protected account on the ODS. The PIs enter the project information (target names, coordinates, priorities within the project, desired observing time, minimum S/N, final rms noise level, weather requirements, observation setup, suggestions for pointing sources, etc), through Internet web forms. This is typically done several weeks before the pool starts. All information can be updated at any time.

The program PIs may also provide an ascii source list to the pool administrator in cases where entering them via the web forms would be too involved. A upload of a complete source catalog via ODS is in preparation. The 30m pool observing catalog is generated by ODS from the source entries. For bolometer projects no further input is required. SIS projects participating in the pool additionally need to sent observing macros which contain details on the receiver configuration, tuning frequencies and target observing routines. Templates of these procedures can be accessed via the pool observing web page (PoolObserving).

From the entries ODS generates an instruction form to aid the observer at the telescope with the execution of the program. The instruction shall be clear enough that even an inexperienced observer can easily follow the steps.

Observation Data Input

The database is fed continuously in that it reads the fits data file headers. This data include the observing time (LST, UT), type of observation, object name, telescope parameter such as pointing, focus, wobbler, reference channel, gain settings, azimuth, elevation. Additional information on the data quality and current weather conditions is computed online using the MOPSIC software package. Among those are the skynoise and results of calibration observations (calibrator fluxes, array parameters). All these parameters are stored in the MySQL database and are instantaneously accessible by the observers and the PIs. Thus each scan read by the ODS is also directly accessible for download and/or pipeline reduction by the users.

Observing support using ODS

Pool of projects

The local observer at the telescope views a list of all projects in the current observing pool. This form is constantly updated, showing the time spent on a given program. The projects are ranked in priority according to the grades received from the TAC, the fraction of time used, or an timely urgency (e.g. gamma ray bursts, comets).

The columns show the program ID, the PI name, the priority (0-6, may vary in time), the program title, the type of observations (SIS or bolometer, on-off or mapping), and the weather requirements (cloud covering fraction, zenith opacity, skynoise level for bolometers, mm water for SIS). Bolometer programs are colored green, SIS yellow, programs with priority 0 or programs already finished (DONE) are colored red. By clicking on the program number a page appears which lists all the sources of the program, each with a list of scans observed, results of the online reduction, and on top, short instructions on how and when to execute the program. The detailed observing instructions provided by the PI can be accessed by pressing the ObsForm link at the upper right of the project page.

LST distribution

The observer has various means to find out which project and which sources is observable at the current local siderial time (LST) and with the current weather conditions. A quick way is to select projects according to weather conditions and plot the LST distribution (visibility) of the sources, where the blue lines indicate sources with a high internal priority, the red lines with a lower priority. The green bar indicates the current LST. Only those projects are shown which can be observed in the preselected weather conditions (as qualified by the opacity or the water vapor, the skynoise, the cloud coverage). Sources which have been completed are not taken into account.

Data reduction

ONOFF-Observations can be reduced from within the DB. Go to "pool", click onto a project, click onto a source name, and a window opens, allowing to select MOPSIC reduction parameters like skynoise yes/no, expected peak flux below 150mJy yes/no, start and end scan. Images of the map and the rms are produced.

Maps have to be reduced offline, using MOPSIC.

XEphem

Once the observer has selected the projects which will be observed in the next few hours she/he can take advantage of the powerful astronomical visualization software Xephem (http://www.clearskyinstitute.com/xephem/xephem.html) running at the observing console. The ODS generates on request a Xephem file for each project. It will display the precise location of each project source on the sky (green symbols). In addition all calibrators (red symbols) and pointing sources (blue symbols) are displayed. This way the nearest location of calibrators and pointing sources can be accessed easily and the observers can avoid long slews across the sky. XEphem also plots the position of the telescope.