NIKA Data Reduction

The following diagram shows the NIKA data flow from the IRAM 30m telescope to the observers' accounts.

Go to the NIKA Main Wiki.

Offline data reduction

Data products v0

The version v0 of the NIKA first open pool products has been made by the NIKA team by the end of the pool (Feb 28,2014) and it was distributed by IRAM. The products are described here.

For each project, there are fits files containing the maps of each scan and combined scans of each object. The data processing has been oriented towards point-source extraction or diffuse emission depending on the aim of your project.

There are also logbook files and illustrative pdf files. On the main directory you will find:

• logfile_all_scans.csv: is a comma separated file ready for your favorite spreadsheet handler (libreoffice or excel). It gives the summary information of all the scans that could be used including opacity measurements, sky noise in Jy (per band of frequency from 1mHz to 10 Hz), camera noise NEFD (in mJy.s^1/2) and finally a flux in mJy of a point-source at the center of the map (fixed central position, fixed FWHM, Gaussian fit) with the error at both wavelengths.

Then for each object, there is a directory that contains:

• the fits files.

• flux_source.csv: a comma separated file giving the flux as measured in the scan maps and the combined scans maps.

• Figures: many pdf files were generated out of the fits files to get a quick look at them. Units in Jy.

To retrieve the data, please type:

scp  XXX-YY@enter.iram.es:~/observationData/nika/v0.tar.gz  /path/in/your/computer

where XXX-YY is your project number. If you get an error message, enter the following command:

ssh-keygen -R enter.iram.es

and then try to retrieve the data again.

CAVEAT: At this stage, the data offline products are not of scientific quality. The selection of scans was unfortunately rendered difficult because of an unexpected problem with the pointing data. We hope to solve that problem with post-processing steps that we did not have time to implement. Instead, scans were manually selected with some mistakes still present. The photometry is probably correct at the 20% level at 1mm and 10% level at 2mm for scans of good quality. Don't shoot. We will prepare the next version in the coming weeks. Contact your NIKA friend of project to give us your feedback.

The NIKA team

Data products v1

This version V1 of the NIKA first open pool products has been made by the NIKA team by the end of May 2014. It is distributed by IRAM. It follows the quick release done at the end of the pool (v0). The products are described here

For each project, there are fits files containing the maps of each scan and combined scans of each object. The data processing has been oriented towards point-source extraction or diffuse emission depending on the aim of your project. The main processing has improved by including more scans and dealing with internal problems more effectively.

There are also logbook files and illustrative pdf files. On the main directory you will find:

- logfile_all_scans.csv : is a comma separated file ready for your favorite spreadsheet handler (libreoffice or excel) It gives the summary information of all the scans that could be used including opacity measurements, sky noise in Jy (per band of frequency from 1mHz to 10 Hz), camera noise NEFD (in mJy.s^1/2) and finally a flux in mJy of a point-source at the center of the map (fixed central position, fixed FWHM, Gaussian fit) with the error at both wavelengths. For pointing sources the position is allowed to vary.

Then for each object, there is a directory that contains:

- The fits files (RaDec2000 projection for target sources, AzEl projection for pointing sources and Planets)

- flux_source.csv : a comma separated file giving the flux as measured in the scan maps and the combined scans maps.

- Figures: many pdf files were generated out of the fits files to get a quick look at them. Units in Jy/beam unless otherwise stated.

The calibration was done assuming a Gaussian main beam of 12.5 and 18.5 arcseconds (FWHM). The primary calibrator is Uranus with fluxes of 36.4 and 15.3 Jy.

For Lissajous scans (V1Liss directories), the pipeline introduces a filtering effect for point sources of about 0.70 and 0.56 (at 1 and 2mm). This effect has been corrected in the .csv files but not in the maps. Thus fluxes in Jy in the .csv files should be correct.

For you to assess the filtering effect, we have rerun the pipeline with a much less agressive decorrelation that should not affect the fluxes. For that just look at V1cLiss directories made for calibration sources only.

For OTF maps (directories V1otf): the main beam to full beam correction is by 1.56+-0.10 at 1mm and 1.35+-0.10 at 2mm. It has not been applied to the maps.

At this stage, the offline products can be used for a first scientific analysis as most of the scans have been included. The point-source photometry is probably correct at the 15% level at 1mm and 10% level at 2mm.

To retrieve the data, please type:

scp  XXX-YY@enter.iram.es:~/observationData/nika/v1.tar.gz  /path/in/your/computer

where XXX-YY is your project number.

The NIKA team

Data products v2

We hope to deliver a final V2 version of the first NIKA open pool data products in the Summer including our best strategy of systematic removals. Your feedback is essential to improve the quality of the v1 data and help the NIKA team to develop the final version.

The NIKA team

Online data reduction

To start the IDL session for the real time analysis, open a terminal and type:

   $ ssh -X nikaw-13@mrt-lx1
   $ ssh_sami
   > idl 

Shortly after a scan is done, the NIKA scientific data, the xml information generated by PAKO (and the AntennaIMBfits shortly after) are written on SAMI and can be processed. There are two types of observation: the “science” scans and the “calibration” scans. There is a specific routine to analyze each type of scan.

Calibration scans

All calibration scans are reduced by the same routine:

IDL> rta_reduce, day, scan_num

day is in the format YYYYMMDD, e.g. 20140218, scan_num is a simple integer.

Informations or actions to be taken by the observer appear on the plots and explicitely in IDL terminal window:

cont_pointing_cross

• The plots show 1D profiles of the azimuth or elevation scans at both wavelenght for the reference detector and a gaussian fit of the source along the timeline. The displayed offsets refer to the reference detectors and are not meant to be passed to PAKO.
• The maps show the source, a cross at the location of the reference pixel and gaussian fit of the source
• In the terminal window, the observer should follow the instruction (e.g.): B 2mm (MAP) (for PAKO) SET POINTING 3.0 0.7
• the pointing reference is the 2mm channel.
• If the sky conditions were correct, instructions at 1 and 2mm should be in good agreement.
• if a glitch corrupts the maps but the 1d profiles are ok, the user can follow the instruction B 2mm, : (TOI) (for PAKO) SET POINTING 3.0 0.7
• In good conditions, all instructions agree to better than 1 arcsec.

• If the pointing is far off, the default reference pixels may not see the source and the timelines may look bad. In the terminal window, you may read the ID of the closest kids to the source and pass them to rta_reduce as keywords e.g. rta_reduce, day, scan_num, numdet1=5, numdet2=494

cont_pointing_liss

• The routine displays one map for each band, an estimation of the flux where the source is fit (red), an estimation of the flux at the center of the map (green). It also provides a quick estimate of the NEFD and the map RMS for 10arcsec equivalent pixels and the values of Tau derived by NIKA.
• The observer should follow instructions given in the terminal, e.g.: B 2mm, : (MAP) (for PAKO) SET POINTING -4.7, 1.4

cont_focus_track

• you may change the reference kids as before e.g. rta_reduce, day, scan_num, numdet1=5, numdet2=494

• By default, optimize the focus at 1mm

cont_skydip

• No action is requested from the observer other than checking that the displayed histograms look reasonable. These results will be used by the NIKA team later on offline.

cont_focus_liss

• As soon as the data of a scan of this sequence are on Sami's disk, you may run rta_reduce, day, scan_num

• When at least 3 of the 5 scans are done, you may call focus_liss, day, [scan_num1, scan_num2, scan_num3] to have a first guess of the optimal focus

• When the 5 scans are done, call focus_liss with the complete list of scans and follow instructions in the terminal window and by default, optimize the focus at 1mm

cont_beammap

• Instead of rta_reduce, please call otf_geometry, day, scan_num

• The top plots compare the current derived beam maps to the reference ones
• The bottom plots display the distance between the current beam parameters and the reference ones in terms of centroids positions, FWHM and absolute calibration.

Science scans

Point source maps

• rta_reduce, day, scan_num

• If you want to coadd several scans, run combine_scans, scan_list. Note however that to allow to combine scans from different days, scan_list is here defined slightly differently, it is an array of strings of the format YYYYMMDDsScan_num, e.g. '20140127s290'

Diffuse emission

• rta_reduce, day, scan_num, /diffuse
• to combine several scans, use combine_scans in the same way as for point source maps