Differences between revisions 3 and 23 (spanning 20 versions)

NIKA Data Reduction

Contents

NIKA Data Reduction

Getting ready

To start the IDL session for the real time analysis, open a terminal and type:

   $ ssh -X nikaw-13@mrt-lx1
   $ ssh_sami
   > idl

Shortly after a scan is done, the NIKA scientific data, the xml information generated by PAKO (and the AntennaIMBfits shortly after) are written on SAMI and can be processed. There are two types of observation: the “science” scans and the “calibration” scans. There is a specific routine to analyze each type of scan.

Calibration scans

All calibration scans are reduced by the same routine:

IDL> rta_reduce, day, scan_num

Informations or actions to be taken by the observer appear on the plots and explicitely in IDL terminal window:

cont_pointing_cross

The plots show 1D profiles of the azimuth or elevation scans at both wavelenght for the reference detector and a gaussian fit of the source along the timeline. The displayed offsets refer to the reference detectors and are not meant to be passed to PAKO.
The maps show the source, a cross at the location of the reference pixel and gaussian fit of the source
In the terminal window, the observer should follow the instruction (e.g.): B 2mm (MAP) (for PAKO) SET POINTING 3.0 0.7
the pointing reference is the 2mm channel.
If the sky conditions were correct, instructions at 1 and 2mm should be in good agreement.
if a glitch corrupts the maps but the 1d profiles are ok, the user can follow the instruction B 2mm, : (TOI) (for PAKO) SET POINTING 3.0 0.7
In good conditions, all instructions agree to better than 1 arcsec.
If the pointing is far off, the default reference pixels may not see the source and the timelines may look bad. In the terminal window, you may read the ID of the closest kids to the source and pass them to rta_reduce as keywords e.g. rta_reduce, day, scan_num, numdet1=5, numdet2=494

cont_pointing_liss

The routine displays one map for each band, an estimation of the flux where the source is fit (red), an estimation of the flux at the center of the map (green). It also provides a quick estimate of the NEFD and the map RMS for 10arcsec equivalent pixels and the values of Tau derived by NIKA.
The observer should follow instructions given in the terminal, e.g.: B 2mm, : (MAP) (for PAKO) SET POINTING -4.7, 1.4

cont_focus_track

you may change the reference kids as before e.g. rta_reduce, day, scan_num, numdet1=5, numdet2=494
By default, optimize the focus at 1mm

cont_skydip

No action is requested from the observer other than checking that the displayed histograms look reasonable. These results will be used by the NIKA team later on offline.

cont_focus_liss

As soon as the data of a scan of this sequence are on Sami's disk, you may run rta_reduce, day, scan_num
When at least 3 of the 5 scans are done, you may call focus_liss, day, [scan_num1, scan_num2, scan_num3] to have a first guess of the optimal focus
When the 5 scans are done, call focus_liss with the complete list of scans and follow instructions in the terminal window and by default, optimize the focus at 1mm

cont_beammap

Instead of rta_reduce, please call otf_geometry, day, scan_num
The top plots compare the current derived beam maps to the reference ones
The bottom plots display the distance between the current beam parameters and the reference ones in terms of centroids positions, FWHM and absolute calibration.

Science scans

Point source maps

rta_reduce, day, scan_num
If you want to coadd several scans, run combine_scans, scan_list. Note however that to allow to combine scans from different days, scan_list is here defined slightly differently, it is an array of strings of the format YYYYMMDDsScan_num, e.g. '20140127s290'

All maps (On-the-Fly and Lissajous) in Total Power Mode can be reduced by the same script:

In the emacs window, edit run_otf_map.pro and update the scan num and day parameters.
If you’re observing a point source, set diffuse = 0. If you’re observing diffuse emission set diffuse = 1.
Save the file
In the idl session, type .r run_otf_map
Several plot windows might be on top of each other

Polarization maps

All maps (On-the-Fly and Lissajous) in Polarization Mode can be reduced by the same script:

In the emacs window, edit run_otf_polar_maps.pro and update the scan num and day parameters.
Save the file
In the idl session, type .r run_otf_polar_maps
Several plot windows might be on top of each other

Data products

Data products for the 1st NIKA pool

Neel pipeline

The following diagram shows the NIKA data flow from the IRAM 30m telescope to the observers' accounts.

alt text

-  ⇤ ← Revision 3 as of 2014-02-06 11:16:03 → 
  Size: 4238
  Editor: gra-lx17
  Comment:
+   ← Revision 23 as of 2014-02-18 11:16:19 → ⇥
  Size: 5580
  Editor: NikaBolometer
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-##acl NicolasBillot,hermelo,CarstenKramer,SamuelLeclerc:read,write,delete,revert,admin Default
+##acl hermelo,CarstenKramer,NicolasBillot,SamuelLeclerc:read,write,delete,revert,admin Default
 Line 11:
-Line 15:
+Line 16:
-analaysis.
 Line 20:
-   > rt
   > emacs &
-Line 26:
+Line 24:
-Shortly after a scan is done, the NIKA scientific data and the AntennaIMBfits are written on SAMI and can be processed.
+Shortly after a scan is done, the NIKA scientific data, the xml information generated by PAKO (and the AntennaIMBfits shortly after) are written on SAMI and can be processed.
-Line 32:
+Line 30:
-The calibration scans expect actions from the observer and interaction with !PaKo.
+All calibration scans are reduced by the same routine:

'''IDL> rta_reduce, day, scan_num'''

Informations or actions to be taken by the observer appear on the plots and explicitely in IDL terminal window:
-Line 36:
+Line 38:
- * In the emacs window, edit run_pointing.pro and update the day and scan num parameters.
 * update the p2cor and p7cor parameters with the “SET POINTING” values displayed in the PAKO window.
 * Save the file
 * In the idl session, type .r run_pointing
 * Several plot windows might be on top of each other
 * Follow instructions returned by the code in the terminal. By default, follow instructions “B (2mm) : (MAP) (for PAKO and pointing model)”.
+{{attachment:cross.png||width=400}}

 * The plots show 1D profiles of the azimuth or elevation scans at both wavelenght for the reference detector and a gaussian fit of the source along the timeline. The displayed offsets refer to the reference detectors and are not meant to be passed to PAKO.
 * The maps show the source, a cross at the location of the reference pixel and gaussian fit of the source
 * In the terminal window, the observer should follow the instruction (e.g.): B 2mm (MAP) (for PAKO) SET POINTING 3.0 0.7
 * the pointing reference is the 2mm channel.
 * If the sky conditions were correct, instructions at 1 and 2mm should be in good agreement.
 * if a glitch corrupts the maps but the 1d profiles are ok, the user can follow the instruction B 2mm, : (TOI) (for PAKO) SET POINTING 3.0 0.7
 * In good conditions, all instructions agree to better than 1 arcsec.
 * If the pointing is far off, the default reference pixels may not see the source and the timelines may look bad. In the terminal window, you may read the ID of the closest kids to the source and pass them to rta_reduce as keywords e.g. '''rta_reduce, day, scan_num, numdet1=5, numdet2=494'''
-Line 45:
+Line 52:
- * In the emacs window, edit run_pointing_liss.pro and update the day and scan num parameters.
 * Save the file
 * In the idl session, type .r run_pointing_liss
 * Several plot windows might be on top of each other
 * Follow instructions returned by the code in the terminal
+{{attachment:pointing_liss.png||width=400}}

 * The routine displays one map for each band, an estimation of the flux where the source is fit (red), an estimation of the flux at the center of the map (green). It also provides a quick estimate of the NEFD and the map RMS for 10arcsec equivalent pixels and the values of Tau derived by NIKA.
 * The observer should follow instructions given in the terminal, e.g.: B 2mm, : (MAP) (for PAKO) SET POINTING -4.7, 1.4
-Line 53:
+Line 59:
- * In the emacs window, edit run_focus.pro and update the day and scan num parameters.
 * Check the value of focusz in the PAKO display window.
 * Do not change the fooffset values unless you have modified the pako script focusp.pako.
 * Save the file
 * In the idl session, type .r run_focus
 * Several plot windows might be on top of each other
 * Follow instructions returned by the code in the terminal
+{{attachment:focus.png||width=400}}

 * you may change the reference kids as before e.g. '''rta_reduce, day, scan_num, numdet1=5, numdet2=494'''
 * By default, optimize the focus at 1mm

=== cont_skydip ===
{{attachment:skydip_results.png||width=400}}

 * No action is requested from the observer other than checking that the displayed histograms look reasonable. These results will be used by the NIKA team later on offline.
-Line 62:
+Line 70:
+{{attachment:pointing_liss.png||width=400}}
{{attachment:fits.png||width=400}}
-Line 63:
+Line 73:
- * In the emacs window, edit run focus_liss.pro and update the day and scan num parameters. Scan num should be the first of the five scans involved in this analysis.
 * Save the file
 * In the idl session in T2, type .r run_focus_liss
 * Several plot windows might be on top of each other
 * Follow instructions returned by the code in the terminal
+ * As soon as the data of a scan of this sequence are on Sami's disk, you may run '''rta_reduce, day, scan_num'''
 * When at least 3 of the 5 scans are done, you may call '''focus_liss, day, [scan_num1, scan_num2, scan_num3]''' to have a first guess of the optimal focus
 * When the 5 scans are done, call focus_liss with the complete list of scans and follow instructions in the terminal window and by default, optimize the focus at 1mm
-Line 69:
+Line 77:
-=== cont_skydip ===

 * In the emacs window, edit run_skydip.pro and update the day and scan num parameters.
 * Save the file
 * In the idl session, type .r run_skydip
 * Several plot windows might be on top of each other
 * Follow instructions returned by the code in the terminal
-Line 78:
+Line 79:
+{{attachment:otf_geometry.png||width=400}}
-Line 79:
+Line 81:
- * In the emacs window, edit run_otf_geometry.pro and update the day and scan num parameters.
 * Save the file
 * In the idl session in T2, type .r run_otg_geometry
 * Several plot windows might be on top of each other
 * Follow instructions returned by the code in the terminal
+ * Instead of rta_reduce, please call '''otf_geometry, day, scan_num'''
 * The top plots compare the current derived beam maps to the reference ones
 * The bottom plots display the distance between the current beam parameters and the reference ones in terms of centroids positions, FWHM and absolute calibration.
 Line 88:
-The science scans are meant to be optimally processed offline with tailored procedures. The real time software mentioned in this note only aims at giving a quick feedback.
-Line 90:
+Line 89:
-=== Total power maps ===
+=== Point source maps ===
 * rta_reduce, day, scan_num
 * If you want to coadd several scans, run '''combine_scans, scan_list'''. Note however that to allow to combine scans from different days, scan_list is here defined slightly differently, it is an array of strings of the format ''YYYYMMDDsScan_num'', e.g. '20140127s290'
-Line 109:
+Line 111:
+== Data products ==

[[ http://www.iram.fr/wiki/nika2/index.php/NIKARun8productv2 | Data products for the 1st NIKA pool ]]

== Neel pipeline ==

The following diagram shows the NIKA data flow from the IRAM 30m telescope to the observers' accounts.

{{attachment:NIKApipeline.png|alt text|height=600}}
-Line 110:
+Line 123:
-Author: Israel Hermelo (IRAM 30m continuum pool manager)

email: hermelo@iram.es
 
Created: 2014.02.06

Last update: 2014.02.06

Diff for "Continuum/NIKA/DataReduction"