Differences between revisions 1 and 27 (spanning 26 versions)

NIKA Data Reduction

Contents

NIKA Data Reduction

Getting ready

To start the IDL session for the real time analysis, open a terminal and type:

   $ ssh -X nikaw-13@mrt-lx1
   $ ssh_sami
   > idl

Shortly after a scan is done, the NIKA scientific data, the xml information generated by PAKO (and the AntennaIMBfits shortly after) are written on SAMI and can be processed. There are two types of observation: the “science” scans and the “calibration” scans. There is a specific routine to analyze each type of scan.

Calibration scans

All calibration scans are reduced by the same routine:

IDL> rta_reduce, day, scan_num

day is in the format YYYYMMDD, e.g. 20140218, scan_num is a simple integer.

Informations or actions to be taken by the observer appear on the plots and explicitely in IDL terminal window:

cont_pointing_cross

The plots show 1D profiles of the azimuth or elevation scans at both wavelenght for the reference detector and a gaussian fit of the source along the timeline. The displayed offsets refer to the reference detectors and are not meant to be passed to PAKO.
The maps show the source, a cross at the location of the reference pixel and gaussian fit of the source
In the terminal window, the observer should follow the instruction (e.g.): B 2mm (MAP) (for PAKO) SET POINTING 3.0 0.7
the pointing reference is the 2mm channel.
If the sky conditions were correct, instructions at 1 and 2mm should be in good agreement.
if a glitch corrupts the maps but the 1d profiles are ok, the user can follow the instruction B 2mm, : (TOI) (for PAKO) SET POINTING 3.0 0.7
In good conditions, all instructions agree to better than 1 arcsec.
If the pointing is far off, the default reference pixels may not see the source and the timelines may look bad. In the terminal window, you may read the ID of the closest kids to the source and pass them to rta_reduce as keywords e.g. rta_reduce, day, scan_num, numdet1=5, numdet2=494

cont_pointing_liss

The routine displays one map for each band, an estimation of the flux where the source is fit (red), an estimation of the flux at the center of the map (green). It also provides a quick estimate of the NEFD and the map RMS for 10arcsec equivalent pixels and the values of Tau derived by NIKA.
The observer should follow instructions given in the terminal, e.g.: B 2mm, : (MAP) (for PAKO) SET POINTING -4.7, 1.4

cont_focus_track

you may change the reference kids as before e.g. rta_reduce, day, scan_num, numdet1=5, numdet2=494
By default, optimize the focus at 1mm

cont_skydip

No action is requested from the observer other than checking that the displayed histograms look reasonable. These results will be used by the NIKA team later on offline.

cont_focus_liss

As soon as the data of a scan of this sequence are on Sami's disk, you may run rta_reduce, day, scan_num
When at least 3 of the 5 scans are done, you may call focus_liss, day, [scan_num1, scan_num2, scan_num3] to have a first guess of the optimal focus
When the 5 scans are done, call focus_liss with the complete list of scans and follow instructions in the terminal window and by default, optimize the focus at 1mm

cont_beammap

Instead of rta_reduce, please call otf_geometry, day, scan_num
The top plots compare the current derived beam maps to the reference ones
The bottom plots display the distance between the current beam parameters and the reference ones in terms of centroids positions, FWHM and absolute calibration.

Science scans

Point source maps

rta_reduce, day, scan_num
If you want to coadd several scans, run combine_scans, scan_list. Note however that to allow to combine scans from different days, scan_list is here defined slightly differently, it is an array of strings of the format YYYYMMDDsScan_num, e.g. '20140127s290'

Diffuse emission

rta_reduce, day, scan_num, /diffuse
to combine several scans, use combine_scans in the same way as for point source maps

Data products

Data products for the 1st NIKA pool

Neel pipeline

The following diagram shows the NIKA data flow from the IRAM 30m telescope to the observers' accounts.

alt text

-  ⇤ ← Revision 1 as of 2014-02-06 10:57:32 → 
  Size: 3390
  Editor: gra-lx17
  Comment:
+   ← Revision 27 as of 2014-02-19 16:27:03 → ⇥
  Size: 5071
  Editor: NikaBolometer
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-##acl NicolasBillot,hermelo,CarstenKramer,SamuelLeclerc:read,write,delete,revert,admin Default
+##acl hermelo,CarstenKramer,NicolasBillot,SamuelLeclerc:read,write,delete,revert,admin Default
 Line 11:
-Line 15:
+Line 16:
-analaysis.
 Line 20:
-   > rt
   > emacs &
-Line 26:
+Line 24:
-Shortly after a scan is done, the NIKA scientific data and the AntennaIMBfits are written on SAMI and can be processed.
+Shortly after a scan is done, the NIKA scientific data, the xml information generated by PAKO (and the AntennaIMBfits shortly after) are written on SAMI and can be processed.
-Line 32:
+Line 30:
-The calibration scans expect actions from the observer and interaction with !PaKo.
+All calibration scans are reduced by the same routine:
-Line 34:
+Line 32:
+'''IDL> rta_reduce, day, scan_num'''

'''day is in the format YYYYMMDD, e.g. 20140218, scan_num is a simple integer.'''

Informations or actions to be taken by the observer appear on the plots and explicitely in IDL terminal window:

----
-Line 36:
+Line 41:
- * In the emacs window, edit run pointing.pro and update the day and scan num parameters.
 * update the p2cor and p7cor parameters with the “SET POINTING” values displayed in the PAKO window.
 * Save the file
 * In the idl session in T2, type .r run pointing
 * Several plot windows might be on top of eachother
 * Follow instructions returned by the code in T2. By default, follow instructions “B (2mm) : (MAP) (for PAKO and pointing model)”.
+{{attachment:cross.png||width=400}}
 Line 43:
+ * The plots show 1D profiles of the azimuth or elevation scans at both wavelenght for the reference detector and a gaussian fit of the source along the timeline. The displayed offsets refer to the reference detectors and are not meant to be passed to PAKO.
 * The maps show the source, a cross at the location of the reference pixel and gaussian fit of the source
 * In the terminal window, the observer should follow the instruction (e.g.): B 2mm (MAP) (for PAKO) SET POINTING 3.0 0.7
 * the pointing reference is the 2mm channel.
 * If the sky conditions were correct, instructions at 1 and 2mm should be in good agreement.
 * if a glitch corrupts the maps but the 1d profiles are ok, the user can follow the instruction B 2mm, : (TOI) (for PAKO) SET POINTING 3.0 0.7
 * In good conditions, all instructions agree to better than 1 arcsec.
 * If the pointing is far off, the default reference pixels may not see the source and the timelines may look bad. In the terminal window, you may read the ID of the closest kids to the source and pass them to rta_reduce as keywords e.g. '''rta_reduce, day, scan_num, numdet1=5, numdet2=494'''

----
-Line 45:
+Line 55:
- * In the emacs window, edit run pointing liss.pro and update the day and scan num parameters.
 * Save the file
 * In the idl session in T2, type .r run pointing liss
 * Several plot windows might be on top of eachother
 * Follow instructions returned by the code in T2
+{{attachment:pointing_liss.png||width=400}}
-Line 51:
+Line 57:
+ * The routine displays one map for each band, an estimation of the flux where the source is fit (red), an estimation of the flux at the center of the map (green). It also provides a quick estimate of the NEFD and the map RMS for 10arcsec equivalent pixels and the values of Tau derived by NIKA.
 * The observer should follow instructions given in the terminal, e.g.: B 2mm, : (MAP) (for PAKO) SET POINTING -4.7, 1.4

----
-Line 53:
+Line 63:
- * In the emacs window, edit run focus.pro and update the day and scan num parameters.
 * Check the value of focusz in the PAKO display window.
 * Do not change the fooffset values unless you have modified the pako script focusp.pako.
 * Save the file
 * In the idl session in T2, type .r run focus
 * Several plot windows might be on top of eachother
 * Follow instructions returned by the code in T2
+{{attachment:focus.png||width=400}}
-Line 61:
+Line 65:
+ * you may change the reference kids as before e.g. '''rta_reduce, day, scan_num, numdet1=5, numdet2=494'''
 * By default, optimize the focus at 1mm

----
=== cont_skydip ===
{{attachment:skydip_results.png||width=400}}

 * No action is requested from the observer other than checking that the displayed histograms look reasonable. These results will be used by the NIKA team later on offline.

----
-Line 62:
+Line 76:
+{{attachment:pointing_liss.png||width=400}}
{{attachment:fits.png||width=400}}
-Line 63:
+Line 79:
- * In the emacs window, edit run focus liss.pro and update the day and scan num parameters. Scan num should be the first of the five scans involved in this analysis.
 * Save the file
 * In the idl session in T2, type .r run focus liss
 * Several plot windows might be on top of eachother
 * Follow instructions returned by the code in T2
+ * As soon as the data of a scan of this sequence are on Sami's disk, you may run '''rta_reduce, day, scan_num'''
 * When at least 3 of the 5 scans are done, you may call '''focus_liss, day, [scan_num1, scan_num2, scan_num3]''' to have a first guess of the optimal focus
 * When the 5 scans are done, call focus_liss with the complete list of scans and follow instructions in the terminal window and by default, optimize the focus at 1mm
-Line 69:
+Line 83:
-=== cont_skydip ===
+----
=== cont_beammap ===
{{attachment:otf_geometry.png||width=400}}
-Line 71:
+Line 87:
- * In the emacs window, edit run skydip.pro and update the day and scan num parameters.
 * Save the file
 * In the idl session in T2, type .r run skydip
 * Several plot windows might be on top of eachother
 * Follow instructions returned by the code in T2
+ * Instead of rta_reduce, please call '''otf_geometry, day, scan_num'''
 * The top plots compare the current derived beam maps to the reference ones
 * The bottom plots display the distance between the current beam parameters and the reference ones in terms of centroids positions, FWHM and absolute calibration.
-Line 77:
+Line 91:
-=== cont_beammap ===

 * In the emacs window, edit run otf geometry.pro and update the day and scan num parameters.
 * Save the file
 * In the idl session in T2, type .r run otg geometry
 * Several plot windows might be on top of eachother
 * Follow instructions returned by the code in T2
+----
----
== Science scans ==
-Line 86:
+Line 96:
-== Science scans ==
+=== Point source maps ===
 * rta_reduce, day, scan_num
 * If you want to coadd several scans, run '''combine_scans, scan_list'''. Note however that to allow to combine scans from different days, scan_list is here defined slightly differently, it is an array of strings of the format ''YYYYMMDDsScan_num'', e.g. '20140127s290'
-Line 88:
+Line 100:
-The science scans are meant to be optimally processed offline with tailored procedures. The real time software mentioned in this note only aims at giving a quick feedback
+== Diffuse emission ==
 * rta_reduce, day, scan_num, /diffuse
 * to combine several scans, use combine_scans in the same way as for point source maps

----
----
== Data products ==

{{attachment:NikaDataProductsV1.pdf  | Data products for the 1st NIKA pool }}

== Neel pipeline ==

The following diagram shows the NIKA data flow from the IRAM 30m telescope to the observers' accounts.

{{attachment:NIKApipeline.png|alt text|height=600}}
-Line 91:
+Line 118:
-Author: Israel Hermelo (IRAM 30m continuum pool manager)

email: hermelo@iram.es
 
Created: 2014.02.06

Last update: 2014.02.06

Diff for "Continuum/NIKA/DataReduction"