Differences between revisions 27 and 49 (spanning 22 versions)

NIKA dry run organization

Contents

NIKA dry run organization

Go back to the NIKA main wiki.

Goals of the run

Check that gaps in NCS data streams do not occur
Debug the IMBFITS formating so that they are at last produced without errors, and finally allow real time processing.
Test for the first time operations with only IRAM Granada personnel at the Telescope and NIKA Grenoble team using remote control.

Organization Plan

The dominant components of the goals 1 and 2 do not necessitate sensitive KID, so no cool down of the instrument is necessary. However the full instrumental setup for observations must be operational; this includes the electronic chain and real scans of the telescope commanded via PaKo.

Remark: We could test the new network cabling installed in the cable spiral room for the future control of the NIKA2 compressors, keeping the electronics at its storage location and making it run to control the detectors acquisition via the network from there. But we have decided to lift it up in the cabin hence mimicking real observations as much as possible with a warm cryostat, since the problems we had may be related to the network setup we used.

We will debug the IMBFITS step by step. People involved in the NIKA data & IMBFITS making & checking (Albrecht, Juan MP, Xavier, Alain, Robert, Samuel, and possibly Walter and Nicolas P) communicate via this wiki page to establish very clearly the main problems and actions to do to solve them (see tables below), and possibly use the data we currently have to start debugging, so that we are as ready as possible for May 7.

Remark: We need confirmation that the warm KIDs with the observation setup planed for this dry run will produce usable (in math. sense) values, otherwise this might cause somewhere problems.

After this dry run, we will make the status, and determine a date for another short test run (e.g. begin of June ?). Hopefully for this one only the problems that necessitate valid KID data remain (e.g. KID flags). In that case this short run would be cold.

Dates

Telescope time booked for the NIKA Dry run test:

Wednesday May 7, from 11:00 to 15:00.
Monday May 12, from 12:00 to 16:00.

They appear in the official telescope schedule under the project name is T21-13 (please update this wiki if you see a mismatch with the telescope schedule).

Table of IMBFITS problems to solve

The NIKA IMBFITS have had problems since their first creation (2^nd test run, Oct 2010). The latest exhaustive list of problems dates from the run 6 (June 2013) processing and has been tabulated on a dedicated wiki page; most of them are still valid. For reading convenience we gather here only the most important problems that need to be solved with the May 7 dry run.

Remark: Note that problems 4, 5 & 6 are most probably related.

Problem number (origin)	Description of the problem	Status	How to identify & diagnose the problem	How to investigate and possibly solve the problem
1 (NCS)	Some scans miss part(s) of the antenna traces.	Pandemic during pool 1 (run 8, Feb 2014), but marginal after mrt-lx1 kernel update by GP; to be verified as goal #1 of this dry run.	Monitor time vs sample number of antenna traces.	Monitor network & mrt-lx1 activity.
2 (IMBFITS & NIKA)	KID parameter coded in IMBFITS mismatch the data (e.g. numbers of KIDs) - fatal error.	Partially solved: AS's IMBFITS making script should now read the NIKA Kidpar, but we miss a description of the Kidpar by the NIKA team.	Compare at least the number of KIDs in the IMBFITS and of the hardware, e.g. with any FITS viewer.	Need better communication between NIKA team (FXD ?) & IRAM (AS, SL, RZ).
3 (IMBFITS or NCS ?)	Mis-synchronization of subscans in the IMBFITS tables: some subscans are incomplete and partially appear in the following subscan table - fatal error.	This problem appeared first time in IMBFITS of run 6; not solved; it is due to a synchronization problem in NCS and/or IMBFITS creation.	Check subscan length in IMBFITS, e.g. with any FITS viewer.	Check the time used as subscan start and end, delay of NCS messages ...
4 (NIKA)	NIKA FITS do not correspond to the time interval of observation.	Permanent problem since the first try in Oct 2010; not solved.	At least the last subscan missing. Visible with any FITS viewer.	Creation of the NIKA FITS (in IDL ?) not OK.
5 (NIKA)	NIKA tables in IMBFITS empty.	Permanent problem since the first try in Oct 2010; not solved.	Visible with any FITS viewer.	IMBFITS created offline do not show this problem, i.e. NIKA FITS were not available when makeIMBFITS was called in auto mode.
6 (NIKA)	IMBFITS are created many minutes after the observation, e.g. after 2 further pointings !	Permanent problem since the first try in Oct 2010; not solved.	Compare the end time of observations and of creation of IMBFITS; the last takes seconds.	#5 & 6 point to a significant delay in creation of the NIKA FITS.
7 (NIKA)	KIDs flags & types wrong (good/bad KID mis-identified).	Solved after run 8 on NIKA side. Need to be double checked.	Create IMBFITS with updated/correct flags and match KID characteristics with flag & type signification.	Need detailed documentation on the KID params.

Step by step guide for the Dry Run

NIKA preparation

Data acquisition

Neel team:
IRAM computer group:

Observing session

The observations will be done using the same account created for the 1st NIKA pool:

$ ssh -X nikaw-13@mrt-lx1 (ask I. Hermelo for the password)

To open the PaKo version for NIKA type:

$ gopako
$ pakodisplay
$ pakoNIKA

Once PaKo is open, you can run the following script:

PAKO> @ observe_DryRun.pako

The script mimics a typical NIKA session of ~2 hours of duration:

PAKO> SET PROJECT t21
PAKO> SOURCE Uranus
PAKO> @ cont_pointing
PAKO> SET POINTING 5 5
PAKO> @ cont_focusZ -1.8
PAKO> SET FOCUS -1.5
PAKO> SOURCE 3C454 /cat *
PAKO> @ cont_onthefly 10 20 0 -45 
PAKO> @ cont_onthefly 10 20 0 +45 
PAKO> @ cont_lissajous 2 
PAKO> @ cont_onthefly 10 20 45 -45
PAKO> @ cont_onthefly 10 20 45 +45 
PAKO> @ cont_lissajous 4  
PAKO> @ cont_onthefly 10 20 90 -45  
PAKO> @ cont_onthefly 10 20 90 +45  
PAKO> @ cont_lissajous 6
PAKO> @ cont_onthefly 10 20 135 -45  
PAKO> @ cont_onthefly 10 20 135 +45
PAKO> @ cont_lissajous 8

Online pipeline

In order to mimic as much as possible a typical NIKA session the online pipeline must be running. The instructions to start the pipeline can be found here.

Continuum/PoolOrganization/DryRunNIKA (last edited 2014-11-18 10:54:14 by NikaBolometer)

-  ⇤ ← Revision 27 as of 2014-04-14 10:14:33 → 
  Size: 2512
  Editor: NikaBolometer
  Comment:
+   ← Revision 49 as of 2014-04-25 10:01:51 → ⇥
  Size: 7192
  Editor: gra-lx17
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 13:
-. Check that the delays and holes in NCS data streams do not re-appear when NIKA is running.
+. Check that gaps in NCS data streams do not occur
 Line 19:
- The dominant components of the goal do not necessitate sensitive KID, so no cool down of the instrument. However the full setup of the instrument of observations on sky must be operational; this includes the full electronic chain and real scans of the telescope commanded via !PaKo. Though we could keep the electronics the cable spiral room and control the detectors acquisition via the network, which would be a good test of this new network link, we have decided to lift it up in the cabin hence mimicking real observations as much as possible with a warm cryostat. Indeed the problems we had may be related to the network setup.
+The dominant components of the goals 1 and 2 do not necessitate sensitive KID, so no cool down of the instrument is necessary. However the full instrumental setup for observations must be operational; this includes the electronic chain and real scans of the telescope commanded via !PaKo. 
 Remark: We could test the new network cabling installed in the cable spiral room for the future control of the NIKA2 compressors, keeping the electronics at its storage location and making it run to control the detectors acquisition via the network from there. But we have decided to lift it up in the cabin hence mimicking real observations as much as possible with a warm cryostat, since the problems we had may be related to the network setup we used.
-Line 21:
+Line 22:
- We will debug the IMBFITS step by step. People involved in the NIKA data & IMBFITS making & checking (Albrecht, Juan MP, Xavier, Alain, Robert, Samuel, and possibly Walter and Nicolas P) communicate via this wiki page to establish very clearly the main problems and action to do to solve them (see table below), and possibly use the data we currently have to start debugging, so that we are as ready as possible for May 7.
+We will debug the IMBFITS step by step. People involved in the NIKA data & IMBFITS making & checking (Albrecht, Juan MP, Xavier, Alain, Robert, Samuel, and possibly Walter and Nicolas P) communicate via this wiki page to establish very clearly the main problems and actions to do to solve them (see tables below), and possibly use the data we currently have to start debugging, so that we are as ready as possible for May 7. 
 Remark: We need confirmation that the warm KIDs with the observation setup planed for this dry run will produce usable (in math. sense) values, otherwise this might cause somewhere problems.
-Line 23:
+Line 25:
- After this dry run, we will make the status, and determine a date for another short test run (e.g. begin of June ?). Hopefully for this one only the problems that necessitate valid KID data (e.g. KID flags) remain. In that case this short run would be cold.
+After this dry run, we will make the status, and determine a date for another short test run (e.g. begin of June ?). Hopefully for this one only the problems that necessitate valid KID data  remain (e.g. KID flags). In that case this short run would be cold.

== Dates ==

Telescope time booked for the NIKA Dry run test:
 * Wednesday May 7, from 11:00 to 15:00.
 * Monday May 12, from 12:00 to 16:00.
They appear in the [[http://www.iram.fr/IRAMFR/PV/sche/14/s19v1.html|official telescope schedule]] under the project name is T21-13 (please update this wiki if you see a mismatch with the telescope schedule).
-Line 27:
+Line 36:
-== Task list ==
+The NIKA IMBFITS have had problems since their first creation (2^nd^ test run, Oct 2010). The latest exhaustive list of problems dates from the run 6 (June 2013) processing and has been [[http://www.iram.fr/wiki/nika2/index.php/RZdataProc|tabulated on a dedicated wiki page]]; most of them are still valid. For reading convenience we gather here only the most important problems that need to be solved with the May 7 dry run. 
 Remark: Note that problems 4, 5 & 6 are most probably related.
-Line 29:
+Line 39:
-|| I. Hermelo     || Create the organization wiki                              || (./) ||
|| Receiver group ||             ...                                           || {X}  ||
|| NIKA team      ||             ...                                           || {X}  ||
|| Computer group ||             ...                                           || {X}  ||
|| I. Hermelo     || Book rooms at the telescope (no transport on Wed 7th May) || {X}  ||
+|| ''' Problem number (origin) ''' || ''' Description of the problem ''' || ''' Status ''' || ''' How to identify & diagnose the problem ''' || ''' How to investigate and possibly solve the problem '''||
|| 1 (NCS) || Some scans miss part(s) of the antenna traces. || Pandemic during pool 1 (run 8, Feb 2014), but marginal after mrt-lx1 kernel update by GP; to be verified as goal #1 of this dry run.  || Monitor time vs sample number of antenna traces. || Monitor network & mrt-lx1 activity.||
|| 2 (IMBFITS & NIKA) || KID parameter coded in IMBFITS mismatch the data (e.g. numbers of KIDs) - fatal error. || Partially solved: AS's IMBFITS making script should now read the NIKA Kidpar, but we miss a description of the Kidpar by the NIKA team. || Compare at least the number of KIDs in the IMBFITS and of the hardware, e.g. with any FITS viewer. || Need better communication between NIKA team (FXD ?) & IRAM (AS, SL, RZ).||
|| 3 (IMBFITS or NCS ?) || Mis-synchronization of subscans in the IMBFITS tables: some subscans are incomplete and partially appear in the following subscan table - fatal error. || This problem appeared first time in IMBFITS of run 6; not solved; it is due to a synchronization problem in NCS and/or IMBFITS creation. || Check subscan length in IMBFITS, e.g. with any FITS viewer. || Check the time used as subscan start and end, delay of NCS messages ...||
|| 4 (NIKA) || NIKA FITS do not correspond to the time interval of observation. || Permanent problem since the first try in Oct 2010; not solved. || At least the last subscan missing. Visible with any FITS viewer. || Creation of the NIKA FITS (in IDL ?) not OK. ||
|| 5 (NIKA) || NIKA tables in IMBFITS empty. || Permanent problem since the first try in Oct 2010; not solved. || Visible with any FITS viewer. ||IMBFITS created offline do not show this problem, i.e. NIKA FITS were not available when makeIMBFITS was called in auto mode.  ||
|| 6 (NIKA) || IMBFITS are created many minutes after the observation, e.g. after 2 further pointings ! || Permanent problem since the first try in Oct 2010; not solved. || Compare the end time of observations and of creation of IMBFITS; the last takes seconds. || #5 & 6 point to a significant delay in creation of the NIKA FITS. ||
|| 7 (NIKA) || KIDs flags & types wrong (good/bad KID mis-identified). || Solved after run 8 on NIKA side. Need to be double checked. || Create IMBFITS with updated/correct flags and match KID characteristics with flag & type signification. || Need detailed documentation on the KID params.||
 

== Step by step guide for the Dry Run ==

=== NIKA preparation ===


=== Data acquisition ===

 * '''Neel team:'''

 * '''IRAM computer group:'''

=== Observing session ===

The observations will be done using the same account created for the 1st NIKA pool:

{{{
$ ssh -X nikaw-13@mrt-lx1 (ask I. Hermelo for the password)
}}}

To open the PaKo version for NIKA type:

{{{
$ gopako
$ pakodisplay
$ pakoNIKA
}}}

Once PaKo is open, you can run the following script:

{{{
PAKO> @ observe_DryRun.pako
}}}

The script mimics a typical NIKA session of ~2 hours of duration:

{{{

PAKO> SET PROJECT t21
PAKO> SOURCE Uranus
PAKO> @ cont_pointing
PAKO> SET POINTING 5 5
PAKO> @ cont_focusZ -1.8
PAKO> SET FOCUS -1.5
PAKO> SOURCE 3C454 /cat *
PAKO> @ cont_onthefly 10 20 0 -45 
PAKO> @ cont_onthefly 10 20 0 +45 
PAKO> @ cont_lissajous 2 
PAKO> @ cont_onthefly 10 20 45 -45
PAKO> @ cont_onthefly 10 20 45 +45 
PAKO> @ cont_lissajous 4  
PAKO> @ cont_onthefly 10 20 90 -45  
PAKO> @ cont_onthefly 10 20 90 +45  
PAKO> @ cont_lissajous 6
PAKO> @ cont_onthefly 10 20 135 -45  
PAKO> @ cont_onthefly 10 20 135 +45
PAKO> @ cont_lissajous 8

}}}

=== Online pipeline ===

In order to mimic as much as possible a typical NIKA session the online pipeline must be running. The instructions to start the pipeline can be found [[ http://www.iram.es/IRAMES/mainWiki/Continuum/NIKA/DataReduction#Online_data_reduction | here ]].

Diff for "Continuum/PoolOrganization/DryRunNIKA"