Daily Reports
Observers on-site: Alice Nucara, Antoine Zakardjian, Juan Macias, Tuan-Anh Duong, Wonju Kim, and Ste B. (AoD)
Operators: Frederic and Sergio
Jan 7th, 2025, Tuesday
Alice, Antoine, Juan, Tuan Anh, Wonju, Ste B. and maaaany others arrived at the telescope.
NIKA2 work fine, the HWP correct rotation has been checked. The new AoD, Stefano, is trained by Angel and Frederic with explanations about AoD specific tasks and safety measures. After maintenance and the AoD training, we start the operations at ~16:20 local time.
16:00-22:00 LT: Wonju + Alice, Stefano. We start with some training of the new NIKA2 observers, explaining in what a typical NIKA2 session consists and clarifying some specific features for NIKA2-pol. Then we proceed with the following observations:
- we start with polarimetry
- calib_1scan on MWC349, NGC7027, CRL2688
- 135-24: one repetition on IRAS21078
- when observing polarimetry, we monitor the instrumental polarization by taking roughly every hour one calib_1scan of either an unpolarized point-like source (Uranus) or a known QSO, depending on availability.
- skydip in between (scan 20250107s25)
- one gummy seal of one lower petal falls off at ~18:30 LT (not our fault, we swear, we're not trying to break the telescope ;)
- 166-24: four repetitions of CasA
22:00-04:00 LT: Antoine & Tuan-Anh. We switch to total power (bad kids 27 4 32) and observe:
- 188-24: two repetitions of A0520
- total power beammap on Uranus, 99 sub-scans (scan 20250107s76)
- 113-24: one repetition on orionb (tau > 0.3)
- 191-24: one repetition on MS1054
Overall very good stability conditions but tau ~ 0.3 - 0.35
Jan 8th, 2025, Wednesday
04:00-10:00 LT: Juan. Low tau (between 0.2 and 0.3) but unstable weather (+-5 to +-20 Jy)
- 191-24: two repetitions on MS1054
- 189-24: one repetition on CTRS27
- we switch to polarimetry ~1h before the end of the shift
10:00-16:00 LT: Alice + Juan, Stefano. The sky is unstable (+/- 5 Jy/b peak to peak @2mm, evaluated on Az sub-scans of pointing scans). The wind is rising (~15 m/s gusts @10:30 LT). Uhm. Evolution of the shift:
- we start in polarimetry mode. Trying to point and focus on 3C273 and 3C286, in preparation of a possible polarized beam map. Hopeless. In the direction of 3C286 the wind influences the tracking less than in that of 3C273, but still too bad. We take a short break.
- we move to the direction of W43 (132-24) and 104316 (134-24) to test the wind, pointing/focus.
- at 10:00 UT we have to stop and park the telescope, because the wind reaches and surpasses 18 m/s three times in a row. Sigh
- at 10:47 UT we start again, but the wind is still important and does not allow us to proceed with science
- we wait a bit, because switching to EMIR at the moment is not yet an option: we don't have EMIR bkp projects for this LST right now (Brisa asked the PIs but only 143-24 has responded so far and it's not covering the morning). Beeeeeeep (wind alarm forever).
- @13:30 LT, taken by desperation, we test the external calibrator: we perform scans in Az, with 1 subscan only: 116 and 117 (w/ HWP, w/o ext. cal.); 118 and 119 (w/ HWP, w/ ext. cal.); 120 and 121 (w/o HWP, w/ ext. cal.); 122 and 123 (w/o HWP; w/o ext. cal.).
- we keep testing the reaction of the telescope to the wind and the tracking deviations until the end of the shift, without success.
16:00-22:00 LT: Wonju + Alice. The sky and wind conditions are calming down and we are able to start again the proper NIKA2-Pol observations at roughly 17:00 LT:
- calib_1scan on MWC349, NGC7027, CRL2688
- 135-24: 2.5 repetitions on source IRAS21078 (-> the first 7 reps of this target are completed!)
- lots of homology errors at ~19:30-20:00
- in between we monitor the instrumental polarization by taking calib_1scan of a known QSO (Uranus will be later).
- 166-24 one repetition of CasA
- at 21:50 we have to stop, because we are now in the fog/clouds. The telescope is parked.
22:00-04:00 LT: Antoine & Tuan-Anh. Fog, clouds, rain all night long.
It's a bit windy today... and you know what happens at Pico with strong wind...
Jan 9th, 2025, Thursday
04:00-10:00 LT: Juan. Fog, clouds, rain all night long. Operations are re-started at ~8:30 UT. After de-icing the telescope is finally relaxed at ~9:30 UT (see next shift).
10:00-16:00 LT: Alice + Juan. We start proper NIKA2-Pol observations at ~ 9:30 UT, after recovering from the morning de-icing. Progress of the shift:
- 134-24 one repetition on 104316
- 134-24 two repetitions on 126348 (completed)
- 132-24 two repetition on W43 (wrong coordinates! because Dec was missing a zero (0) in the seconds value). [AB: my fault, sorry. Coordinates in all project folders were rechecked: ok]
- 115-24 one repetition on G28.34
- in between we monitor the instrumental polarization by taking calib_1scan of a known QSO (Uranus will be later).
- IMPORTANT NOTE: all four sources above are close to each other and close to the Sun.
16:00-22:00 LT: Wonju. The NIKA-Pol observations continue:
- 166-24: 2,5 repetitions of CasA (many cancellations because of spindle and homology problems)
- in between we monitor the instrumental polarization by taking calib_1scan of a known QSO (Uranus will be later).
- 135-24: one repetition of NGC7538 IRS1
22:00-04:00: Antoine & Tuan Anh. We switch to total power at the beginning of the shift. Progress:
- 113-24 two maps of Orion-B while waiting for tau to decrease
- tau(225) is ~ 0.26 -> proceed with 188-24 (SZ clusters)
- 188-24 one repetition of A0520
- lots of homology and spindle errors
- Tau starts to increase ==> Switch to Orion B for one repetition
- tau < 0.3 and decreasing ==> Observing MS1054 for one repetition (rep 4)
- Lots of spindle and homology errors, had to cancel scans several times
- tau tau 0.2~0.25 now, second repetition on MS1054 (rep 5)
- movie night
NOTE by SB @22:50 UT: today we had a very large number of spindle and homology errors, much more than the past days. At some point of the afternoon, Philip (OHB) called. We wonder if OHB made any intervention and changed anything, because now the errors are too often. Basically we have one of these errors every 4-5 scans... in the evening, at the time of writing (23:08 UT), this happened every second scan, or 15 minutes. SIGH. Sergio will ask.
Jan 10th, 2025, Friday
04:00-10:00 LT: Juan.
- 191-24 : 3 repetitions of 8 scans. Good weather: tau225 ~0.24 and stable +-5 Jy/beam peak to peak rms, no issues with telescope
- 189-24: 1+3/4 repetitions of 4 scans. Good weather: tau225 ~0.27 and stable +-2 Jy/beam peak to peak rms, no issues with telescope
- SB: we lost ~1h at ~9:00 LT of 2025/1/10 because of problems with the HWP server, when switching to polarimetry. We could not start the HWP and the server seemed not to be there. We started the server again but it gave a connection problem. We called Dave and he also had issues. We then realized that there were two instances of the HWP server running at the same time. We killed them and started everything again. It worked.
10:00-16:00 LT: Alice & Juan.
- focus on 3c273, focus ok
- 134-24 1 repetition on 104316
- 134-24 2 repetitions on 113872. Good weather: tau225 ~0.28 and stable +-2 Jy/beam peak to peak rms, no issues with telescope
- 1 skydip polar [nikaw-24] - pointing, calib1san on 3C454, focus (stable), calib1scan [nikaw-24] - @~11:20 UT the beam is unstable, because we're close to the Sun. [CK: How close to the sun do you notice effects on the beam shapes? -> we were 20-30 deg away, facing the Sun for almost the whole morning] We give up observing 132-24 and 115-24 (that are close to 134-24) and move elsewhere
- 135-24 1,5 repetitions of NGC7538 IRS1
- @14:00 difficulties at finding a new focus: probably it's a critical moment for focus (afternoon, +5 degrees Celsius outside, strong Sun) despite the new paint
- we are considering to switch to EMIR but... it's 14:33 UT now, the only available EMIR target is L694-2 (126-24, 7:44-17:32 UT), followed by L1521 (126-24, 15:48-03:02 UT). We switch to EMIR @14:45 UT.
16:00-22:00 LT: Wonju + Stefano. EMIR time!
- 126-24 1 repetition on L6942 with setup nr 2 (25 min, source/setup completed). NOTE that the setup file had set project 033-24 instead of 126-24, therefore the used time DID NOT go into the 126 pools account and is not shown in the pools page. we have not changed it, because we thought it was done on purpose, to have the new data in the same account as the old ones, but we now think it was not on purpose.
- 126-24 we change the setup file such that it now uses prj nr 126-24 correctly and the time we'll observe next will be correctly accounted for in the pools page.
- 126-24 source L1521E setup nr 1, NOPE: pvw ~70 mm, System T ~700 K, and we have standing waves... we are too close to the Moon! Probably there will be an occultation in a couple of hours too. Aborted.
- there are no more EMIR backup sources available right now. We switch back to NIKA2-Pol @17:10. The HWP rotation is again being capriceuse. Current focus 0.64, the dish might be still affected by the Sun heat received during the afternoon. It might still take some time to relax. Ten minutes later: 0.78. It takes forever to stabilize. @19:00 UT the focus reaches 1.32 mm.
- 166-24 1,5 repetitions on CasA
- beam map in polarimetry mode on Uranus (scan 20250110s230), 99 subscans -> the new make-imbf with the libray fixed by Juan works fine also with 99 and polarimetry
22:00-04:00 Antoine & Tuan Anh. We switch to total power:
- we try twice to take a total power beam map on Uranus (scans 20250110s232,233), but we have to abort both times because of homology problems. Then we give up.
- Tau > 0.3, observing orion B to save good weather time for higher priority porjects
- 113-24 ... maps of Orion-B (cancelled because crate 2 dies at 22:21 UT)
- At 21:57 UT a long series of syslog messages appears on out terminals (see below)
- at 22:21 crate 2 dies. SB tries to reboot it without success. Sergio goes in the cabin and performs a hardware reset of the crate. All is good. Only 15 minutes lost because of this problem.
- SB: spindle and homology problems make us lose much more time, because we have to cancel several (long) scans after a significant amount of time they were running. These problems seem to happen more often in the late evening and night than during the day. Could this be a hint to help finding their cause?
- Still observing Orion B. Spindle alarm at 23:30 UTC
- Spindle alarm at 01:12 UTC during Orion B scan followed by homology error
- Spindle alarm at 01:51 at beginning of track to pointing source. Spindle alarm again on new track scan at 01:55 once on pointing source.
- Spindle alarm at 02:00 UTC during focus
- Starting repetition 8 of MS1054 at 02:20 UTC
SYSLOG messages appeared at 21:57:08 UT (repeated many times):
At 21:57 UT a long series of syslog messages appears on out terminals:
Message from syslogd@mrt-lx1 at Jan 10 21:57:08 ...
kernel: [35243084.986817] Stack:
Message from syslogd@mrt-lx1 at Jan 10 21:57:08 ...
kernel: [35243084.986817] Call Trace:
Message from syslogd@mrt-lx1 at Jan 10 21:57:08 ...
kernel: [35243084.986817] Code: ee 48 c7 c7 84 8c 4c 81 31 c0 e8 74 04 32 00 48 83 c4 18 89 d8 5b 5d 41 5c 41 5d c3 90 90 90 9c 58 c3 57 9d c3 fa c3 fb c3 fb f4 <c3> f4 c3 66 66 66 90 66 66 90 c3 66 66 66 90 66 66 90 c3 0f 06
Jan 11th, 2025, Saturday
04:00-10:00 LT: Juan. keep total power observations:
- 191-24: 3 repetitions, Good weather: tau225 ~0.29 and stable +-5/6 Jy/beam peak to peak rms, degraded a bit for the last scans
- spindle error on 2nd repetition scan 35
- 189-24
- we switch to EMIR at ~08:15 UT
- we observe 143-24. The 143-24-set script had a set project 051-24! AGAIN same story as yesterday. This time we set it to 143-24 since the start!
- we start 143-24, M17 (see next shift)
10:00-16:00 LT: Alice + Juan. Progress:
- 143-24 one round of horizontal maps 1,2,3,4 of M17
- We switch to NIKA2-Pol at ~10:20 UT.
- calib_1scan on MWC349, NGC7027, CRL2688, followed by skydip (scans 20250111s126,127,128,129)
- 115-24 1 hour of G28.34
- 132-24 4 repetitions (~1h in total)
- in between we monitor the instrumental polarization by taking calib_1scan of a known QSO (Uranus will be later).
16:00-22:00 LT: Wonju + Stefano. NIKA2-Pol observations continue:
- tau(225) ~ 0.1
- 135-24 1,3 repetitions on NGC7538 IRS1 and maaaaaaaaany spindle & homology errors that forced us to cancel maaaaaaaany scans. Sigh
- SB: the error beeep rang so many times that it's now broken. No kidding.
- SB: we're wasting tau~0.1 sky with all these spindle errors 
- @18:00 we switch to total power. Lost KIDs: 10/3/34
- beam map, Uranus, total power, 99 subscans, 20250111s208 (keep fingers crossed)
- 188-24 one repetition on A0525
21:15-04:00 Tuan Anh & Antoine. We switch to polarimetry. It takes a while to set up NIKA2: board K is not responding, SB reboots it; them several other boards have a high number of lost KIDs and it takes a couple of reboots of the DAQ to wake them up. Lost KIDs: 11/4/38. Progress of this shift:
- 015-17 one repetition of NGC1333.
- at 22:40 UT the sky degrades and becomes very unstable (~50 Jy/beam peak to peak at 2 mm in Az subscans of pointing scans!!!) -> we switch to EMIR, despite tau(225) is still ~0.1... the sky is fully covered with high clouds
- there are only two sources available for EMIR at this time, and until ~3 a.m. UT: NGC1977 (143-24) and L1521 (126-24), better than nothing
- 143-24 one map of NGC1977 <<BR - @00:05 the sky is completely clear again. We switch back to NIKA2-Pol
- 015-17 one repetition of NGC2264, calib_1scan on Mars failed after 6 attempts (all canceled with "scan stuck" alarm)
- NOTE by SB: Mars has been used for calibration. I've clarified in advance that Mars is at opposition these days, i.e. it's at its closest distance to Earth and it's therefore extended and very bright.
Jan 12th, 2025, Sunday
04:00-10:00: Juan. Total power observations. Progress of the shift:
- 191-24 two repetitions of MS1054
- 189-24 one repetition of CTRS27
10:00-16:00: Alice + Juan. We start with NIKA2-Pol. Progress of the shift:
- 132-24 3 repetitions (~1h in total)
- 134-24 source 103459 1 rep (~20 min). Weather is degrading too much to continue with the observation of this source.
- 11:05 UT, bad weather, we move to EMIR
- 143-24 observed M17 vertical scans. 13:37 UT after pointing wrong backend, we have to stop and cannot relauch because too low elevation. We have added @~/143-24/backend.pako to the scripts. We noticed that vertical scripts for NGC1977 were wrong and had command "sou M17", we have changed to right source name. - 14:50 UT. We switch to NIKA2-Pol, BUT the wind is whistling through the door. It's a north wind, therefore the anemometer does not measure properly. We measure it manually and the wind speed reaches >~24 m/s. The telescope is now stopped. The temperature dropped to -4 C. tau(225) dropped to 0.12. SIGH.
16:00-22:00: Wonju. The tetescope is parked because of the high wind.
22:00-04:00: Tuan-Anh & Antoine. We are still parked for most of the shift. At 00:20 UT we start the NIKA-Pol operations again with BFUN:
- pointing + focus on Uranus
- calib_1scan canceled with homology alarm, so we do again.
- track to 0458-020 canceled due to spindle alarm, we try again.
- 1st map of NGC2024 (~15min), 2nd map canceled due to spindle alarm.
- pointing + focus + calib_1scan on Mars (first calib_1scan canceled due to spindle alarm).
- 1st map of NGC2264, 2nd map canceled due to spindle alarm.
- SYSLOG messages appeared at 02:03:28 UT (repeated many times):
Message from syslogd@mrt-lx1 at Jan 13 02:03:28 ...
kernel:[35430665.444192] Stack:<<BR>> Message from syslogd@mrt-lx1 at Jan 13 02:03:28 ...
kernel:[35430665.444208] Call Trace:<<BR>> Message from syslogd@mrt-lx1 at Jan 13 02:03:28 ...
kernel:[35430665.444226] Code: ee 48 c7 c7 84 8c 4c 81 31 c0 e8 74 04 32 00 48 83 c4 18 89 d8 5b 5d 41 5c 41 5d c3 90 90 90 9c 58 c3 57 9d c3 fa c3 fb c3 fb f4 <c3> f4 c3 66 66 66 90 66 66 90 c3 66 66 66 90 66 66 90 c3 0f 06
- one repetition of NGC2264.
Jan 13th, 2025, Monday
04:00-10:00 LT: Juan. Back to total power observations after finishing B-FUN ; very good conditions tau < 0.1 :
- 191-24: 1 repetition, exceptional weather: tau225 ~ 0.05 and stable +- 1 Jy/beam peak to peak rms
- 189-24: 1 repetitions, exceptional weather: tau225 ~ 0.05 and stable +- 1 Jy/beam peak to peak rms, we see the cluster !
- Polarization beam map in 3C286: 20250113s60, exceptional weather, tau225 ~ 0.07
- came back to 189-24 total power, while waiting for pol projects to rise !
- 189-24: 3/4 of a repestition of CTRS27
- switched back to NIKA2-pol at the very end of the shift
10:00-16:00 LT: Alice + Juan. We start NIKA2-Pol observations:
- 115-24: homology problem during pointing before observing NGC28, attempt to solve. 9:00-9:20 UT homology problems, give back ~20 minutes to project. 9:25 UT homology problems solved, 1 repetition of @ ~/115-24/otf_G28p34 launched. 3rd repetition for this source completed.
- 10:43 UT moving to 3C286 for pointing, calibration, and focus.
- 134-24 1 repetition. Records of the morning: (11:25 UT) attempting 1 repetition for source 113872. Couple of minutes of homology problems, relaunching script manually from the second scan. 12:04 We discovered afterwards that problem were not really solved, give back ~25 minutes to project. Relaunching script for 113872. 3rd repetition for this source completed. 12:20 UT attempting another repetition.
- 13:15 UT new calibration and focus. Some homology problems during the track, solved soon.
- 132-24 3 rounds (starting time 13:37 UTC)
- 14:20 UTC calib_1scan and flux calibration.
- 135-24 1 repetition (starting at 14:53 UTC).
16:00-21:00 LT: Wonju
- 15:00 UT - 17:45 UT: observed project 135-24 without any errors or technical problems. Very stable pointings all the time
- 19:15 UT: no imbfit files from scans 212 (2nd scan of focus) -216 have been not created while "make_imbfits_form_file2.py" was running.
- A;brecht calls us saying that antenna imbfits file seem not to be created. Sergio and Frederic reboot the NCS to solve the missing imbfits files.
- SB it turns out that the old make-imbf produce all files correctly, but the new one didn't. We temporarily use the old make-imbf. 25 minutes lost. Hopefully the new make-imbf is fixedsoon, because we're about to make a beam map and then start B-FUN and the old make=imbf imply long waiting times. Jean restarts the new make-imbf for a tapas bar and it now works fine again (@19:50, scan 223). He does not know what happened. He'll check tomorrow.
- polarimetry beam-map (95 subscans): scan 20250113s225
21:00-04:00 LT: Tuan-Anh & Antoine: 015-17 with very good condition, tau < 0.1
- 21:00 LT: pointing, focus and calib_1scan on Uranus. Spindle alarm during NGC1333 1st scan and homology alarm with 2nd scan, both canceled.
- 22:00 LT: 1 successful repetition of NGC1333.
- 23:30 LT: polarimetry beammap1scan99sub (scan 254) - 00:05 LT: canceled NGC1333 scan due to spindle alarm.
- 00:25 LT: 1 repetition of NGC1333, spindle alarm at near the end of scan (subscan 1.35), so we try to keep it.
- 01:40 LT: 1 repetition of NGC2024
- 02:40 LT: Too windy, the telescope is parked.
Jan 14th, 2025, Tuesday
04:00-10:00 LT: Juan.very good conditions tau < 0.1 but windy:
- restarted and try to focus in polarization for a while for doing a beammap but no possible
- 189-24: 3/4 repetition, exceptional weather: tau225 ~ 0.07 and stable +- 2-4 Jy/beam peak to peak rms, we see the cluster ! but too many tracking errors we need to stop
- ...
- @06:40 UT we stop because of high wind
- we start again at ~8:05 UT and try to observe, but when we finally manage to reach a good focus it is time to shut down and leave space to the weekly maintenance.
Goodbye Pico Veleta and NIKA2, we had a really great time with you!