Daily Schedules and Hiccups

The place to discuss the LHC. Commissioning, operation, issues, events ....
Danny252
LHCPortal Guru
Posts: 169
Joined: Mon Nov 23, 2009 4:45 pm

Re: Daily Schedules and Hiccups

Post by Danny252 » Sat Dec 05, 2009 5:07 pm

gluon wrote:First 450 GeV collision of the day moved to 20:00 GTM.
CERN runs on GMT+1, actually :-)

Saturday/Sunday Timetable

--
Comments: Multi-bunch beam and collision testing this weekend. Collisions happened 1300-1400, so no idea what's planned this evening.
--

Morning:
- Finish beta beating and dump tests
- 10:00 re-establish reference orbit, with ALICE and LHCb
spectrometer bumps.
- 11:00 re-establish collimator nominal protection

Afternoon:
- 13:00 two beams, set up , 4x4 4e9 checks, finish MPS
for stable beams

Evening/night:
- 19:00 two stable beams, single pilots, then dump
- 20:00 450 GeV collisions, 4x4, 4e9, several fills
of the order of 3 hours length

Tomorrow:
- morning: checks for higher intensity bunches
- test ramp late afternoon
- evening/night: continue with collisions

Danny252
LHCPortal Guru
Posts: 169
Joined: Mon Nov 23, 2009 4:45 pm

Re: Daily Schedules and Hiccups

Post by Danny252 » Sat Dec 05, 2009 10:45 pm

Saturday/Sunday Schedule

--
Comments: Who knows what they'll actually do? ;)
--

Evening/night:
- RF rearrange from 4 cavities to 6 cavities for B1.
- Flip polarity of ALICE solenoid to positive with
at least one beam circulating to check effect
(and correct if needed).
- Finish B1/B2 injection/dump protection:
* ongoing B2 check of async dumps, 30min more
* checks with 4 bunches in beam
(parasitic: BPTX experts can check RF buckets).
* Debunch test with both beams and 4x4
* ~3 hours to finish off
- 1h stable beams 1x1, no adjust, hands off
- ~20 min break before refill
- 4x4 refill, 4e9 bunches, then adjust ref orbit if needed,
transverse adjust each IP, then, and then only,
declare STABLE BEAMS. Then LHCf will move in.
No beam adjustements to be made after that without
going to ADJUST (use handshake). Preferably, no adjustements
at all. Observe stability overnight. Refills will be
done according to the agreed upon handshake mechanism.
Dump for refill, if needed, will be announced by phone
call and Page1.

Tomorrow:
- 9:00 meeting: review nightly events...
- morning: checks for higher intensity bunches
- test ramp late afternoon
- evening/night: continue with collisions

Danny252
LHCPortal Guru
Posts: 169
Joined: Mon Nov 23, 2009 4:45 pm

Re: Daily Schedules and Hiccups

Post by Danny252 » Mon Dec 07, 2009 11:22 am

Hola wonderful LCH Watchers, apologies for the lack of updates.

Monday:

--
Comments: Fairly large cryo problem at Point 4/Sector 45 (cryo screen has plenty of red). Page 1 predicts midnight for cryo to be back to normal - so this schedule probably went into the bin a while back.
--

- Access till 19:00 (update in afternoon).
- Set up SPS with few e10 intensity, bring beams
down the transfer lines (Pt2 and Pt8 closed).
- Re-establish beams, then STABLE BEAMS 4x4 6-7e9 p/bch
for the night.
- Keep taking data till ~8:00 tomorrow.
- Then, one hour of data with LHCb dipole off (change
in ADJUST at end of fill or with no beam and refill).

Look ahead:

- Alternate between STABLE BEAMS, studies for increased
intensity and trial ramps.

gluon
Posts: 28
Joined: Fri Dec 04, 2009 2:40 pm
Location: Lisbon, Portugal

Re: Daily Schedules and Hiccups

Post by gluon » Mon Dec 07, 2009 12:45 pm

According to the e-logbook a glitch in the electrical network was at the origin of all this problems in the cryo systems.

Hope they learn how to avoid it in the future.

Danny252
LHCPortal Guru
Posts: 169
Joined: Mon Nov 23, 2009 4:45 pm

Re: Daily Schedules and Hiccups

Post by Danny252 » Mon Dec 07, 2009 9:31 pm

Monday Night Schedule

--
Comments: Cryo recovery faster than predicted, precycling as I post this (2230). Mostly easily decipherable tonight
--

Evening:
- Recover from access, expect cryo back by about 22:00-00:00.
- Precycle.
- Put LHCb dipole back ON (same polarity as yesterday, B field
pointing down, "+" polarity)
- Put (or keep) ALICE dipole OFF
- Reverse ALICE solenoid polarity (from "+" to "-")

NB1: apparently the power glitch last night tripped LHCb and ALICE dipoles.
NB2: ATLAS toroids and solenoids are and stay off

Night:
- Re-establish beams.
- Stable beams 4x4 pilots of 6-7e9 p/bch.
- End of fill time to be decided around 08:30
- Then, short fill with LHCb spectrometer OFF.
Turn dipole back ON after the test (same polarity).
- Miscelleanous activities:
* RF phase adjustment for collision point.
* Transverde damper noise checks.

gluon
Posts: 28
Joined: Fri Dec 04, 2009 2:40 pm
Location: Lisbon, Portugal

Re: Daily Schedules and Hiccups

Post by gluon » Mon Dec 07, 2009 10:08 pm

There's a cryogenics red alert for S56 (CS ITR5). :(

User avatar
DCWhitworth
LHCPortal Guru
Posts: 581
Joined: Mon Nov 30, 2009 8:13 am
Location: Norwich, UK

Re: Daily Schedules and Hiccups

Post by DCWhitworth » Mon Dec 07, 2009 10:29 pm

gluon wrote:There's a cryogenics red alert for S56 (CS ITR5). :(
Yes I saw that, looks OK now.
DC

The LHC - One ring to rule them all !

User avatar
DCWhitworth
LHCPortal Guru
Posts: 581
Joined: Mon Nov 30, 2009 8:13 am
Location: Norwich, UK

Re: Daily Schedules and Hiccups

Post by DCWhitworth » Mon Dec 07, 2009 10:38 pm

DCWhitworth wrote:
gluon wrote:There's a cryogenics red alert for S56 (CS ITR5). :(
Yes I saw that, looks OK now.
This is odd, different pages are showing different things. Cryo Page 1 shows red, HCC field page shows yellow and the Cryo Main Page for sector 56 shows green !
DC

The LHC - One ring to rule them all !

gluon
Posts: 28
Joined: Fri Dec 04, 2009 2:40 pm
Location: Lisbon, Portugal

Re: Daily Schedules and Hiccups

Post by gluon » Mon Dec 07, 2009 10:47 pm

DCWhitworth wrote:
DCWhitworth wrote:
gluon wrote:There's a cryogenics red alert for S56 (CS ITR5). :(
Yes I saw that, looks OK now.
This is odd, different pages are showing different things. Cryo Page 1 shows red, HCC field page shows yellow and the Cryo Main Page for sector 56 shows green !
Yeah, I saw those discrepancies too. I believe Cryo Page 1 shows circuits status for starting the beams and HCC shows status to maintain the beams.

User avatar
seabass
Posts: 18
Joined: Sun Dec 06, 2009 2:36 pm
Location: Chicago

Re: Daily Schedules and Hiccups

Post by seabass » Tue Dec 08, 2009 3:40 am

not to sound pessimistic... but i wonder if electrical issues are going to be a recurring theme. I mean this thing needs a lot of juice and there are a lot of connections along with micronic resistance tolerances. It really is mind boggling how many things have to work together in order for this clock to tick. I think when you have such a complex collaborative experiment like this, complexity compounds on itself. my optimism lies in the brilliant troubleshooters that protonically drive this 17 mile subway.

User avatar
March_Hare
Posts: 135
Joined: Tue Dec 01, 2009 6:09 pm
Location: The Netherlands

Re: Daily Schedules and Hiccups

Post by March_Hare » Tue Dec 08, 2009 7:34 am

My guess: yes, there will be more of this. Perhaps not all of it is new technology, but this machine is on a scale, humanity has never accomplished before.

Getting it all to work smoothly together and error free, is going to take some time.
Nothing travels faster than the speed of light with the possible exception of bad news, which obeys its own special laws.
~Douglas Adams

User avatar
jmayes
Posts: 86
Joined: Sun Dec 06, 2009 4:46 pm
Location: USA

Re: Daily Schedules and Hiccups

Post by jmayes » Wed Dec 09, 2009 5:58 am

Problems will be a daily thing forever with something this size, just jump over and read a few pages of the Tevatron log (Fermilab) and you will see with even a mature machine like that they go through the same thing constantly. At least it's just tripping circuits and not making smoke!

I expect it will settle down quite a bit though.

J

User avatar
Xymox
Site Admin
Posts: 1022
Joined: Wed Nov 11, 2009 12:55 pm
Location: Phoenix, Az USA
Contact:

Re: Daily Schedules and Hiccups

Post by Xymox » Wed Dec 09, 2009 6:25 am

Hey Danny252 ??? where did ya go ?

This thread is getting ALOT of views...

User avatar
DCWhitworth
LHCPortal Guru
Posts: 581
Joined: Mon Nov 30, 2009 8:13 am
Location: Norwich, UK

Re: Daily Schedules and Hiccups

Post by DCWhitworth » Wed Dec 09, 2009 2:03 pm

seabass wrote:not to sound pessimistic... but i wonder if electrical issues are going to be a recurring theme. I mean this thing needs a lot of juice and there are a lot of connections along with micronic resistance tolerances. It really is mind boggling how many things have to work together in order for this clock to tick. I think when you have such a complex collaborative experiment like this, complexity compounds on itself. my optimism lies in the brilliant troubleshooters that protonically drive this 17 mile subway.
Hopefully the Christmas shutdown will give them a chance to review the issues they have had and make some tweaks/changes to help reliability.

Of course there are always going to be issues. Even if you had 99.999% reliability in every component that would still give you lots of failures.

Protonically ? Great word, I must try and work this into my conversations :)
DC

The LHC - One ring to rule them all !

Danny252
LHCPortal Guru
Posts: 169
Joined: Mon Nov 23, 2009 4:45 pm

Re: Daily Schedules and Hiccups

Post by Danny252 » Wed Dec 09, 2009 2:13 pm

It's schedule time! Wednesday's:

--
Comments: Seems stable multibunch beams are today's objective. Not as fun as collisions at >2TeV...
--

Morning:

- Re-establish beams
- 1h stable beams with LHCb dipole off, test transverse
steering in IP5.
- Continue with tests for higher intensity.

Afternoon/evening:

- Continue with tests for higher intensity, aiming for
a first stable beam fill with 4x4 2e10 p/bch starting
in the late evening.
- NB1: access (~2h) needed for abort gap keeper and
pt6 interlocked BPMs, before completion of higher
intensity studies.
- NB2: ATLAS magnets could start ramp up around 20:00

Post Reply