27/05/2015 Major power failure at Belgocontrol

Join this forum to discuss the latest news that happened in the world of commercial aviation.

Moderator: Latest news team

Nevihta
Posts: 444
Joined: 24 Dec 2008, 16:31

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Nevihta »

Acid-drop wrote:how can it be anything else than 0% working or 100% working ?
the cause of the problem was electrical...
There might be somme issues about hardware or software (degraded modes etc...), but have you been thinking about the human factor ? Not sure controllers can handle such tights movements after such a trauma, and perhaps not trusting their tools...
I think people really don't realize how close we were from a disaster...

regi
Posts: 5140
Joined: 02 Sep 2004, 00:00
Location: Bruges

Re: 27/05/2015 Major power failure at Belgocontrol

Post by regi »

http://www.standaard.be/cnt/dmf20150528_01703369
( Dutch )
so the first conclusions from an independent expert:

The equipment is recent (10 years old) and well maintained. The technical design is no point of discussion at this moment.

The electrical installation of 10 year old was part of a certification by a recognised organisation/body in June 2014. This certification is performed every 5 years.

The check of the generators is a monthly control.

So this means that this electrical installation has been certified by this organisation only once - in June 2014.

I have many questions but will try to wait to send through my worries.

User avatar
sn26567
Posts: 40828
Joined: 13 Feb 2003, 00:00
Location: Rosières/Rozieren, Belgium
Contact:

Re: 27/05/2015 Major power failure at Belgocontrol

Post by sn26567 »

regi wrote:http://www.standaard.be/cnt/dmf20150528_01703369
( Dutch )
so the first conclusions from an independent expert:
Full text in English from Belgocontrol: https://www.aviation24.be/miscellaneous/be ... nt-expert/
André
ex Sabena #26567

airazurxtror
Posts: 3769
Joined: 17 Nov 2005, 00:00

Re: 27/05/2015 Major power failure at Belgocontrol

Post by airazurxtror »

La panne a été déclenchée lors d’un test sur le moteur diesel d’un générateur de secours. Une défaillance technique est apparue dans le moteur d’un refroidisseur, entraînant un pic de courant de 400 volts (pour 230 en temps normal) pendant au moins 10 secondes.
Comme le générateur en question n’était pas relié à la terre, ce surplus électrique n’a pas pu être évacué et est resté dans le système. Le système de protection de surtension a été endommagé ainsi que les « static switch ». Ceux-ci ont brûlé, empêchant le courant de passer dans la salle opérationnelle, qui est donc restée sans courant.
Le contrôle aérien espère pouvoir fonctionner à nouveau à 100 % lundi matin après une nouvelle série de tests.

http://www.lesoir.be/892833/article/act ... orde-terre


The outage was triggered during a test on a diesel backup generator . Technical failure has occurred in the engine of a cooler , resulting in a 400 volt peak ( 230 in normal time ) for at least 10 seconds.
As the generator in question was not grounded , this electric surplus could not be evacuated and remained in the system. The overvoltage protection system was damaged and the " static switch" . They burned , preventing current from flowing in the operations room , which therefore remained without power .
The ATC hopes to work again 100% on Monday morning after a new series of tests.
IF IT AIN'T BOEING, I'M NOT GOING.

Passenger
Posts: 7266
Joined: 06 Dec 2010, 20:54

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Passenger »

airazurxtror wrote:http://www.lesoir.be/892833/article/act ... orde-terre

The outage was triggered during a test on a diesel backup generator. Technical failure has occurred in the engine of a cooler, resulting in a 400 volt peak (230 in normal time) for at least 10 seconds. As the generator in question was not grounded, this electric surplus could not be evacuated and remained in the system. The overvoltage protection system was damaged and the "static switch". They burned, preventing current from flowing in the operations room, which therefore remained without power. The ATC hopes to work again 100% on Monday morning after a new series of tests.
The good news for airports, airlines and passengers is that they've found the cause.
The bad news for the insurers is that the cause is human error: someone forgot to earth/ground the thing.

jan_olieslagers
Posts: 3059
Joined: 24 Jun 2006, 08:34
Location: Vl.Brabant
Contact:

Re: 27/05/2015 Major power failure at Belgocontrol

Post by jan_olieslagers »

The text mentioned by André at https://www.aviation24.be/miscellaneous/be ... nt-expert/ does not mention the missing or faulty earth/ground connection, and I must say I have some misgivings about that.

I can fully imagine the first part of the story: installation running on emergency power, emergency generator produces much too high voltage, one or more protection systems or devices blown up, bang, and the rest is silence. Not an unusual story, actually.

The second part - which, again, is not in the official communiqué - has it that the excess power ought to have been diverted by the earth circuit, which was however inop. I cannot imagine this: the earth/ground (aarding / mise à terre) exists to eliminate faults in insulation, avoiding parts open to human access to carry lethal voltages and thus protecting human life. But it never had anything to do with getting rid of excess power, nor can it be expected to.

Any expert information?

Passenger
Posts: 7266
Joined: 06 Dec 2010, 20:54

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Passenger »

jan_olieslagers wrote:The text mentioned by André at https://www.aviation24.be/miscellaneous/be ... nt-expert/ does not mention the missing or faulty earth/ground connection, and I must say I have some misgivings about that.
Indeed. I've read that also. But you're referring to an article from Thursday 28th, whilst all news sources report additional information given by Belgocontrol on Friday evening, 29th.
jan_olieslagers wrote:Any expert information?
One doens't need to be an expert to understand what Dominique Dehaene, Belgocontrols' press spokesman, yesterday said in very understandable Dutch to all Belgian tv stations and to Belga. Het Nieuwsblad reports about that statement from Belgocontrol: "De stroompanne bij luchtverkeersleider Belgocontrol, waardoor het Belgische luchtruim woensdag urenlang gesloten was, is het gevolg van een combinatie van factoren, maar met centraal in het verhaal een niet-geaarde dieselgenerator" (the cause is a combination of factors, but the non-earthed power unit is in the middle of the story).
http://www.nieuwsblad.be/cnt/dmf20150529_01705220

deredactie.be (VRT) also spoke to Dominique Dehaene and deredactie.be writes about that interview: "Over de verantwoordelijkheid voor de fout, kan hij niets zeggen. Het verhaal lijkt echter eerder te wijzen naar de ontwerper, de installateur en/of de keurder van de installatie". Dominique Dehaene cannot tell who's responsible for the error, but it seems to point towards the designer, the installer or the inspector of the power unit.
http://deredactie.be/cm/vrtnieuws/binne ... =1.2354809

Like I said: the power failure is not caused by overmacht/force majeure, but by technical failure, caused by human error. Which is bad news for one or more insurers. But then again: errare humanum est. That's why we have insurances.

jan_olieslagers
Posts: 3059
Joined: 24 Jun 2006, 08:34
Location: Vl.Brabant
Contact:

Re: 27/05/2015 Major power failure at Belgocontrol

Post by jan_olieslagers »

Glad to agree on essential points: "errare humanum est" or, translated to contemporary English: "shit happens". And indeed that's why we have insurance.

I will also accept there was a succession of issues leading to the major failure. Come to think of it, this story must be remarkably akin to an aviation mishap: if one thing goes wrong, it can usually be dealt with. If several things go wrong, or were not fully ok from the beginning, the domino game begins to topple. Remember the holes in the cheese lining up?

Still, I think the story about the missing earth being central to what happened is technologically bullshit. If proper earthing fails, people may get nasty electrical shocks, up to and including electrocution. Which we certainly do not want, the earth must be ok. If it wasn't then someone is to blame; correction is certainly required and perhaps some damage claim can be made.

But the earth connection has, as far as I can see, neither the design purpose nor the ability to get rid of excess voltage/power. That is the point I should like to see commented by an expert, that is to say an expert on high power high availability electrical installations. Myself having some basic insights in the matter but not claiming expertise. But I fail to see a causal influence of the missing/failing earth connection on the chain of failures.

Lysexpat
Posts: 151
Joined: 31 May 2013, 11:44

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Lysexpat »

When newspapers are writing about my domain, more often than not they talk nonsense, so I wouldn't base any judgement on what you read.
As to the reduced capacity: there still is possible lack of back up power, for this reason separation between aircraft is increased as a precautionary measure. So if eg landing rate is reduced from maximum 40 to 30 an hour the capacity is said to be 75%.

jan_olieslagers
Posts: 3059
Joined: 24 Jun 2006, 08:34
Location: Vl.Brabant
Contact:

Re: 27/05/2015 Major power failure at Belgocontrol

Post by jan_olieslagers »

Yes, Lysexpat, but for this once it seems the journalist faithfully rendered the words of Dominique Dehaene*. I originally this was another case of poor journalism, of which we have seen more than enough; but either Mr. Dehaene had it wrong or some technician told him an oversimplified story.

Regarding the reduced capacity: that seems strange to me, too, but there are suggestions that not all servers are up and running again. This may imply that certain functionalities are not yet available, which in turn may cause increased separation.


*le fils du père?
Last edited by jan_olieslagers on 30 May 2015, 19:31, edited 1 time in total.

Homo Aeroportus
Posts: 1489
Joined: 24 Feb 2007, 18:28
Location: 2300NM due South of North Pole

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Homo Aeroportus »

jan_olieslagers wrote:Glad to agree on essential points: "errare humanum est" or, translated to contemporary English: "shit happens". .....
The sentence in Latin continues as "... perseverare diabolicum." Do you have a translation in English for this part? ;)
Maybe ".. and some like it"? :silent:

H.A.

Homo Aeroportus
Posts: 1489
Joined: 24 Feb 2007, 18:28
Location: 2300NM due South of North Pole

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Homo Aeroportus »

jan_olieslagers wrote: ....
Still, I think the story about the missing earth being central to what happened is technologically bullshit. If proper earthing fails, people may get nasty electrical shocks, up to and including electrocution. Which we certainly do not want, the earth must be ok. ...
But the earth connection has, as far as I can see, neither the design purpose nor the ability to get rid of excess voltage/power...
You are perfectly correct Jan. Presenting the earth connection as some kind of an overflow path for excessive voltage as if it were water in a tub is nonsense.

I don't know the details of the installation and what happened but I note that reference is made to "400V where 230V was expected".
Well these two numbers are linked in a 3-phase system : when you have 230V between phase and neutral, you have 400V between phases.

There is also another thing that is the type of earthing (or grounding) design. In a 3-phase system, you need a neutral conductor (N) and an earth/ground conductor (PE, for Protective Earth).
This can be achieved by two separate conductors, i.e. a "N" and a "PE". This is called a TN-S design ; T for "terre" (as the IEC/CEI uses both English and French as official communication languages), N for neutral and S for separate).
Or, you can have a TN-C, (C for combined) and then you have only one common conductor, called PEN (Protective Earth & Neutral). Saves on cabling, perfectly legal.

So what happened is probably something around this lack of connection to earth/ground but in the sense that it affects the functioning of a 3-phase distribution system where the neutral is "lost" while you need it.

Well, I realize that by the above I probably have filled the background with so many information that the foreground goes ... underground ! :?
Sorry for that.

Time will tell and hopefully lessons will be learnt.

H.A.

jan_olieslagers
Posts: 3059
Joined: 24 Jun 2006, 08:34
Location: Vl.Brabant
Contact:

Re: 27/05/2015 Major power failure at Belgocontrol

Post by jan_olieslagers »

Time will tell and hopefully lessons will be learnt.
Exactly. I understand there will be an investigation by the Air Accident Investigation Unit, and they have a solid reputation for working slow - they have been known to take more than a year to publish a report - but to dig out matters to the bottom, _and_ to document their analysis and conclusions in depth. Their biggest difficulty seems to be that they are not always allowed to explicitly state all they know... some of their reports require reading between the lines.

Passenger
Posts: 7266
Joined: 06 Dec 2010, 20:54

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Passenger »

Johan Decuyper, CEO Belgocontrol, was one of today's guests at De Zevende Dag, Sunday morning's talkshow on Flemish tv VRT. Mr Decuyper told that a final test of all equipment has been done last night, that it was successfull, and that Belgocontrol therefore will have no restrictions anymore as from Sunday night.

He also told that the "human error" happened about ten years ago, when the diesel power unit was installed.

Source:
http://deredactie.be/cm/vrtnieuws/binne ... =1.2355971

De Zevende Dag is not online yet
http://www.een.be/programmas/de-zevende ... -deze-week

(edited : source added)

regi
Posts: 5140
Joined: 02 Sep 2004, 00:00
Location: Bruges

Re: 27/05/2015 Major power failure at Belgocontrol

Post by regi »

The human error happened 10 year ago?
And what about the certification in 2014?

Inquirer
Posts: 2095
Joined: 14 Feb 2012, 14:30

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Inquirer »

Just seen on De Standaard:

http://www.standaard.be/cnt/dmf20150605_01716413

according to De Standaard, the so called 'black out bus' is an illustration of their campaign slogan 'We go the extra smile'.

No idea how many passengers made use of this free city tour that day, but it's definitely being noticed and having an image impact way beyond just those people.

b-west

Re: 27/05/2015 Major power failure at Belgocontrol

Post by b-west »

In today's Het Laatste Nieuws, granted, not the most serious of newspapers, an article states that a disaster was only narrowly avoided during the power outage and apparently the ATC controllers were relying on flightradar24 to keep an eye on the skies... Even though I take articles like this with a pinch of salt, it remains a scary thought to imagine what could have happened that day!

Link to the article (in Dutch) http://m.hln.be/hln/m/nl/33746/Stroompa ... onItemId=1

Passenger
Posts: 7266
Joined: 06 Dec 2010, 20:54

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Passenger »

b-west wrote:In today's Het Laatste Nieuws, granted, not the most serious of newspapers, an article states that a disaster was only narrowly avoided during the power outage and apparently the ATC controllers were relying on flightradar24 to keep an eye on the skies... Even though I take articles like this with a pinch of salt, it remains a scary thought to imagine what could have happened that day!

Link to the article (in Dutch) http://m.hln.be/hln/m/nl/33746/Stroompa ... onItemId=1
The article in HLN.be refers to an article initially published by De Morgen (same ownership though). De Morgen is even worse then not the most serious newspaper. De Morgen is the successor of "De Volksgazet", the newspaper from the Flemish-Belgian socialist movement. It's thus not abnormal that a left wing newspaper supports an anti-govermnent social action.

I just wonder: will an increase of ATC's salaries and better retirement conditions avoid that a failure of the backup system happens again? Or wouldn't it be fair to state that the power back up has been checked at least twenty times since the 2015 breakdown, making that a one-time only failure?

Today's original source for this news from 27th May 2015:
http://www.demorgen.be/binnenland/-belg ... -b747da14/
(free access limited to basic content)

Nevihta
Posts: 444
Joined: 24 Dec 2008, 16:31

Re: 27/05/2015 Major power failure at Belgocontrol

Post by Nevihta »

Nevihta wrote:I think people really don't realize how close we were from a disaster...
ATCO's used what they had, only personal stuff.
Regional towers were still working too...

jan_olieslagers
Posts: 3059
Joined: 24 Jun 2006, 08:34
Location: Vl.Brabant
Contact:

Re: 27/05/2015 Major power failure at Belgocontrol

Post by jan_olieslagers »

There is a lot of sensationalism there, how could there ever have been a catastrophe pending? If dispatchers have less reliable/precise tools, separation is increased. If dispatch/traffic control becomes impossible, the airspace is closed. Drastic, yes, and of course nobody would willingly do so. But safety has not at any single moment been endangered, for as far as I can observe.

Without wishing to enter into politics, I can only confirm the media are _not_ to be trusted. Not in matters of aviation, and why sould they be different on other subjects? They are either incompetent or willingly lying, I often suspect both at the same time.

Post Reply