Incident Management Matters - Archive
An irregular blog with thoughts and challenges on all aspects of Crisis Management, Emergency Preparedness & Response, Incident Risk and anything else that comes to mind.
Exercise Is Good For You!
Why do so many companies exercise their Crisis Management and Incident Response plans so infrequently and so poorly?
Bold words, I know, but this is based on 20 years of attending exercises with companies, governments and response organisations. Don’t get me wrong; of course there are some great examples of Good Practice out there. But my experience of the majority of exercises, if they happen at all, is that they just do not hit the mark.
I think the underlying cause is borne of a fear of failure, or if not fear of failure just a concern about losing face in front of one’s peers.
It means that people put off having exercises; we’re too busy, we responded to a real incident last year, it will put too much stress on the team, we did a regulator exercise recently so don’t need to do an internal one as well.
(Regulator exercises by the way, are not really exercises if we are honest; they are a demonstration of capability which it is right and proper for the Regulator to expect. Truthfully though they are not a real stress-test of the Incident Management plan)
It is why senior leaders avoid taking their roles; “it would be good for a junior person to gain experience”, “I’ve got a migraine”,” I’ve got to deal with an urgent matter” etc.
It is why that people just make stuff up to deal with challenges (particularly around logistics) they are facing in the exercises. People, vessels and freight aircraft just seem to get magic-ed up out of no-where to solve the problems being faced. A three line email somehow deals with an inject about an NGO protest. Funds needed to support the response are instantly available from the nearest money tree.
This avoidance really frustrates me. Frankly, it amounts to cowardice. It even frustrated me when I was the Incident Commander in exercises at Oil Spill Response Ltd.
I remember well an exercise where an inject had a vessel casualty blocking the entrance to the port where our Capping Stack was going to be deployed from. I was just enjoying the challenge of finding a solution when overnight the SimCell told us that the vessel had been salvaged and all was well. I was genuinely disappointed to have that issue removed so easily.
As far as I’m concerned the best exercises use real-time, real-life data and have injects that make them tough as hell. “train hard and flight easy” is a great military maxim and it applies equally to Incident Management exercises.
If you happen to be a Star Trek fan, then think of the Kobayashi Maru simulation!
So what makes a good exercise?
The IOGP/IPIECA Report 515 gives Good Practice guidance on Planning and Developing your exercise scenario and injects etc. I feel it is a little light on the Conduct of the exercise itself however.
I strongly advocate using real time data, especially in relation to logistics. Just what is the global airfreight market on the day of the exercise?Exactly where are the support/deployment vessels that will be required going to come from? How long does it really take to obtain visas and work permits for the in-field team? Honestly, who is genuinely available and willing to be deployed in-theatre? These are the issues that trip of up in real events, so let’s play then for real in the exercise.
The team running the exercise must be ready with a stock of potential injects to keep the exercise moving and also, dare I say it, to put pressure on the Incident Command team. If the SimCell see a potential weakness in the response chain, they should be prepared to ditch their original exercise plan and drill down.
The aim of the exercise is not to catch people out of course, but it should explore every potential weakness in the Crisis Plan delivery, so that lessons can be learnt and the plan be made more robust. It is far better to “fail” in an exercise than to mess up in a real incident after all!
Recovery, Rest and Reflection
As we Christmas and a break from work for many of us, I thought now would be a good time to talk about Recovery, Rest and Reflection. Not least because I feel we sometime confuse these terms and I think we should be more deliberate in our work and home lives about the differences between these activities.
(I should say that dictionary definitions do not help us much as they vary and overlap, so I am making my own distinctions here.)
In this season of joy, make sure that despite the mayhem of food, family and fun, you are able to find the time to recover, rest and reflect....
Working long, high intensity, shifts in an Emergency Operations Centre there is little time for rest, but a great need for recovery between shifts in order to maintain energy and focus. A good analogy may be a decathlete who will go through a routine of warming down, ice baths, and massages between events. This athlete is not resting, but they are recovering in order to be ready for the next challenge. The same applies to an Incident Commander who may be working demanding shifts over a prolonged period; just as the athlete’s muscles suffer a build-up of lactic acid which needs to be removed so the human brain needs time to cleanse, re-order and repair. The magical solution to this is sleep! If you have not read Matthew Walker’s excellent book; Why We Sleep, then I recommend that you add it to your list for Santa to deliver this Christmas. Prioritising good sleep is a life-saver during a crisis.
Short breaks during our work day help enormously; even if it is just to walk down the corridor to grab a coffee. These breaks give space to pause, reflect and get your thoughts in order, and somehow walking helps that process. They are neither rest nor recovery, but these brief times of reflection give you super-powers through the incident.
To return to the sportsman/woman analogy; there are few if any sports that are played competitively all the year round. There is a close season for almost all sports and this is when athletes can truly rest, as opposed to just recovering. For us the holidays are our close season; so how can we use them best to rest?
Rest is a complete re-charge and re-set that allows you to return to work, or whatever you do in life, energised and full of great ideas. Rest can take many forms; for me, I love to walk for miles in the mountains, for others it may be reading, knitting, listening to music, gardening or cycling. It can be physical, it can be cerebral; but whatever form it takes, rest is about switching off from work and finding purpose in other things.
During these times of rest, there is also the opportunity for deeper reflection. If you are someone who meditates, prays or practices mindfulness then this may come quite naturally, but taking time to “be” and not to worry about work issues, but to let your mind process those challenges you may be facing is so important. It requires some detachment to avoid spiralling into worry or self-flagellation. One antidote to this is to view yourself and the situation as a third party, dispassionate, observer. However you do it though, these times of deep reflection are vital if you are to grow as a person.
Pitch Perfect Process
I do love a good bit of process. Process helps to keep us safe, it helps protect ourselves and our businesses from liabilities, it offers us reassurance that we are doing the right thing.
There’s all sorts of reasons why process is good. Process though, has its risks; especially in a Crisis situation. The two process problems that I have often encountered in Emergency Operations Centres during live events and in exercises are:
1. Becoming fixated on process at the expense of managing the incident; and,
2. Using processes that work in the routine business environment but are not helpful in managing a dynamic incident
Process Fixation
It really is so easy to become lost in process; and I would argue that this may be of either of two reasons.
The first is that process can become a means by which difficult decisions are avoided. In a crisis situation an Incident Commander rarely has all the information that they would like available and so has to trust their judgement a lot more than they would in the normal business environment. Process tends to demand data and want certainty.
We do not have the luxury of time in an emergency and so have to make decisions before all the data is available. Incident Commanders who are not comfortable with this uncertainty will tend to avoid making the decision by demanding more data and so an infinite do-loop of information requests and clarifications ensues. A wise colleague of mine always preached that “any decision is better than no decision” and this advice remains sound today.
The second reason we sometimes get lost is process is in our desire to deliver the product possible. The same consultants that produce our 1,000 page Contingency Plans (yes I know I’m exaggerating) in peacetime can become absorbed in producing Incident Action Plans of the same depth; fully justifying every recommendation therein with reams of supporting information. Voltaire told us that “the Perfect is the Enemy of the Good” in the 18th Century and he was right.
Process Overlay
Sound Corporate Governance and strong Compliance Cultures are, rightly, drilled into all us and so our businesses all have many layers of legal, financial and quality procedures to keep us safe, protect our company’s reputation and financial position. The problem comes when these procedures are brought into the Emergency Operations Centre and imposed on the Crisis Management procedures already in place. The risk here is that the very review and approvals processes which, with an abundance of good caution, protect our businesses in steady state will slow down or even prevent decisions being made in a timely manner. Perversely, the corporate checks and balances that protect the financial and reputational well-being of a business in the steady-state environment, can result in reputational and financial damage during a crisis. Slow decision-making will burn through cash and impact negatively on the reputation of a business, that will be portrayed as fiddling whilst Rome burns.
What does Good Look Like?
Atul Gawande in his excellent book The Checklist Manifesto[1], makes a distinction between the steady state, routine scenario of a DO – CONFIRM checklist, versus a dynamic, emergency situation of a READ – DO checklist. The point he makes is that we need different (and shorter) processes in an emergency than we do in the routine world. In a crisis we need to react quickly to recover and our standard business models with their, often lengthy, procedures and sign-off requirements are simply not nimble enough to cope in a very dynamic environment.
Why not challenge yourself and your team to develop Crisis Management procedure that distil down to:
- A Role Card,
- a Flowchart; and,
- a couple of “READ – DO” Checklists
…for each role in the command team?
It may sound scary, especially to your Quality Managers, but really are you going to read the 10 page (or even 10 paragraphs) introduction to your Crisis Management Plan when you walk into the Emergency Operations Centre?
If we keep it simple, then we give our incident command teams the handrails they need at 2am in the morning without wrapping them in red tape.
What is more important, however, than the right procedures; is the right people. Good people will manage an incident despite poor procedures, but poor people will fail in a crisis even with good processes. More on that later…….
[1] The Checklist Manifesto. Atul Gawande. Profile Books. ISBN: 978-184668 3145
The Curse of Complacency
Over-reliance on engineering design and business continuity processes can lead to over-confidence around Crisis Management scenarios for Governments, industry and other enterprises. Worse still, push-back against lobbyists make even make it difficult to admit that crises could ever occur. How then do Crisis Management / Emergency Response professionals engage with senior leaders to prepare for Black Swan events?
Engineered Out
Back in the day, I was responsible for the safety case for a major new civil nuclear facility. Working with the engineering design team, we poured over the process flow diagrams, piping and interlock drawings and all the other design documentation. I ran HAZID’s and chaired more HAZOP’s than I care to remember. With my team, I developed insanely detailed fault trees for ever risk scenario we had identified and dug through reems of tribology and other historic failure data to generate probabilistic safety assessments. Then we would undertake critical path analysis to identify weaknesses in our safety barriers and either engineer them out or add additional mitigations. After all this I ran a Failure Modes and Effects Analysis process for every engineering drawing to make sure that the HAZOP had not missed anything. Once complete, I proudly presented my Fully Developed Safety Case to the UK Nuclear Installations Inspectorate, who accepted that the risks from the facility were “As Low As Reasonably Practical” and the facility went ahead.
Never at any stage did I think of any Crisis Management / Emergency Planning scenarios. I was so confident in the engineering design, that it never crossed my mind that there could be any meaningful radioactive release from “my” plant.
I now hang my head in shame thinking about my over-reliance and blind confidence in engineering design.
Perfectly Mitigated
More recently, I was involved in a Crisis Management exercise for a cyber attack scenario. Our story-board had a hacker who had gained access to admin passwords deep in the Company’s system and had launched a ransomware attack. The crisis team looked at how they could run the business on good old pen and paper and, if they could, for how long. Meanwhile the IT team told us not to worry as they just has to roll back the system to the day before the ransomware note, re-install and we’d be back up and running within 48 hours. With respect to any personal and financial data that might have been accessed, the IT team told us that this was impossible due to the Multi-Factor Authentication in place.
As gently as I could, I pointed out that they did not know how the hacker had accessed the Company’s data and how long they had been in there. Simply rolling back to the previous day’s data risked leaving the same entry point still available to the hacker, who could in any case have made themselves a few other access points whilst they were there, potentially including a way of spoofing the MFA processes. It could well take weeks of forensic analysis to find the entry point that the hacker found and close it, meanwhile re-installing peta-bytes of information from the cloud would also take weeks not days. As a result the IT infrastructure could be down for a month or more, not the 48 hrs claimed.
I do not believe that the IT team were deliberately lying here; but in a major exercise with the Company’s senior leadership team, they were so keen to make a good impression and demonstrate their competence that they found it hard to admit that actually the Company could be off-line for weeks if not months in this scenario.
It Could Never Happen Here
The many and various anti-industry and NIMBY pressure groups are also pushing Companies, at a senior level, to use that dangerous word; “never”. There is a potentially counter-productive feedback loop, where the anti lobbyists exaggerate the risk of the new nuclear facility, the proposed fracking campaign or the expansion to the Xylene process train. The more that these lobby groups shout about the risks, the more the spokespersons for industry play them down. The worry as these debates escalate is that the senior leaders within industry begin to believe their own publicity or, more insidiously, decide that these risks should not be recorded on their Risk Register as such a document may be “discoverable” by the anti lobbyists, who would then hold it up as proof of the danger of the plant.
Whilst I have no evidence of this extreme of thinking, I do recall vividly an inland well blow-out (or loss of well control incident, to give it its formal title) for a former Soviet well being run by an SME which I attended. On return, I happened to be speaking to a senior leader of a major IOC and suggested that it would be good to run a crisis management exercises with them around a similar incident. His response was that no IOC would ever have a well blow-out; the engineering design, process safety and blow-out preventer technologies were such that such an incident would never occur.
Sadly, we know what happened for one major IOC a few years later in the Gulf of Mexico….
The Need For Challenge
So how then can Crisis Management professionals get the attention of senior leaders in their organisation without sounding like Chicken Licken shouting that the sky is falling down?
Where we can point to incidents that have occurred in other companies or near misses that could have resulted in emergencies for our own, that helps. It is evidence-based and hard to argue with; loss of well control incidents are now very much on the agenda for all oil & gas companies now for instance. There is still the risk of ‘it couldn’t happened here’ as a response though; the Chernobyl melt-down can be dismissed as arrogant Russian technocrats undertaking unsafe experimentation on a dodgy RMBK reactor – it could never happen in a western-run PWR could it now?
We can argue that by playing out a worst-case risk we can find out much about all levels of our preparedness. Talking about the importance of thinking through “Black Swan” events may offer a soft way in.
When participants in an exercise drift into the Harry Potter school of Crisis Management with overly-simplistic and/or barely credible responses, we need to be prepared to challenge vigorously but respectfully. Put on your auditor hat and probe the rigour of the solutions proposed and, if needs be, call out any prevarication from your participants
“.... if I am in the devil of a hole and want to get out of it, give me Shackleton every time"
Was Shackleton Really A Great leader?
September 2023
Let me start with a confession; I am a massive Ernest Shackleton fan-boy. I have an entire library of books about Shackleton and his unbelievable achievements leading the ill-fated Imperial Tans-Arctic Expedition to Antarctica. Trapped in unexpectedly heavy pack-ice in the Antarctic summer in January 2015 the boat was eventually crushed and sank. Shackleton kept his team safe and well on the ship and then on the ice pack and, via an amazing voyage in a life-boat to South Georgia, eventually all the party were rescued without a single loss of life 18 months later.
Shackleton is often quoted as an example of exemplary leadership. Indeed, my alma mater; the Henley Business School use one of my favourite books on Shackleton’s leadership: Leading at the Edge (Dennis N T Perkins, HarperCollins 2018) on their Leadership courses. But is he really the leadership model that we should all aspire to?
Manger, Leader or Something Else?
One thing is crystal clear from all the books on Shackleton; he was supremely unsuccessful in his life away from the Antarctic. He jumped from job to job and was consistently poor in all of them, eventually dying heavily in debt as he headed out on another Antarctic expedition where maybe he hoped to recover his lost glories.
By any modern definition, Shackleton was a pretty poor manager. He was unwilling or unable to use established processes when given to him, for instance with Robert Scott in a Royal Navy management structure on the Discovery expedition. He hired on the basis of whether he liked the look of a man, not on basis of their skills, experience (or for that matter their psychometric profile and the results of an assessment centre). He was often self-absorbed, he made many enemies and he was disinclined to share his knowledge with others.
Yet still, I love him.
Although Shackleton is much vaunted as an exemplar of leadership, I am far from convinced that the traits he showed are those exhibited by the senior leadership teams in many large organisations. Surely the mark of a great leader would be to bring success to the ventures they were involved with; improving shareholder value, generating a strong cashflow, winning awards and so on. The truth is that he never did that. He did manage the persuade a number of wealthy patrons to invest in his ventures, but they saw little return on investment, unless one counts having a glacier named after you as a good use of your cash. By most objective measures, Shackleton was really a bit of a failure as a leader.
Despite this, I think he is a great role model.
The reason that I admire Shackleton and reference his leadership style in training is summed up by a quote from Apsley Cherry-Garrard, a contemporary of Shackleton’s in the polar expedition field. He wrote “For a joint scientific and geographical piece of organisation, give me Scott; for a Winter Journey, Wilson; for a dash to the Pole and nothing else, Amundsen: and if I am in the devil of a hole and want to get out of it, give me Shackleton every time"[1]. This is what makes Shakleton so special; he was not a manager, he was even not really a leader; but he was the best Incident Commander in a crisis that you could wish for.
[1] Wheeler, Sara. Cherry: A life of Apsley Cherry-Garrard. 2001: Jonathan Cape.
The Great Incident Commander
Many books have been written and careers built on taking about the differences between management and leadership, but few about the difference between the steady-state of managing/leading an organisation and the highly dynamic challenge of leading in a crisis. In any organisation, and especially large organisations, we rely (quite rightly) of having strong processes to guide our activities. Truthfully the line between management and leadership is blurred in these organisations, as the people referred to as ‘senior leaders’ have in most cases got to that position because they have been excellent in how they have implemented management processes as they have climbed through the ranks. This can mean that in a crisis, these same people may be tempted to go back to the “safe space” of process when confronted by the challenges of VUCA[2] world.
Process, of course, has an important role to play in managing a crisis; it offers us a handrail, it helps us control our limbic freeze/flight/flight reflexes, it provides us with checklists and other helpful tools. A crisis/incident management process which we are familiar with and have trained against gives any organisation a firm foundation on which to build their response.
If we stop there, however, we are missing a critical success factor in delivering a safe and successful response. Those managing (or should I say, leading?) the response need to model many of the traits that Ernest Shackleton showed over one hundred years ago. Perkins[3] distilled Ten Strategies for Leading at the Edge from his analysis of Shackleton and I highlight eight of these here that I think should be learnt by a potential Incident Commander / Crisis Manager.
[2] VUCA: volatility, uncertainty, complexity and ambiguity
[3] Perkins, Dennis. Leading at the Edge. HarperCollins 2018
- Victories and Vision. Never lose sight of the ultimate goal but focus energies on short term objectives
- Personal Example.
Set a personal example with vivid symbols and gestures - Balanced Optimism.
Instil optimism and self-confidence but stay grounded in reality - Personal Stamina.
Take care of yourself; maintain your stamina and let go of guilt - The Team is Everything. We are one, we succeed or fail together
- Lighten Up.
Find something to celebrate. Something to laugh about - Risk.
Be willing to take a risk and make a decision - Tenacious Creativity. Never Give Up; there’s always another move.
Too often I have seen exercises and live incidents become absorbed in process, rather than having a clear ‘end-point focus’ (point 1) which required decision-making (7). Often teams become trapped in the repeated use of a single strategy which is clearly not working (8), rather than improvising and trying something different. Maintaining a calm and optimistic mindset (3, 6) has a disproportionate impact in a crisis; if the Incident commander is assured and confident, then so is the team. The team, though, is everything and the Incident Commander is only as good as their team, so they need to be clearly part of it (2, 5). Finally the need to manage yourself and your emotions (4) is often overlooked. How you manage your own mindset in a crisis makes a huge difference to the outcomes.
In a crisis, it pays to be more Ernest.
Haunted by Ghost Ships
August 2023
The risk of a spill in the Mediterranean is increasing - are we prepared?
Russia Rewired
Just 18 months after the invasion of Ukraine and the application of sanctions by Western governments, exports of Russian crude oil and refined products such as Diesel and Heavy Fuel Oil are back at pre-invasion levels (Source: Bruegel). What has changed though is the destinations, means of transportation and distribution. These changes have resulted in an increased risk of oil spills, which needs to be addressed.
Crude oil is mainly heading to India and China with Turkey a close third (Source: CREA). Meanwhile, according to Energy Intelligence, North Africa and Turkey are favourite destinations for Diesel, whilst Asia and the Middle East are the top destinations for Heavy Fuel Oil.
Cargoes are not all travelling direct to these locations however. Smaller shipments are sailing from the Russian Baltic and Black Sea ports to meet VLCCs where Ship-to-Ship (STS) operations are being undertaken to bulk up the cargo for onward transportation.
This is not done for purely commercial reasons though; these vessels are frequently sailing under false names, and without their AIS locator switched on. The STS operations are taking place in international waters away from any Nation States’ jurisdiction. All in order to hide the cargoes from sanctions.
Holidays in the Sun
Three areas of the Mediterranean Sea are hot spots for these questionable activities; offshore Greece, Offshore Malta and around the Straights of Gibraltar. The Marine Executive and Lloyds List tell us that Vessel Management companies domiciled in Greece, Dubai and India have suddenly acquired large fleets of tankers. More than 440 tankers above 30,000 dwt tonnes, with an average age of 20 years old (which would normally mark the retirement age for a tanker) have been identified as solely deployed for these operations; an increase of 180 vessels in the last 12 months . Less than half of these vessels have insurance via the International Group of P&I Clubs, rather they have Russian or Indian cover of sometimes dubious strength. The degree of vetting prior to chartering is also up for debate.
The IMO is doing what it can, but as a UN Organisation which still includes Russia and her supporters within the family, it is limited in what it can do. The worry from the IMO is that this could also result in a participating shipowner evading its liability under the relevant liability and compensation treaties (e.g. International Convention on Civil Liability for Oil Pollution Damage (CLC) and the International Convention on Civil Liability for Bunker Oil Pollution Damage (Bunkers Convention)) in the case of other ships, placing also an increased risk on coastal States and the International Funds for Compensation for Oil Pollution Damage.
All this serves to increase the risk of crude and HFO spills in the Mediterranean. Although international efforts continue, there is little immediate prospect of reducing the risk, so our focus should be on mitigating the consequences of any incident.
A Mediterranean Mess?
Whilst this trade continues the oil spill risk to the coastlines of all Mediterranean states, but especially Greece, Malta, Morocco, Spain and Gibraltar is increased. Worse still, there is a high likelihood is that there will be no easily identifiable Responsible Party or that if there is there will not be adequate insurance cover from the Ship Owner nor from International Conventions. These nations now face the real prospect of a crude or HFO spill impacting their waters and polluting their coastlines with the responsibility and cost of clean-up falling solely on the National Competent Authorities.
With support from REMPEC, West MOPoCo reported on the state of oil spill preparedness in the Western Mediterranean and, once any political gloss is removed, it does not make great reading for states such as Malta and Morocco. Whilst Spain and Gibraltar, for instance, both have the financial means and the incident response structures to respond (not least with Membership of Oil Spill Response Ltd), there are certainly gaps in the ability of many Mediterranean states in their ability to respond to an incident; especially if those states have to bear the costs of the incident management themselves.
We should not, either, rely on International Offers of Assistance to solve the problems in the event of a spill. The idea is great of course, but generally this provides additional expertise and maybe some spare equipment. It does not give deep pools of people and equipment and nor is it a solid funding mechanism.
Meanwhile, according to the EU’s Copernicus Climate Change Service, surface temperatures in the Mediterranean are at unprecedented highs. Whilst this may be great if one is swimming on the Costa Brava, these high surface temperatures provide the energy for more and more intense storms (see for instance the New Scientist).
Hope is not a Strategy
We are, potentially, looking at the perfect storm of an elderly vessel, with no meaningful insurance, being hit by a significant weather event during loading operations. Worse still the ensuing oil spill then polluting the beaches of a Nation that is ill equipped to deal with it and has insufficient funds to call in third party resources.
How then can we be ready?
Projects such as West MOCoPo address generic concepts in preparedness, but with a real and present danger of spills from an unknown or un-funded Responsible Party, it is time to develop and test some more specific and realistic Worst Case Scenarios to test the continency planning and preparedness of those states at risk. IMO, UNEP and REMPEC all have a part to play here is supporting these efforts with in-kind help and, maybe, targeted funding. The lesson of FSO Safer; where the UN family saw a threat and moved pro-actively to address it, rather than waiting for a oil spill to occur, is a great example of how this can work.
Certainly, National Oil Spill Contingency Plans need to be updated, fully resourced and stress-tested to ensure readiness. Regular and realistic National Exercises need to be undertaken to tease out every potential crack in the plans and find solutions. In these exercises, political posturing needs to be put on one side; National Competent Authorities must be prepared to say that “we are not prepared”, otherwise no progress will be made.
The energy industry have a part to play here, just as they did with FSO Safer with both pro-bono support and funding where it is needed. Operators in the waters of states at risk may feel a particular desire to help as part of their ESG efforts.
The threat is real; let’s not just stand by with our fingers crossed.
We need your consent to load the translations
We use a third-party service to translate the website content that may collect data about your activity. Please review the details in the privacy policy and accept the service to view the translations.