You Need Humans to Keep the Grid On

Deck: 

ResilientGrid

Fortnightly Magazine - April 2021

More than ever, the importance of the electric grid is obvious to all of us, who are grateful in this time of pandemic that electricity flowed, allowing us to work where and when needed. That's why resiliency goes hand-in-hand with the operation of the grid, making sure electricity is there, because we certainly are aware when it isn't.

Here, PUF talks to the founder of a unique company, ResilientGrid, who has a powerful story to tell, and an important one, about what it takes to empower the people behind the critical infrastructure operations. This is about the humans at the heart of providing electricity.

CEO Mike Legatt and his team of energy professionals have been in the control room with the operators managing critical infrastructure through multiple threats and have used that experience to build the tools to make sure the lights stay on. Listen in.

PUF's Steve Mitnick: Describe the unique approach you take on resilience.

Mike Legatt: Our approach to resilience comes out of the discipline of resilience engineering, which identifies that the primary source of resilience is human. Technology can help, but it's ultimately humans who build the technology, implement it, and then have to use it in emergency situations.

Mike Legatt: You would think people work better under stress, but it turns out there’s this middle level of stress. If it’s too low, you’re not paying attention and you’re bored, and if it’s too high, you’re making quick decisions.

It's humans that create resilience. Our focus is just like a power-systems engineer. A power systems engineer uses physics, chemistry, and other sciences in order to set up a grid for success.

From an optimization standpoint, we do the same thing on the human side, or specifically about the teaming between humans and the system. Ultimately in order to create resilience, you have to set human beings up for success, where, for a variety of reasons, regulatory and otherwise, we're not going to have computers running the grid with no human beings.

We're at this point where technology is continuing to advance so much, which is incredible, but we've seen this happen before in many other industries where, because the humans weren't factored into the loop appropriately, you would end up having new failure modes you never had before.

As an example, if you look at fatal airline crash rates, they continued to drop until the advent of the autopilot, which created a new kind of failure because the automation pushed humans out of the loop. That had to be fixed before the rates could continue dropping.

PUF: How do you teach others about this?

Mike Legatt: A lot of what we do comes into two categories. There's the human factors and cognitive systems engineering work, and then there's developing software and systems to help utilities.

The best way to explain it is, if you look at someone like me who's not particularly athletic, I can go to the store and buy a pair of shoes and that's great. But if I were a world-class sprinter, I would need a specially made set of shoes that can help me run at my highest potential, specifically tailored for my feet.

We work to understand with utilities what their end use is. We ask the system operators, the people in the field, what they need and the challenges they're facing, and make sure we configure them, our systems, or our approaches to meet those needs, just like the sprinter's shoe.

Getting to know people is important, but also under-standing the specific challenges they face isn't necessarily the complete picture. You have to look at the whole big system. A process they have to complete gets complex, so through a couple of little tweaks, something that would take them fifteen minutes can take thirty seconds.

Those kinds of optimizations are part of what we can bring in. We also look at collaboration. Believe it or not, today the biggest challenges that operators face are still in terms of interruptions: alarms, telephones, and people coming up to them usually because alarms and telephones are blaring.

If we can make a change where an operator takes twenty seconds in the middle of an urgent event to log something that prevents that other person from interrupting them, it may save ten minutes and reduce risks of human error. We've vastly improved the quality of life for them, but also empowered them to be able to use all of their capabilities to the maximum. That's what we're trying to do.

PUF: Have you all studied what kinds of outage events, or service interruptions you can attack this way?

Mike Legatt: A lot of the time, what we end up doing is looking at events historically, as well as running simulations or drills to try to understand where these problems have happened. I have been in several control rooms during several disasters, as well as working with operators and doing interviews.

A lot of the time, the big question is not what went wrong. It's what went right? Also, how do we make that happen easier next time?

PUF: How did you get into it?

Mike Legatt: I was in graduate school doing my dissertation in psychology and catching up on my laundry. I was in my New York apartment, putting the quarters in the machine. When I put the last quarter in the last dryer, the lights went out and I thought I flipped a breaker. But it was August of 2003, and the Northeast blackout.

Living in my Westchester County apartment, I was close enough to the EOC. So, I grabbed my ham radio and spent the night as net control and watching behind the scenes what happens when the lights go out and then reading the root cause analysis later.

That analysis said, it wasn't technological failures primarily, it was human error, specifically due to lost situational awareness and difficulties collaborating. I thought, I'm a psychologist who loves computers, and found these human-computer interactions fascinating. This may be an opportunity to do something that can help people.

PUF: What kind of people work with you?

Mike Legatt: What's exciting about our team is that we're diverse. Some of us, for example, have a background in neuroscience and power systems engineering. We have a cognitive system engineer on our team, and also have people who've come from lots of different industries, from electric utilities to high-speed financial trading to video game design to aircraft design.

We're recognizing that in all of these industries, we face the same challenge, which is, somebody needs to understand quickly what's happening, and make a decision that is going to have a huge impact in lives, dollars, reputation, trust, etcetera.

There's no hit the pause button. All of us come together from that diversity of experience. We all come together to try to solve this problem, because each of these different industries have had slightly different ways of attacking the problem.

PUF: When someone asks, what does that have to do with people, what do you say?

Mike Legatt: We're engineers and at the end of the day, we want to create outcomes. The way we do that is by looking at the environment, constraints, and inputs, and we try to create that optimality to get the results we want.

One of the challenges is that a lot of us in engineering programs are taught to simplify equations, so we factor out humanity. In many ways you can take psychology, sociology, and anthropology, and plug them into engineering formulas, just as you would with physics, chemistry, or math, and by doing that, you can achieve a higher level of performance. It's now a co-optimization problem.

A lot of what you'd see in the industry these days, like self-healing grids, people talk about them as resilience. I disagree. I think of them as robustness.

Imagine that you have a tree and the tree's job is to be with the roots in the ground and the leaves in the air, and that's success. Reliability is basically, is that happening right now? That's straightforward. It is, or it isn't.

Robustness is how much wind, or impact of a truck or whatever else will take the tree from the good state to the bad state. Resilience is, how much will it take or how quickly can you go from the bad state to the good state.

Part of the problem is we tend to think of these things independently, but they're not. Imagine you've got a tiny sapling. The wind blows just a little, and the sapling flies over, but how much energy does it take you to put it back with the roots in the ground and the leaves in the air? You pick it up with your hand, and it's easy.

But when you have one of these big, seventy-five-year-old oak trees, it's going to take a lot of wind. When that tree finally does get knocked over, it takes a whole lot of energy to put it back up and a whole lot of people to do that resilient act. There's a lot more complexity trying to get it done, and it's much easier for confusion to lead to problems. And it may require a lot of creativity.

Part of the challenge is we're building robust systems more than we are building resilient systems. What ends up happening is the way we solve that is when you try to make something more robust, or to achieve a higher level of efficiency, you're increasing complexity, and it means it's getting hard for those humans in those emergencies to solve it. If you want resilient systems, then you need to make sure people are set up for success.

PUF: In cybersecurity we're adding complex systems, but that opens up all sorts of avenues for things to break down.

Mike Legatt: Right. The fact that you can see megawatts on a line from a remote location, helps operations a whole lot. But the fact that someone else could see that too, is a bit of a problem.

PUF: Give us an example. You come in and what do you do?

Mike Legatt: A lot of the time, for a customer, what we're looking at is deploying our software in a control room and the best way to explain it is it's like a heads-up display on an airplane.

You can fly a plane with six pilots and a lot of knobs and gauges, but you can have a heads-up display that shows what you're trying to do, or the situation you're in. There's information you need from all these different sources, consolidated, and shown to you in a way that maximizes how quickly you can read it, based on the situation you're in at that moment.

We do that basically for the grid. From that perspective, a lot of the human factors work is identifying, what do their operators need to be successful within that heads up display?

Then we implement software and do things like training. A lot of the training is not just focused on, here's a piece of software, but also how do we build habits? Because if you have to think about how you're going to make the tool do this thing in an emergency, chances are, you're not getting the full support you need.

It's about creating the situation where, I don't have to think about it. I know exactly how to make my tool show me exactly what I need to see, and then I can focus all my brain power on the problem I'm trying to solve.

You would think that people work better under stress, but it turns out there's this middle level of stress. If it's too low, you're not paying attention and you're bored, and if it's too high, you're making quick decisions.

You'll do anything to get out of the trouble. In a controlling environment, the failure mode usually is two a.m. Things are boring and all of a sudden alarm bells start blaring. That's what we try to prevent.

PUF: What is your greatest achievement within this scope?

Mike Legatt: We don't want to be Batman, but we want to be part of Batman's tool belt. One day we heard an operator say, today you helped me save a lot of lives. That's it.

PUF: Where does this go from here?

Mike Legatt: The grid is changing. In the '70s, power systems were on the tip of the spear of innovation and because of a variety of reasons, we now need to go from being kind of behind, to being the leaders again.

At the same time, our regulatory requirements for human beings having situational awareness, complying, and doing all of these other things are not going away. We've given these operators this responsibility and our lives are in their hands.

What can we do to set them up for success? From increased renewable integration, distributed energy resources, cyber, and alarm management, we're working in all of these different ways, basically around the question, of what's coming tomorrow, building people up for success.

At the end of the day, setting up a highly successful grid is easy. All you have to do is set up the people doing the work for success and let them do what they want to do as effectively as possible. As things get harder for them, we hope to be able to find more and more ways of doing that.