Sticking Together while Staying Apart: Resilience in the time of global pandemic

A presentation at Austin DevOps - September Virtual Meetup in September 2020 in Austin, TX, USA by fen aldrich

Million-to-one chances… crop up nine times out of ten. ̶Sir Terry Pratchet GNU

Everythingʼs a little bit broken all of the time… but it keeps working anyway

Resilience Rebound Robustness @CrayZeigh

Resilience Graceful Extensibility Rebound Robustness @CrayZeigh

Resilience Graceful Extensibility Rebound Robustness Sustained Adaptability @CrayZeigh

Rebound Return to “normal” after a surprise or traumatic incident. Work done ahead of time. @CrayZeigh

Robustness The ability to withstand and absorb well-modeled disturbances. Knownknowns @CrayZeigh

Graceful Extensibility The ability to stretch with challenges to operational boundaries. Opposed to brittleness. @CrayZeigh

Sustained Adaptability Recognizing and managing adaptive capabilities over long timescales @CrayZeigh

Bone • Continuously created and destroyed @CrayZeigh

Bone • Continuously created and destroyed • Reconstruction directed by mechanical strain @CrayZeigh

Bone • Continuously created and destroyed • Reconstruction directed by mechanical strain • Process directed by signals through layered networks at cell-level @CrayZeigh

Rebound Graceful Extensibility Robustness @CrayZeigh

Conway’s Law Designed systems represent an organizationʼs communication structure @CrayZeigh

Blunt end Sharp end Closest to the work, practitioners @CrayZeigh

Blunt end Removed from experience, upstream decision makers Sharp end Closest to the work, practitioners @CrayZeigh

Sharp end Closest to the work, practitioners @CrayZeigh

• Constantly building and destroying systems Sharp end Closest to the work, practitioners @CrayZeigh

• Constantly building and destroying systems • Strong signaling Sharp end Closest to the work, practitioners @CrayZeigh

• Constantly building and destroying systems • Strong signaling • Improve systems based on strain Sharp end Closest to the work, practitioners @CrayZeigh

• Constantly building and destroying systems • Strong signaling • Improve systems based on strain Sharp end • Will do so naturally given ownership Closest to the work, practitioners @CrayZeigh

Teams that do well dealing with impact [surprises/incidents] are those that have a strong common ground ̶J. Paul Reed (@jpaulreed), Failover Conf

If we want to improve a teamʼs resilience, we must build a strong common ground ̶Me, Just now.

Common Ground • Basic Compact @CrayZeigh

Common Ground • • Basic Compact Goal Alignment/ Commitment @CrayZeigh

Common Ground • • • Basic Compact Goal Alignment/ Commitment Inter-predictability @CrayZeigh

Common Ground • • • • Basic Compact Goal Alignment/ Commitment Inter-predictability Sustain & Repair @CrayZeigh

Building Common Ground • Blameless Postmortems @CrayZeigh

Building Common Ground • • Blameless Postmortems Chaos Engineering @CrayZeigh

Building Common Ground • • • Blameless Postmortems Chaos Engineering Game Days @CrayZeigh

Building Common Ground • • • • Blameless Postmortems Chaos Engineering Game Days Modeling Vulnerability @CrayZeigh

Resilience is about creating the conditions that maximize everyoneʼs potential ̶Rein Hendrichs, >Code Podcast, 174: Resilience

What happens when governments fail? @CrayZeigh

Community Building is Resilience Engineering ̶Me again, just now again.

Strong Communities • • Diverse High Trust & Safety @CrayZeigh

Strong Communities • • • Diverse High Trust & Safety Sustain & Repair @CrayZeigh

Strong Communities • • • • Diverse High Trust & Safety Sustain & Repair Inter-predictability @CrayZeigh

Strong Communities • • • • • Diverse High Trust & Safety Sustain & Repair Inter-predictability Loosely Coupled, layered networks @CrayZeigh

There were drink coolers sorted and labeled: Water, Sparkling water, Juice, Gatorade. A generator-powered phone-charging station enabled people to stay longer while staying in communication with the outside world. A free library—with no late fees!—was established early on and stocked with the words of Black revolutionaries and poets. By July 1, the occupation was offering free COVID-19 testing, too. -(referencing NYC CHAZ) https://crimethinc.com/2020/07/02/the-cop-free-zone-reflectionsfrom-experiments-in-autonomy-around-the-us

Enable potential and get out of the way @CrayZeigh

Slides & Resources speaking.crayzeigh.com OSMIhelp.org Aaron Aldrich @CrayZeigh EmotionalAPI.co m devopsdays.org

I love you Do good out there Weʼre all in this together @CrayZeigh

Further Reading/Watching/Listening Four concepts for resilience and the implications for the future of resilience engineering - David Woods https://bit.ly/3bITTdc The Marvelous Resilience of Bone - Dr. Richard Cook, REdeploy 2019 https://www.youtube.com/watch?v=8LbePBiOvZ4 Greater Than Code, 174: Resilience https://www.greaterthancode.com/resilience The Worst Year Ever, How to Save your Community When The Government Fails https://ihr.fm/3eVNFbI @CrayZeigh

Further Reading/Watching/Listening Behind Human Error(2nd Edition) - Woods, Dekker, Cook, Johannessen, Carter The Woolworths Experiment https://safetydifferently.com/the-woolworths-experiment/ The Field Guide to Understanding Human Error - Sydney Dekker Literally every video from REdeploy: https://www.youtube.com/channel/UCHbJcI6KfyxflRqdv26b3Qw On Borrowing From Yourself - Aaron Aldrich https://dev.to/crayzeigh/a-reflection-on-borrowing-from-yourself-3jhf @CrayZeigh

Further Reading/Watching/Listening @CrayZeigh

Further Reading/Watching/Listening support your local journalists @CrayZeigh

Further Reading/Watching/Listening support your local journalists support local political candidates @CrayZeigh

Further Reading/Watching/Listening support your local journalists support local political candidates vote @CrayZeigh

fen aldrich
@crayzeigh

1 / 111

Everything is a little bit broken all of the time. Sometimes, like recently, as in right now, the core truths and assumptions about the universe seem to be shifted out from underneath us without warning. And yet, Netflix streams movies, GitHub serves code, we all keep going and working from home and surviving. These systems, both technical and social, are resilient. And through their resilience they are able to handle all those one-in-a-million occurrences that crop up nine-times-in-ten. But what does it mean to be resilient? And further, how can we recognize and grow this resilience to better deal with the systemic surprises we’re dealing with right now and in the future? What does a resilient team look like, and how do you foster that?

This talk will answer:

What is Resilience?
How can we create resilience in ourselves, teams and systems?
How do our networks of systems (people, teams, organizations, applications) create resilience for each other?
What happens when major systems and networks fail?
How can we discover and foster resilience in our people, teams and systems?

Resources

The following resources were mentioned during the presentation or are useful additional information.

Four concepts for resilience and the implications for the future of resilience engineering - David Woods

white paper from David D. Woods
The Marvelous Resilience of Bone - Dr. Richard Cook

Video from Dr. Cook at REdeploy 2019
Greater Than Code, 174: Resilience

Greater Than Code Podcast episode regarding resilience
The Worst Year Ever, How to Save your Community When the Governement Fails

The Worst Year Ever podcast episode specifically around community action in the absence of government resources. You should listen to a number of episodes on this one, though.
Behind Human Error (2nd Edition) - Woods, Dekker, Cook, Johannessen, Carter
The Field Guide to Understanding Human Error - Sydney Dekker
The Woolworths Experiment

Background on Safety Differently and how empowering employees at the sharp end and removing beurocratic top-down policies improved safety overall.
All the REdeploy videos

Seriously every talk from this conference was on point one way or another. Some are more practical, some are more philosophical, all of them are about resilience and worth taking the time to watch and think about. Some more than once.
On Borrowing from Yourself

My blog post reflecting on the costs and alternatives to borrowing work time from your home time in this moment of crisis. Some shameless self promotion, but words I truly want to share and couldn’t fit in the talk.

Sticking Together while Staying Apart: Resilience in the time of global pandemic

Link for this presentation:

HTML code for embedding:

Share on social media:

Resources

Four concepts for resilience and the implications for the future of resilience engineering - David Woods

The Marvelous Resilience of Bone - Dr. Richard Cook

Greater Than Code, 174: Resilience

The Worst Year Ever, How to Save your Community When the Governement Fails

Behind Human Error (2nd Edition) - Woods, Dekker, Cook, Johannessen, Carter

The Field Guide to Understanding Human Error - Sydney Dekker

The Woolworths Experiment

All the REdeploy videos

On Borrowing from Yourself