Sticking Together while Staying Apart: Resilience in the time of global pandemic

A presentation at Deserted Island DevOps by fen aldrich

Sticking Together…together while Staying Apart…apart Resilience in the time of global pandemic Aaron Aldrich Developer Advocate @CrayZeigh

Everythingʼs a little bit broken all of the time… but it keeps working anyway

Million-to-one chances… crop up nine times out of ten. ̶(GNU) Sir Terry Pratchet

Resilience Graceful Extensibility Rebound Robustness Sustained Adaptability @CrayZeigh

Rebound Return to “normal” after a surprise or traumatic incident. Work done ahead of time. @CrayZeigh

Robustness The ability to withstand and absorb well-modeled disturbances. Knownknowns @CrayZeigh

Graceful Extensibility The ability to stretch with challenges to operational boundaries. Opposed to brittleness. @CrayZeigh

Sustained Adaptability Recognizing and managing adaptive capabilities over long timescales @CrayZeigh

Bone • Continuously created and destroyed • Reconstruction directed by mechanical strain • Process directed by signals through layered networks at cell-level @CrayZeigh

Rebound Graceful Extensibility Robustness @CrayZeigh

Conway’s Law Designed systems represent an organizationʼs communication structure @CrayZeigh

Blunt end Removed from experience, upstream decision makers Sharp end Closest to the work, practitioners @CrayZeigh

• Constantly building and destroying systems • Strong signaling • Improve systems based on strain Sharp end • Will do so naturally given ownership Closest to the work, practitioners @CrayZeigh

Teams that do well dealing with impact [surprises/incidents] are those that have a strong common ground ̶J. Paul Reed (@jpaulreed), Failover Conf

If we want to improve a teamʼs resilience, we must build a strong common ground ̶Me, Just now.

Common Ground • • • • Basic Compact Goal Alignment/ Commitment Inter-predictability Sustain & Repair @CrayZeigh

Building Common Ground • • • • Blameless Postmortems Chaos Engineering Game Days Modeling Vulnerability @CrayZeigh

Resilience is about creating the conditions that maximize everyoneʼs potential ̶Rein Hendrichs, >Code Podcast, 174: Resilience

What happens when governments fail? @CrayZeigh

Community Building is Resilience Engineering ̶Me again, just now again.

Strong Communities • • • • • Diverse High Trust & Safety Sustain & Repair Inter-predictability Loosely Coupled, layered networks @CrayZeigh

Enable potential and get out of the way @CrayZeigh

Slides & Resources speaking.crayzeigh.com OSMIhelp.org Aaron Aldrich Developer Advocate EmotionalAPI.com devopsdays.org @CrayZeigh

I love you Be safe out there Weʼre all in this together @CrayZeigh

Further Reading/Watching/Listening Four concepts for resilience and the implications for the future of resilience engineering - David Woods https://bit.ly/3bITTdc The Marvelous Resilience of Bone - Dr. Richard Cook, REdeploy 2019 https://www.youtube.com/watch?v=8LbePBiOvZ4 Greater Than Code, 174: Resilience https://www.greaterthancode.com/resilience The Worst Year Ever, How to Save your Community When The Government Fails https://ihr.fm/3eVNFbI @CrayZeigh

Further Reading/Watching/Listening Behind Human Error(2nd Edition) - Woods, Dekker, Cook, Johannessen, Carter The Woolworths Experiment https://safetydifferently.com/the-woolworths-experiment/ The Field Guide to Understanding Human Error - Sydney Dekker Literally every video from REdeploy: https://www.youtube.com/channel/UCHbJcI6KfyxflRqdv26b3Qw On Borrowing From Yourself - Aaron Aldrich https://dev.to/crayzeigh/a-reflection-on-borrowing-from-yourself-3jhf @CrayZeigh

fen aldrich
@crayzeigh

1 / 47

Everything is a little bit broken all of the time. Sometimes, like recently, as in right now, the core truths and assumptions about the universe seem to be shifted out from underneath us without warning. And yet, Netflix streams movies, GitHub serves code, we all keep going and working from home and surviving. These systems, both technical and social, are resilient. And through their resilience they are able to handle all those one-in-a-million occurrences that crop up nine-times-in-ten. But what does it mean to be resilient? And further, how can we recognize and grow this resilience to better deal with the systemic surprises we’re dealing with right now and in the future? What does a resilient team look like, and how do you foster that?

This talk will answer:

What is Resilience?
How can we create resilience in ourselves, teams and systems?
How do our networks of systems (people, teams, organizations, applications) create resilience for each other?
What happens when major systems and networks fail?
How can we discover and foster resilience in our people, teams and systems?

Video

Resources

The following resources were mentioned during the presentation or are useful additional information.

Four concepts for resilience and the implications for the future of resilience engineering - David Woods

white paper from David D. Woods
The Marvelous Resilience of Bone - Dr. Richard Cook

Video from Dr. Cook at REdeploy 2019
Greater Than Code, 174: Resilience

Greater Than Code Podcast episode regarding resilience
The Worst Year Ever, How to Save your Community When the Governement Fails

The Worst Year Ever podcast episode specifically around community action in the absence of government resources. You should listen to a number of episodes on this one, though.
Behind Human Error (2nd Edition) - Woods, Dekker, Cook, Johannessen, Carter
The Field Guide to Understanding Human Error - Sydney Dekker
The Woolworths Experiment

Background on Safety Differently and how empowering employees at the sharp end and removing beurocratic top-down policies improved safety overall.
All the REdeploy videos

Seriously every talk from this conference was on point one way or another. Some are more practical, some are more philosophical, all of them are about resilience and worth taking the time to watch and think about. Some more than once.
On Borrowing from Yourself

My blog post reflecting on the costs and alternatives to borrowing work time from your home time in this moment of crisis. Some shameless self promotion, but words I truly want to share and couldn’t fit in the talk.

Buzz and feedback

Here’s what was said about this presentation on social media.

Amazed at how well @crayzeigh summed up my own philosophy on Community Management. 😁 #DIDevOps #cmgr pic.twitter.com/eGgYIbmTpn
— Kris Freedain (@KrisFreedain) April 30, 2020
Great talk @crayzeigh! #DIDevOps
— Rich Burroughs @ DIDevOps (@richburroughs) April 30, 2020
"Community Building IS Resilience Engineering"
- @crayzeigh
— Chrissy Keygen (@rgegriff) April 30, 2020
Awesome talk by @crayzeigh going on at #DIDevOps regarding resilience engineering. I'm very invested in this due to the lack of it at my current org and because I've seen the benefits it brings to everyone at my previous org.
— Ashhar Hasan (@hashhar) April 30, 2020
#DIDevops @crayzeigh: Strong communities have high trust & safety. They sustain and repair themselves - this is why we have codes of conduct and non-violent communication.
— Heidi, The Sticker Thoughtleader (@wiredferret) April 30, 2020
#DIDevOps : @crayzeigh takeaways 🥰

1 ) Bone as an analogy for resilience
2 ) Conway's Law - system design rep. org. communication structure
3 ) Resilient teams have common ground
4 ) Resilience = creating conditions for success pic.twitter.com/hkdvBvLkHh
— 🎇Quintessence {🥑/} - Stress Testing DevRel (@QuintessenceAnx) April 30, 2020
The Sharp End and Blunt End. Solutions during incidents usually come from the sharp end. @crayzeigh #DIDevOps pic.twitter.com/KyV3SDMvoF
— Rich Burroughs @ DIDevOps (@richburroughs) April 30, 2020
The system itself can't adapt. The humans who are building and consuming it that can. These are sociotechnical systems. @crayzeigh #DIDevOps
— Rich Burroughs @ DIDevOps (@richburroughs) April 30, 2020
I didn't know @crayzeigh is a Haley Joel Osment impersonator. #DIDevOps
— Rich Burroughs @ DIDevOps (@richburroughs) April 30, 2020
We can build more Common Ground with things like blameless postmortems and game days. @crayzeigh #DIDE pic.twitter.com/Mpyy6F8RMm
— Rich Burroughs @ DIDevOps (@richburroughs) April 30, 2020
#DIDevops @crayzeigh: What we can do to help each other is worth doing, and joy is worth seeking and capturing. Experiencing joy is something that makes us stronger and more resilient.
— Heidi, The Sticker Thoughtleader (@wiredferret) April 30, 2020
@crayzeigh loved the talk! Resilience was such an appropriate topic for the state of the world.
— Shawn (@shauber) April 30, 2020
Very appreciated! It's been an amazing day.
— Jodie Struthers (@shriker) April 30, 2020
We have an especially heavy cognitive load right now, and it's important to care for yourself. When we're overloaded, we manage that by leaning on one another, so having a strong and supportive community is esp. important. #didevops @crayzeigh pic.twitter.com/72KN5TuYVk
— Kaslin Fields (@kaslinfields) April 30, 2020
sad I can only watch the first half live, but looking forward to this talk by @crayzeigh #DIDevOps https://t.co/k3Q37zUUof pic.twitter.com/ANY6ZNt6cX
— B. Dimcheff | 🐝🔅👨‍🍳 (@bdimcheff) April 30, 2020
What happens when the larger systems fail? Smaller groups can still maintain resilience right now. If governments won't do it, it's up to us. Community building is Resilience Engineering. [I had never thought of this.] @crayzeigh #DIDevOps
— Rich Burroughs @ DIDevOps (@richburroughs) April 30, 2020
We have to take care of ourselves too. You can't pour from an empty cup. @crayzeigh #DIDevOps
— Rich Burroughs @ DIDevOps (@richburroughs) April 30, 2020
#DIDevops @crayzeigh: You cannot pour from an empty cup. Our first duty to community is to keep ourself as sturdy as possible, and to then enable others in our community to do the same.
— Heidi, The Sticker Thoughtleader (@wiredferret) April 30, 2020
@crayzeigh I enjoyed your mention of emotional intelligence work at the end of your #didevops talk! Reminded me of a book I'm reading right now - No Hard Feelings. It's about emotional intelligence at work. Have you seen it? Thought you might like it.
— Kaslin Fields (@kaslinfields) April 30, 2020
I love you
Be safe out there
We're all in this together@crayzeigh #DIDevOps pic.twitter.com/1MnEldRd37
— Jenn (@geekgalgroks) April 30, 2020
Sometimes personalized avatars in digital space can make you feel connected in meat space.@crayzeigh #DIDevOps pic.twitter.com/r1aZQZEyq6
— Jenn (@geekgalgroks) April 30, 2020
Strong communities are diverse, they enable trust and safety, they are made up of communities on communities. They are how we support each other and make life easier. Even in ACNH, taking pics for livetweeting you sometimes catch the speaker blinking 😆 @crayzeigh #didevops pic.twitter.com/JOgCRmPGbu
— Kaslin Fields (@kaslinfields) April 30, 2020
*comes back to #didevops from a meeting* "What happens when governments fail? Then it's up to us." Oh. Wow. DIDevops got real srs while I was away. Hearing @crayzeigh talk about how community will get us through this global pandemic. pic.twitter.com/p9Dh4SNi0U
— Kaslin Fields (@kaslinfields) April 30, 2020
Community building is resilience engineering@crayzeigh #DIDevOps pic.twitter.com/XLPbNvvKSB
— Jenn (@geekgalgroks) April 30, 2020
Think about the last incident you had to deal with. The solution probably came from the "sharp end", and here's why. They work together from a common ground. @crayzeigh #DIDevOps pic.twitter.com/8QAv5vUFDP
— Chris Eng (@chriseng) April 30, 2020
Beyond the edges of every architecture diagram are the engineers and humans building those systems. @crayzeigh #DIDevOps 🙌💯 pic.twitter.com/RN7ChjFdAF
— Taylor Barnett (@taylor_atx) April 30, 2020
Just to remind you of the awesomeness of this conference.@crayzeigh #DIDevOps pic.twitter.com/x5FWnl2Vt4
— Jenn (@geekgalgroks) April 30, 2020
#DIDevops @crayzeigh: Teams that do well with incidents tend to have a strong common ground. What is that? Common ground is a basic compact and agreement, goal alignment and commitment, and interpredictability, and you can sustain and repair the team culture/common ground.
— Heidi, The Sticker Thoughtleader (@wiredferret) April 30, 2020
You are part of your town, your family, your neighborhood.

All are different networks, all intersect through you.@crayzeigh #DIDevOps
— Jenn (@geekgalgroks) April 30, 2020

Sticking Together while Staying Apart: Resilience in the time of global pandemic

Link for this presentation:

HTML code for embedding:

Share on social media:

Video

Resources

Four concepts for resilience and the implications for the future of resilience engineering - David Woods

The Marvelous Resilience of Bone - Dr. Richard Cook

Greater Than Code, 174: Resilience

The Worst Year Ever, How to Save your Community When the Governement Fails

Behind Human Error (2nd Edition) - Woods, Dekker, Cook, Johannessen, Carter

The Field Guide to Understanding Human Error - Sydney Dekker

The Woolworths Experiment

All the REdeploy videos

On Borrowing from Yourself

Buzz and feedback