• A short piece on onboarding I wrote about my first 6 months at Datadog.
  • This was quickly typed out as a response to a question on RLS; it’s a bit stream-of-consciousness but interesting.
  • The summary video and notes on Watkins’ The First 90 Days linked here are particularly useful I think.
• When moving to my current role in late 2022 - which is larger than the one I started in - I took similar approaches, in particular the ongoing list of “hypotheses/questions” to guide early conversations.
  • I also collected notes on “hopes” - a common question I have when meeting folks in a new area is what they’re hoping for (or worried about) as a new senior person steps into the space.
  • I find at some point I’ve built up enough context and stop updating these lists; usually after 2 months or so in the new space.
  • Compare The Blue Tape List.
• An alignment exercise I use is to outline my job description from time to time and share it, in various formats.
  • This might not land well in every environment, but has worked well for me.
  • Note that there’s a section on this in the Staff Engineer’s Path which is worth owning if you don’t have it already.
• Starting a new job as a Staff+ from Amy Unger is very well framed and full of good ideas.

An incomplete set of examples that I feel are most relevant to engineers follows. Note that as we move “up” levels of abstraction here, the value of the metrics is increasingly about the conversation that “pushing on the numbers” can stimulate.

See Will Larson’s notes for more discussion and a broader taxonomy, mostly focused on “investment-level” metrics below.

1. Operations-level metrics

Emitted by our software services: the stuff we put on service dashboards, representing workloads and how they’re doing. May have alerting attached, but probably shouldn’t.

Precision is king, here: we want each metric to have a clear meaning and implications. “Muddiness” or subtlety is the enemy. A good example of the latter is Linux load average. Caveat operator.

2. Service-level metrics

Emitted by our services and the infrastructure around them (for example, synthetic probes). Often used as service level indicators to understand customer experience and user journeys; may also include adoption and capacity metrics. Often have alerting attached.

While we want to define these precisely - see for example the Art of SLOs workshop - it’s OK for there to be uncertainties, subtleties, and warnings attached. Often we have to choose proxies for customer experience, or it’s prohibitively expensive to measure some important part of a user journey.

So we’re fine with service-level metrics needing to have an explanation attached: some bullet points, or a few paragraphs.

3. Investment-level metrics

Gathered in lots of different ways, often across a broad swath of systems and services. Used to drive or inform a specific large-scale investment by a group or organization: a difficult migration, the health of some org-level process. As likely to be delivered by a more traditional BI system as by Datadog dashboards. Rarely have alerting attached.

Again, we want a reasonably clear definition of a metric like this. However, it can be even less precise than a service-level metric, and have more caveats and edge-cases attached. It’s almost always a proxy - often a “trailing edge” indicator - for some important effort we’re trying to drive across the business.

It’s OK for a metric like this to have a page or a whole document explaining it, as long as we agree it provides a useful “needle we can move” over time.

Hi folks, my name is Cian Synnott. I’m senior staff engineer with Runtime Infrastructure - that’s our production compute, networking, identity & access management, and deployments systems. These days I’m focusing mainly on issues around cloud resource management.

The title of this talk is “Effective communication”. I’ll make a few general points, and then focus on the idea of surprise in communication.

I’ll talk about ways to reduce the frequency and impact of surprise, and how to react productively to it. Finally I’ll talk a little about the practice of empathy for our colleagues.

We’ll have time at the end for a discussion. I’m curious to hear whether the ideas I present resonate with you.

Communication at work is an enormous topic. I’ll touch on a couple of general points and then I’m going to focus on one interesting facet: surprise.

First up, what’s “effective” communication? Not just “impactful” - it’s easy to have a big impact with communication, positive or negative. “Effective” implies a goal: communication that has the desired effect.

There’s a goal behind most communication at work: convince someone of something; share context so they understand something that we know; learn ourselves, or teach; make sure we have the same understanding of a problem, or shared priorities.

If you’re trying to communicate something, particularly to a larger audience, and you can’t express what your goal is in doing so, maybe get back to the drawing board.

Second, the myth of the lone genius programmer is persistent in our industry, but the truth is that getting large things accomplished means working as part of a team - where you are communicating constantly.

As such, communication skills are first order skills for software engineers. We tend to valorize pure technical capability, but communication skills are easily as important.

Third, everything we communicate is to some audience of fellow humans: another engineer, our manager, our team, partners, customers, our organization. Not one person in this world sees things exactly the way I do or you do. We each have our own background, context, worries, preoccupations.

So we’re crossing a chasm when we communicate: trying to get our meaning, our understanding, our intention across to the people on the other side. To be effective - to achieve our goals - we have to try to understand where our audience is at and meet them there.

Let’s talk about surprise. Our team, our work group, our company is a community or a social system. And that combines with our technical systems to make our system of work - you hear people talk about “socio-technical” systems.

What do we call a big surprise in our technical systems? An incident! An incident requires response - often by many people - incident management, cleanup. It can be very chaotic. Usually an incident turns out to contain a series of surprises; some of those are in our social system. A good incident review will address these directly.

Our technical systems aside, communication is full of surprises. We can experience these surprises as positive or negative. Many are inconsequential. But big, negative surprises - just like a technical incident - can create a lot of chaos and upset.

As a few examples: unexpected negative feedback; realizing that you and a colleague have a totally different conception of a problem you’ve worked together on for weeks; finding out your company or your customers don’t value something your team has sunk a lot of time into.

Surprise in communication is the enemy of effective communication. It gets in the way of our goals, creates churn that we have to manage, and potentially derails whole efforts. So broadly speaking “don’t surprise people!”

Much easier said than done. Just like in our technical systems, surprise in our system of communication is inevitable. But what we can do is:

• limit how often we surprise people;
• limit the impact when we do; and
• react productively to surprise - ours or others.

How do we surprise people less often and less extremely? There’s a framing from Elizabeth Ayer that I love and use: “radiating intent”. Her analogy is a turn signal on the road: if you indicate what you intend to do, you won’t surprise the people around you when you proceed.

So say you want to make a technical change. You say “I intend to do X”. Repeat that in a few contexts: for example 1:1s, team meetings, customer meetings, an RFC for something bigger. You can get a few different reactions.

• Radio silence? Sounds like assent to me, go ahead and do it. You might still surprise someone, but you created a lot of opportunity for feedback.
• More commonly, you’ll get some feedback: maybe your team says “yeah that’s great, please do it”; maybe your manager says “Y is equally important and more urgent, could you take a look at that instead?” Maybe a colleague tells you that X is a terrible idea, we tried it three years ago, it’ll never work. That’s a good start to a conversation; you’ll learn something.

Note how different this is from “asking forgiveness rather than permission”. We’re still not asking permission: just stating our intent. But by doing that up front we create the opportunity for feedback. And we turn a potentially big surprise - “I went ahead and did this” - into a small one - “I intend to do this”.

Similarly, we can limit surprise by sharing the context we have with others. A big source of surprise in communication is unspoken assumptions, background we don’t share, missing information, differing incentives.

So we can “radiate context” as well as “radiating intent”: actively and readily sharing what we know as well as making connections between efforts and people.

For example, a little FYI or heads-up can go a long way - “in case you missed it, X is working on something similar from another direction, did you see their doc?”.

Or realizing in a meeting with a partner team that they’re unaware of the broader reasons behind how you’re prioritizing, and sharing those directly.

A great note from Denise Yu on what makes a strong senior engineer: “in every conversation you’re part of, create clarity and reduce chaos.”

Reducing frequency and impact aside, surprise is inevitable. So what about when you’ve surprised someone, or you’re surprised? Emotions are immediately heightened. Depending on the situation, you may experience a physical reaction: an adrenaline spike, heart racing, face flushing. And I don’t know about you, but these are things I do not enjoy at work.

But: just like in technical incidents, every surprise is an opportunity to learn. And we learn most when we don’t jump straight to blame and judgement. I won’t call those a natural reaction - who knows - but they’re certainly a common default.

Instead, we can take a breath, feel what we feel, and then set that aside and get very curious. What happened here? What does this person know that I don’t? Have I misunderstood something? What’s important to them in this situation?

A friend of mine calls this “leading with curiosity”, which I love because of the double meaning: both the sense of starting and of leadership. I often think of it as a means of “lowering the temperature”, calling back to Denise Yu’s note on clarity and chaos.

I find people react well to curiosity. I mean, genuine curiosity - not necessarily 20 questions. But “Huh, what happened here?” and a real attempt to understand their perspective.

Each of these ideas - radiating your intent and context; leading with curiosity - is a practical exercise of empathy: we’re trying to meet our colleagues where they’re at when we communicate.

That is frequently not where we’d like them to be: every person brings their own stuff with them to every conversation. So communicating effectively - meeting our goals - is always going to require navigating and dealing with surprise.

The good news is that these are skills! We can practice and get better at them. Just like blameless incident retrospectives are a trained response in an organization, we can train our personal response to surprise.

Sometimes, maybe particularly in our industry, we speak in a very “fixed” way about so-called soft skills: Cian “is” an empathetic engineer, like I was born that way. I assure you I was not.

We can learn to share our perspectives more effectively, and to understand other people’s better; to turn our rapid judgements into hypotheses fuelling our curiosity; and ultimately to achieve our goals more often when we communicate.

To summarize:

• Don’t surprise people BUT
• Surprise is inevitable SO
• Radiate your intent and context AND
• Lead with curiosity AND
• Remember these are learnable skills.

Thank you!

We have a few minutes left, so I’m interested to hear any questions you have, or any thoughts this raised. Does this resonate? Have I over-simplified?

A few references:

• Code and Coders: components of the sociotechnical system - Jessitron
• Don’t ask forgiveness, radiate intent - Elizabeth Ayer
• Operating effectively in high surprise mode - Surfing Complexity
• Kind Engineering - Evan Smith

There’s an important common factor in all of these career steps: being seen as a safe pair of hands. This becomes central as you become more senior.

Yonatan Zunger presented a great talk at LeadDev last year that I find myself referencing a lot: Role and Influence: the IC trajectory beyond Staff.

Zunger’s framing made sense of my roles over the last decade in a way that staff archetypes didn’t. Rather than wondering about what archetype I fit in any particular quarter, it’s much easier to think about the mix of technical, people, product, and project disciplines I’m applying.

The hidden fifth discipline is “adult supervision”, and I think that’s really what I’m talking about here.

A thing I love to see - and experience! - in my colleagues is where they take something on and I know it’ll be done right. Not exactly like I would do it; not some kind of ideal that stands independent of our working context; but right.

The problem is solved; the crisis is handled; the relevant people are informed and involved; risks are surfaced early and when shit goes wrong - as it will! - no-one is caught out.

This is level-independent! It’s perfectly possible for someone to do this in a level-appropriate way. The problems and relationships you’re handling may get a lot hairier as you step up in seniority, but the basic ideas don’t change much.

How trustworthy are you with your work? Do you often surprise people? Can I expect you to be accountable¹ or do I need to rely on someone else for that?

If you can answer well - no matter where you are in your career - then you’re building a solid foundation for your next step. You’re a safe pair of hands.

1. I’ve heard the word “accountable” thrown around a lot in industry, often without definition. Here’s mine:

Being accountable for an effort as an engineering leader has two components: ownership, and communication.

1. Ownership: the effort is “yours”, and you act that way. There may be sub-components spread across people and teams, but overall you’re the one who’s on the hook. Your performance is assessed against the results of the efforts you lead. Judiciously - not every effort will succeed, and that’s OK.
2. Communication: you can tell the detailed story of why we’re doing it, how it relates to other efforts, how it is progressing. You actively raise blocking issues or risks and get the necessary people together to address them. Where you can’t, you escalate effectively.

The format is far from fixed, though. Here are a few examples from my current job in 2022.

Early on, I sketched an “engagement model” with three modes:

• Consult:
  • Work alongside a team to help understand problems, direct energy, guide solutions.
  • Connect teams and individuals dealing with similar problems: enable a “system of theft” of good practice across teams.
• Embed:
  • Given a specific problem, dig in with the team to help bootstrap initial work, or redirect/turn around struggling work.
  • Focus on disambiguating problems to the point that they are a 10-20% stretch for folks on the team.
  • Back off on details but continue to offer decision support.
• Coach:
  • Work with individuals (for example, coming out of consult or embed modes), enabling them to effectively own specific problems and grow via them.
  • Focus mostly on senior engineers and team leads, with the goal of enabling them to do the same for less experienced engineers.

The framing of “engagement model” comes mainly from my work in Site Reliability Engineering, and borrows more recently from Team Topologies. I’ve found this resonates with other engineers too. A colleague was struggling with the transition to “more conversation and less code” in working more broadly across teams: thinking through their different scopes and priorities in terms of an engagement model proved useful.

A little later, my colleague Drew and I expanded on the above to share a longer “your staff engineers and you” doc explaining how we intended to support our group. An excerpt:

How you can use us

The staff engineer, team lead, and senior engineer roles are all “scaling” or “multiplicative”: we help everyone around us to be more effective. Differences are mainly in scope, focus, and expected impact.

The amount and type of support a TL wants from a staff engineer depends on the TL’s focus: some are more interested in the management path, others in the technical path. Similarly, senior engineers want different guidance depending on their experience and current projects.

When it comes to technical direction and decisions, each individual’s appetite for accountability and responsibility is different. We want you to take on as much as you are able to, and support you in growing that capacity.

Things we can help with:

• Batting ideas around;
• Advice and direct support in navigating cross-team or cross-org issues;
• Partnership and review on technical approaches, RFCs, roadmaps;
• Advocacy and signal amplification for your ideas;
• Coaching and mentoring.

Most recently, I proposed embedding with a specific team in my area. I wrote yet another “job description” for this, outlined as:

Why?

• For me
• For the team

How?

• Timeline
• Things I expect to do
• Things I will not do

Other engagements

Success criteria

What you put into a “job description” like this depends a lot on the audience: the first was mainly for my manager and peers; the second for my whole group; the third for a specific team.

In all cases this is about transparency and alignment. “Very senior” engineering roles are frequently confusing, not just for us but for the people we work with. Articulating what we’re trying to achieve and how is both personally and organizationally useful.

Note that Tanya Reilly covers this idea towards the end of chapter 1 of The Staff Engineer’s Path, and offers a lot of useful guidance in figuring out what you do here.

I’ve used various formats - from “incident review” style through “4Ls” - but always with the intention of sharing more broadly than just to me and my manager. My colleagues have generally appreciated the perspective, and some have taken up the practice.

One of my favourite things about doing this is that it’s for me. Yes, it’s useful to my manager (and a useful way of “managing up”). I use it as input to the local performance and review system. But I’m the main audience and it’s my practice. I hate the kinds of performance mangement systems we see in tech companies. Having something that “works around” them to my benefit is very freeing.

Anyhow, Stig and I wrote up some notes and guidance about the idea at $current:

• A guide to personal retrospectives in engineering

It can be difficult to find public examples of compensation structures to use.

These notes are from a quick survey of existing stuff I could find via discussions in opsy chats, the Internet, and direct questions to my network.

Asking questions

First, for those on the job hunt, a list of questions to ask about oncall, gathered from the Irish Tech Community:

• Do you compensate being oncall (i.e. value the stress) or just when you get called (bullshit) or never (warning sign)?
• What is the response time? Is it 5 mins (no life), 15-30 mins (some life, depending on if you have kids), or an hour (you can go to the cinema with your laptop)?
• What percentage of your time is operations, when you’re oncall?
• How many people are in the rotation? If < 6, is there a realistic plan in place to fix that?
  • You need at least 4 people for a reasonable shift pattern, plus one for maintenance (e.g. holidays) + one for emergency (e.g. attrition).
• Is there one person oncall in a shift or is it a primary/secondary kind of thing?

Notes from the ‘net

Second, some posts that cover oncall compensation in various detail:

• https://www.atlassian.com/incident-management/on-call/on-call-pay
  • Kind of marketing white-paper stuff for OpsGenie which seems to have some support for oncall comp calculations. But some good notes on comp structures.
• https://sre.google/sre-book/being-on-call/#compensation
  • Google’s (short) take on it in the SRE book.
• https://johnbarton.co/posts/developers-on-call
  • Has a good section on comp with philosophy as well as basic structure ideas.
• https://copyconstruct.medium.com/on-call-b0bd8c5ea4e0
  • Long, philosophical, no comp structure details but good thinking around why it should be comp’d in “Toward a more sustainable and humane on-call” section.

Example structures

Finally, a set of example compensation structures from various companies.

A fintech company in south America:

• If you are oncall but not working, +33% of equivalent hourly rate.
• Paged and start working, +300% of your hourly for that period.
• Some more extras for nights or weekends.
• They just exported data from Pagerduty: time working was acknowledgement → resolution.
• People would not resolve until they were finished any report or comms work that had to be done out-of-hours.
• This apparently was just how labour laws in that country apply - works the same way for doctors.

A medium-sized SaaS company operating across US / EU:

• Time off as standard if you actually get paged out of hours: ½ day per four hours or part thereof in responding.
• Comp at 25% for oncall time regardless.
• Comp → 100% for the time you’re responding.
• Because of how their shift structure works, this all tends to amount to roughly a 10% lift in salary, plus time to recover.

A large multinational:

• Some teams have business-hours only shifts for internal infra APIs.
• Other teams have customer-facing services and much stricter on-call.
• Those latter get paid per shift, get a mifi, and get time off etc.
• ^ didn’t get exact comp structure here.

Another large multinational:

• Three tiers of oncall, depending on pager SLA.
• Tier 1: >= 99.9% availability SLA, 5min pager response SLA.
  • Comp paid at ⅔ for outside hours
  • That is, outside business hours accrue hours at 2h for every 3h oncall.
• Tier 2: >= 99.9% availability SLA, > 5min but <= 15min pager response SLA.
  • Comp paid at ⅓ for outside hours.
  • That is, outside business hours accrue hours at 1h for every 3h oncall.
• Tier 3: everything else, not comped.
• Mon-Fri comp paid outside 9-6 core hours. Sat & Sun all comped.
• So if you were oncall 6am-6pm Mon-Sun that’d be like
• 3 x 5h for Mon-Fri
• 2 x 12h for Sat-Sun
• So 39h compensatable, converting into pay as 13h at tier 2 or 26h tier 1.
• You could take this as either time in lieu (at 8h/day) or cash (pro-rated to salary).

A medium-sized SaaS multinational:

• Shifts are either weekday or weekend.
• Pay according to 60h week (hourly equiv. from salary) if weekday shift.
• According to 40h week + 24h if weekend shift.
• Payout doubles if schedule includes public/bank holidays.
• Contact there mentioned this was very similar to structure in last job, another similar-sized SaaS.

Intercom’s oncall implementation:

• Former Ruby monolith sharded out over the last few years into services. Heavy on AWS and running less software.
• An unusual structure, but interesting: specifically because they have modified their approach to avoid having “too many people/teams oncall”.
• Virtual team, volunteers from any team in the org.
• 6-month rotations in that virtual team, having taken a handful of shifts.
• Oncall went from being spread across more than 30 engineers to just 6 or 7.
• “We put in place a level of compensation that we were happy with for taking a week’s worth of on call shifts.”
  • Not sure of precise structure, presumably a bonus per week oncall.

Criteo, medium-sized Adtech HQ’d in France. This is from a 3y old Reddit thread:

• SREs are oncall. Pager response time is 30 minutes. (!)
• They are paid for oncall for nights/weekends etc. Exact comp unspecified.
• If you are paged, you get comped time as well in exchange (½ day at least).
• Internet & phone bill reimbursed for oncall engineers.
• If you work during the night, you have to stay home until you get 11h consecutive rest (French law).

• Figure out what I want from life, and how work can support that;
• Use role models and writing to imagine possible futures;
• From those, map out the skills and capabilities I want to develop;
• Lean on my current work environment - or change it - to support my development.

From time to time I talk with colleagues and friends about this. These are some notes to make sharing easier.

Assumptions

I’m writing about software engineering, systems engineering, and adjacent careers. Mainly for individual contributors, because that’s what I know best.

We are on the hook for managing our own careers: a common mistake for less experienced engineers is believing that their manager will do this. Sometimes, managers will be a help; often they won’t.

Everything is learnable. We’re not fundamentally limited by our current roles, skills, whatever.

Starting out

I think about what I want my work to enable in my life.

For some people, that’s wanting to travel, or to learn specific things, or to settle down in a beautiful place, or to found a company, or things related to family, or all of the above.

My personal list is rather general at this point, but well tested. I want to:

• Support my family in a good life;
• Help other human beings do things that are meaningful to them;
• Work with teams I trust;
• Be trusted in turn with a high degree of autonomy;
• Learn a great deal;
• Solve interesting problems;
• Leave things better than I found them.

This is in rough priority order. Had I thought about this more clearly, earlier in my career, I think the order might be different. At times, particularly when changing jobs, I might have had more specific ideas. Many companies and types of work can support everything above. Some simply can’t.

I like having this list because it helps orient me around the things that I need or am looking for. If I can check off everything above, or I am making positive progress, then I feel like my career is in a good place. If there are things lacking, maybe I need to take action to fix that.

It also makes the classic “where do you see yourself in five years?” feel somewhat tractable. :o)

Drawing a map

We have a starting point, but assuming that we want to meet some specific need, how do we figure out what moves to make, where to go?

I like to have a map, a way of identifying the gaps between where I am and my possible futures.

I often use role models for this - colleagues current and former, industry people I respect. What do I value about what they do and how they behave? What do I want to be when I grow up? :o)

Another useful resource is published “job ladders”. These are a good way to look at the things various organizations value and how they see careers progressing. Examples:

• Rent the Runway’s ladder;
• CircleCI’s competency matrix;
• Dropbox’s career framework.

It’s worth having a think about how much you value the skills and capabilities listed, and how different companies’ ladders differ, particularly at senior levels.

Finally, there’s an expanding literature around how senior engineers work:

• Staff Engineer: Leadership beyond the management track - Will Larson
• No Idea Blog - Tanya Reilly’s writing;
• On Being A Senior Engineer - John Allspaw.

There’s a lot to like, to learn, and to model in all of these.

Filling in the gaps

I tend to think in terms of skills and capabilities.

Given all of the above, which skills do I have but could develop further? Which do I lack entirely? What capabilities do I think my role models have that I don’t? Which do I value the most? Which would have helped me in my recent work? Which will enable me the most in future?

Now I have a list of specific things I want to be better at: the beginnings of a plan! I look for ways I can manoeuvre myself into work that will stretch me in those skills, build those capabilities.

Where I think it will be useful, I advertise! Let my manager, my mentors, my team know that I am trying to develop these skills. Can they help me find opportunities to improve? If I can find ways to work directly with some of the people I find inspiring, on those specific kinds of work, even better.

Note that this can form a good basis for annual or quarterly goals, so that’s another pain out of the way. ;o)

Considering context

While you work on your plans and your goals, pay attention to what’s going on around you.

Keep notes on what’s working and what’s not: in your own work, your team, your org, your production systems. This can be a useful source of ideas for specific projects or development opportunities.

If you can, find ways to make your goals work with those of the people around you. Developing your career can and should be something where everyone benefits.

Wrapping up

I have a particular perspective, and I’m certain I am missing a lot: I generally think in terms of skill acquisition and deployment, and I’m not sure how well any of this will apply to different kinds of work or different kinds of people. Take all of my advice with a grain of salt. We’re all just working this out.

To recap:

• Think about what you want out of life, and try to make your career serve that.
• Imagine possible futures by using role models, formal career “level” guides, and the best writing you can find on being an engineer.
• Move towards those possibilities by mapping out a path in terms of skills and capabilities.
• Ask for help getting what you want, and try to make it work well with what everyone around you wants.

With thanks to Tanya Reilly and Niall Richard Murphy.

Wrote a few notes for a colleague about driving a “long incident” as an engineer.

That is, one of those “important AND urgent” things that’s going to take weeks and multiple cooperating teams to get done.

Focus here is on senior IC behaviour, but cf. Managing Incidents from the SRE book.

Overcommunicate.

If you’re running a meeting, have a clear agenda, a plan for what you want out of it. Take notes. If you need to switch audience, write something separately.

Maintain notes in Slack of what’s going on in Zoom.

Summarize and update status frequently.

If you feel like you’re communicating too much, it might be enough. :o)

Communicate clearly to your different audiences.

What do your team, or a sibling engineering team, or your support colleagues, or the execs need to know?

Think about their points of view, and frame things for them.

Engineer to engineer communications are vital. Keep managers and other parties informed, but focus on assembling and directing the engineering team who will solve the problem.

If necessary, fight for the people and resources you need to make progress.

Treat the problem with urgency, but push back on “we don’t have enough time / we can’t test X”.

Take an engineering perspective: get as many facts on the table as you can.

Test assumptions about those facts.

Work constantly to reduce uncertainty.

Map out areas of risk: lean on your experts and help them identify questions we don’t have answers for yet.

Push on getting answers to the tractable questions, given the time and resources available.

Keep an eye out for anyone spinning their wheels.

Stay calm and focused. Help everyone else to do the same. Always worth a re-read: Good Medics Don’t Run.

If you’ve been somewhat insulated from ops, there’s lots of literature out there to help reflect on incidents and long-running issues, and build up those production muscles.

I’d start probably with the Google SRE books, with the usual caveats about $megacorp vs. $tinycorp.

It comes down to people skills - hard skills, and the ones senior engineers most need to cultivate.

Since this is largely about communications, I must include my theme song: Write It Down.

I literally never tire of this video. Sorry / not sorry.

Niall’s thread is as thoughtful, judicious, and balanced as I’d expect. Allow me to be a bit more negative about Objectives and Key Results. :o)

First, the planning horizon. At $megacorp and anywhere that follows that playbook, OKRs are a quarterly process.

“Be careful what you measure”: if your planning cadence is quarterly, there’s a strong chance it’ll become your delivery cadence too. That is too slow.

I love having a plan. Honestly, I enjoy the chaos as we watch it crash into reality, particularly in teams that are constrained by operational concerns (read: basically everyone with real customers).

Plans go out the window. That’s fine, they’re still useful.

But a quarter - for most teams and orgs I’ve worked with, including at $megacorp - is too big a chunk to let us respond well to inevitable change.

So OKRs become a big, expensive activity that swallow a bunch of team and management bandwidth without allowing room to react.

Second, “KRs considered harmful” (with caveats).

Making good KPIs is a known hard problem, right? As an engineering example, consider the difficulty of producing meaningful SLOs.

Coming up with a brand new set of good KPIs each quarter is high overhead and prone to failure.

“KR 1: produce the metrics we’re going to use to track this objective.” :o)

It can feel really hard to tie quantitative measurement of KRs - often proxy metrics - to the real outcomes we’re trying to drive.

Finally though, it’s not all gloom.

Objectives are a good tool for driving alignment. They can build a coherent narrative around strategy and tactics, from execs through to delivery teams.

I like to de-emphasize the measurement part though.

Where we do have KRs, ideally they’re tied to longer-term KPIs that we can iterate on and come to understand better over time, on the order of months or years.

SLOs are a good example. Or from a team perspective, the DORA metrics. Or “north star” product / business metrics.

As for planning horizons: I am strongly attracted to ~6 week “bets” like Basecamp’s or Intercom’s.

Where I’ve worked like this, it’s felt lighter-weight and more agile in the best senses.

All that said, whatever your particular team, org and/or company are doing to plan their time: get involved, and try to make it work as well as possible.

Channel your reservations into creativity. Use and improve the system. Try experiments. Share them.

Planning is good!