Reading rollup, 2014-09-06.

| categories: reading

  • Debuggers: Russ Cox

    Debuggers post about a lock concurrency problem in code on Plan9. As always, Cox writes brilliantly and clearly about deep technical topics. Note the links to debugging history notes at the end.

  • The Discovery of Debugging - Brian Hayes

    Linked from rsc's article above. Great history piece on EDSAC and its first programmers.

    Software was very much the "stunted sibling" of hardware in the earliest years. "It appeared that programming might be tedious but not fundamentally difficult. A crucial discovery was yet to be made: the discovery of debugging."

    "The human reader passes over such errors without even seeing them, unconsciously filling in the intended meaning, but the machine is resolutely literal. It does only what it is told to do."

    Nice description of the hellish misery of encoding programs for the EDSAC. :o) By September 1949 (6 months in), then-student David J. Wheeler had already written a basic relocatable-code loading facility, basically a linker, amazing.

    Some really nice insight into the early days here. I wonder if there's something that comes *after* software - scalable architectures, maybe? - that we just haven't thought much about yet because we're so obsessed with the software itself.

  • Pitfalls of Object Oriented Programming - Tony Albrecht

    Deck about memory layout in high-performance games and how OO paradigms screw with optimal (columnar, for caching) layout of data in memory. Well worth a read, though these are problems I don't really want to have. :o)

Generated from pinboard by pinroll.

Reading rollup, 2014-08-26.

| categories: reading

I signed up for Pinboard a little over a month ago, finally cleared my "must read this" email labels to it, and haven't looked back. I find myself spending a lot of time in it now, reading documents and taking notes. Recommended.

Following jmason's example, I thought I'd start rolling some of the links I'm reading into blog posts, starting with older stuff.

My pins are private, so I haven't bothered including tags, and I'm doing this on a pull basis rather than daily. :o)

Generated from pinboard by pinroll.

Designing for Brobdingnag

| categories: sysadmin, design

On the topic of talks, while at Google I was involved in an interesting SRE Classroom event in late 2013. These events were run by many of the SRE offices as a way of sharing design approaches with software and systems engineers who might lack exposure to large-scale systems.

I later gave my talk from the Dublin event to in February 2014. PDF with speaker notes from that event:

Designing for Brobdingnag: considerations in scaling large web systems.

Today, we're focusing on technical designs in large-scale software environments, and patterns which we've found work well.

These can be relatively difficult to get exposure to.

This won't be an exhaustive treatment, but we hope to give you:

  • an appreciation of the kinds of constraints and opportunities we have at scale;
  • some tools for approaching design;
  • and a better idea of the kinds of systems SREs engage with.

Note the "Resources" slide towards the end: lots of good links on design and distributed systems there.

Based on presentations at the London and Mountain View SRE Classroom events by Matt Brown, John Neil and Robert Spier. I also include ideas from Jeff Dean's talk Building Software Systems At Google and Lessons Learned. Many thanks to Niall Richard Murphy, Alex Perry, Laura Nolan and Pete Nuttall for assistance, ideas and review.

monendi te salutant

| categories: sysadmin, fun

Talking with Niall on IRC today, I had a brainwave: with just one letter changed, the famous Latin quote morituri te salutant - "those who are about to die salute you" - could become monituri te salutant - "those who are about to be paged salute you".

A noble and fitting handover note for oncall engineers?

Sadly, as is often the case when I take enthusiastic flights into classical translation, I'm off here: moriturus is a somewhat irregular thing from the verb morior. moniturus is active, not passive: it means "about to warn/advise/notify".

To get this right, we need a future passive participle, which it turns out is supplied by the gerundive. So that gives us monendus.

monendi te salutant

Not as sweet a solution as the one-letter change complete with "monitoring" embedded, but not bad. Vale!

Alerting in production systems

| categories: sysadmin

Anyone who has been (or is going) through the wringer of a noisy pager rotation may enjoy this talk, delivered at in Dublin on July 1st:

Pager Bound: high-signal alerting in production systems.

  • Good alerting is something that needs to be designed: organic growth tends to not go so well;
  • page, ticket or log: eliminate email alerts;
  • if we must be paged out of bed, it should be for something that really needs human attention;
  • we can only handle ~2 events well per shift;
  • service-level objectives are a really useful way to orient our alerting to customer experience & business priorities;
  • page on the symptom as it relates to our SLOs, not the cause.

Following Rob Ewaschuk's philosophy on alerting.

Next Page ยป
Site managed with Blogofile.