Week Four is the final week in the Warfighting Book Club, relating the Marine Corps doctrinal publication Warfighting to business agility.

This is probably my favorite chapter in the book. Several of the sections (Philosophy of Command, Commander’s Intent, Surfaces and Gaps) would be worth an entire week’s discussion on their own.

For facilitators, adding a week for an organization-specific wrap-up would be a great idea. In that week, focus on what was learned from the book and how it could be applied. Keep in mind that application could be short-term or long-term—and making time for both in a wrap-up discussion is important. Short-term changes are easy to identify and make. Long-term changes give a direction to help with sustaining change once the initial excitement wears off.

If you are interested in help facilitating, or help in identifying useful changes for your organization, drop me a line!

---

Warfighting Week Four Leader’s Notes and Questions

[Page numbers and quotes are taken from Warfighting, MCDP 1, 2019 Edition]

• Sections:  The Challenge – Maneuver Warfare – Orienting on the Enemy – Philosophy of Command – Shaping the Action – Decisionmaking – Mission Tactics – Commander’s Intent – Main Effort – Surfaces and Gaps – Combined Arms – Conclusion
• Questions:
  • How might you rephrase the introductory paragraph for a development project?
  • “The Challenge” for us is developing a concept of knowledge work development projects consistent with our understanding of the nature and theory of work and the realities of the modern workplace.  But the conditions are pretty similar to warfighting in a lot of ways, except we don’t get shot at.  But we want to “win quickly…with minimal casualties and limited external support.”
  • Our purpose isn’t exactly “systemic disruption”, but it’s related.  We’re not trying to tear apart an enemy.  But in our workplace version of maneuver warfare, we are trying to achieve our goals with less cost.  Just like maneuver warfare, this requires:
    • Speed
    • Focus
    • Surprise – equivalent being “inside” the enemy–thoroughly understanding it/them.
  • Let’s start by revisiting a question we asked earlier.  We need to completely understand “the enemy”.  Who or what is “the enemy”?    enemy = problem to solve? enemy = customer?
    • In Marketing, when we talk about getting inside the enemy’s through processes, it’s probably the customer
    • In development projects, when we talk about the enemy, it’s probably the problem to solve
  • Last sentence on 4-7 is important  (and people with those traits will chafe in a just-follow-orders world).  What happens if in our projects we are not thinking “above our own level”, or “acti[ng] in consonance with the requirements of the larger situation”?
  • Philosophy of Command – compare / contrast to philosophy of command for our projects?
    • “A centralized system theoretically needs only one competent person, the senior commander, who is the sole authority.”
      • Are our projects centralized or decentralized?
      • Is there a “sole authority” on our projects?
      • Are the answers to those questions consistent?
  • “As part of our philosophy of command, we must recognize that war is inherently disorderly, uncertain, dynamic, and dominated by friction.”  Is that necessarily true of “knowledge work”?  Why or why not?
    • And what happens if you don’t acknowledge that?
  • In “Shaping the Action” it says “the higher our echelon of command, the greater is our sphere of influence and the further ahead in time and space we must seek to shape the action.”  How far ahead in time and space do you need to look in your work?
  • Can someone summarize what “mission tactics” means? [Assigning a subordinate mission / part of a project without specifying how it must be accomplished; this gives freedom (and accountability!) to the subordinate to accomplish the sub-mission; it’s a “contract” and a means of developing subordinates.]
    • How does “mission tactics” fit with the “sphere of influence” discussed in “Shaping the Action”?
  • “Commander’s Intent” is one of the key concepts in the entire book.  What is “Commander’s Intent”?
  • How often do you provide “commander’s intent” when assigning work?  How do you verify that intent was received?  How often do you “understand the intent of the commander at least two levels up”?
  • “Main Effort” is a good section for comparing/contrasting to team structures.
    • What is the “main effort” of the project you’re on now?  How can you best support the main effort?

This is Week Three in the Warfighting Book Club, relating the Marine Corps doctrinal publication Warfighting to business agility.

This section is not my favorite, but raises a lot of good questions to consider organizationally.

---

Warfighting Week Three Leader’s Notes and Questions

[Page numbers and quotes are taken from Warfighting, MCDP 1, 2019 Edition]

• Sections:  Force Planning – Organization – Doctrine – Professionalism – Training – Professional Military Education – Personnel Management – Equipping – Conclusion
• Questions:
  • If you saw this chapter’s section headings in a table of contents, with just a couple of words changed (maybe “Force” for “Team”, and either remove “Military” or focus it more on your craft), what kind of book would you think it was?
  • What is the non-military equivalent of force planning?  How often do we do that?
  • Can someone summarize the “organization” section for us?
    • We could camp out in “Organization” all day and have a great discussion, but we won’t.  But the second-to-last paragraph caught my eye.  3-5 “Operating forces should be organized for warfighting and then adapted for peacetime rather than vice versa.”
      • Let’s pause there and ask what the equivalent of “warfighting” and “peacetime” are for us.  [Project work = warfighting; training, education, learning, retros = peacetime?]
        • That has interesting implications by itself—what happens to military units that are always on a war footing?
      • What would it mean for our “operating forces” to be organized for “warfighting” (= project work)?
      • How do you react to the last sentence:  “Units should be organized according to type only to the extent dictated by training, administrative, and logistic requirements”?
  • What does “Professionalism” mean to you?
  • Do you agree with the book that “trust is an essential trait among leaders”?
  • What effect do mistakes have on your ability to trust someone else?
  • Do you err on boldness or timidity?  Does [YOUR COMPANY]?
  • READ paragraph on 3-8 “Relations among all leaders…”.  What’s your reaction to that paragraph?  How does that apply in the workplace?
  • Training is interesting to dig into:
    • How much time do we allow for OTJ training?
    • How are our training programs structured?
    • What does training typically look like for your craft?
    • In what way—and level of frankness—are critiques provided for training (or actual) events here?
  • Whose responsibility is professional development?
    • (Three-tier:  education establishment, commanders, individual Marines)
    • If you’re not growing professionally, who is responsible for that?
    • If your employees are not growing professionally, who is responsible for that?
  • What do you think of “the mind is an officer’s principal weapon”?
  • “Doctrine provides the basis for harmonious actions and mutual understanding” — do we have “Doctrine” in the workplace?
    • What is the effect of having or not having doctrine?
  • In the workplace, how much time do we spend “waging war” — doing the work — and how much do we spend “preparing for war” — learning, growing, becoming educated?
    • And what’s the effect of that ratio on our effectiveness, growth, and long-term sustainability?

This is Week Two in the Warfighting Book Club, relating the Marine Corps doctrinal publication Warfighting to business agility.

This was a great, meaty section. I left a lot of possible topics and questions off the list because we only had so much time to discuss.

---

Warfighting Week Two Leader’s Notes and Questions

[Page numbers and quotes are taken from Warfighting, MCDP 1, 2019 Edition]

• Sections:  War as an act of policy – Means in War – The Spectrum of Conflict – Levels of War – Initiative and Response – Styles of Warfare – Combat Power – Speed and Focus – Surprise and Boldness – Centers of Gravity and Critical Vulnerabilities – Creating and Exploiting Opportunity – Conclusion
• Questions:
  • Let’s start with the von Clausewitz quote at the beginning (“The political object is the goal, war is the means of reaching it, and the means can never be considered in isolation from their purposes.”).  How often do we reach back to the “purposes” we are trying to achieve to make sure we’re aligning our craft–our “means”–with those purposes?
  • How do politics, policy, and war map into our corporate world?
    • “Commanders must recognize that since military action must serve policy, these restrictions on military action may be perfectly correct.  At the same time, military leaders have a responsibility to advise the political leadership when the limitations imposed on military action jeopardize the military’s ability to accomplish an assigned mission.” (2-4)
  • This book talks about “the enemy” a lot.  What is “the enemy” when we are talking about our development projects?
    • When trying to understand “the enemy”, think about what we are exerting energy against. Hopefully it is not internal departments; instead it should be something external like competitors, market forces, inertia, entropy, …
  • Would someone summarize the difference between “attrition warfare” and “maneuver warfare”?
  • What would an “attrition warfare” style of development look like?
  • What would a “maneuver warfare” style of development look like?
  • What are the relative strengths and weaknesses of those styles of development?
  • Let’s talk about “Levels of War” and apply that to our work. 
    • “The lowest level is the tactical level.  Tactics refers to the concepts and method used to accomplish a particular mission…”  What are the tactics used in your discipline/craft?
    • “The highest level is the strategic level.  Activities at the strategic level focus directly on policy objectives.”  What does “strategic” mean to you?
      • [This is a great place to expand and discuss if time permits.]
      • Is “strategy” something we need to concern ourselves with?
    • What is your focus?  Does your job have you focused on strategy, operations, or tactics?  Where are you most useful?
    • Where are most of the discussions you have?  Strategic?  Tactical?
  • Let’s dig in on “Speed and Focus” for a minute.  Speed is rapidity of action; focus is convergence of effects in time and space on some objective.  What level of speed and focus do you see in your work?
  • Do most people have a “theory of [projects / development / work]” that structures their approach?  Or just experiences and assumptions?
    • DO YOU?
    • What is your “Theory of [software, product, graphic, etc] Development”?
    • Have you ever tried to articulate it?  [FWIW, I haven’t.  But it might be useful.]

One of the best books on agile development or business agility is Warfighting, Marine Corps Doctrinal Publication #1. It takes a little bit of work to translate from the military domain to the business domain, but it’s worth it, in much the same way that The Art of War is.

I recently led a book club / discussion group on Warfighting for a group of leaders (including both managers and individual contributors) at WP Engine.

I’m posting the leader’s notes and questions for anyone else that would like to lead their own discussion group. If you do, drop me a line and tell me how it went!

(Or, if you’re considering leading one and you’d like some guidance, I’d be happy to help!)

---

Warfighting Week One Leader’s Notes and Questions

[Page numbers and quotes are taken from Warfighting, MCDP 1, 2019 Edition]

• Sections:  War Defined – Friction – Uncertainty – Fluidity – Disorder – Complexity – The Human Dimension – Violence and Danger – Physical, Moral, and Mental Forces – The Evolution of War – The Science, Art, and Dynamic of War – Conclusion
  • The von Clausewitz quote at the beginning of the chapter (“Everything in war is simple…”) is pretty applicable to software / product development.
• Questions:
  • What does “war” have to do with “software development” or “Agile”?
  • If you substitute “development” in for “war”, what do you think of the intro?  “A common view of development among Corporate Marketing is a necessary base for the development of a cohesive doctrine because our approach to the conduct of development derives from our understanding of the nature of development.”
  • Product development is typically not violent, and the parties involved are not always hostile.  Is there anything from “War Defined” that seems to be applicable to web dev projects?  (“Not an inanimate object to be acted upon…”, bottom of 1-3)
  • “In this dynamic environment of interacting forces, friction abounds.” (1-5)  What types of friction have you / your teams experienced over the past month?
  • What is your response to uncertainty in development projects? 
    • Are development projects uncertain?  How so?  What sources of uncertainty bother you the most?
  • “The occurrences of [development] will not unfold like clockwork.  We cannot hope to impose precise, positive control over events.”  Do you believe that?  Why or why not?
    • What is the alternative?
    • This dovetails with Complexity (1-12), “Efforts to fully centralize [development] operations and to exert complete control by a single decisionmaker are inconsistent with the intrinsically complex and distributed nature of [development].”  How do stakeholders that want to be that “single decisionmaker” respond to finding out they can’t be?
    • There are two common responses to uncertainty, fluidity, and disorder in corporate settings:  impose order, or create systems that can respond to disorder.  What are the trade-offs involved?
  • Most of the parts about violence aren’t terribly applicable to what we do.  We’re not facing physical danger, causing destruction, and needing physical courage.  So, that said, how are things like courage, leadership, unit cohesion, and esprit applicable to development activities?
  • What do development projects depend on most:  science, art, or social dynamics?
    • “We thus conclude that the conduct of [development] is fundamentally a dynamic process of human [interaction] requiring both the knowledge of science and the creativity of art, but driven ultimately by the power of human will.”  Is that a reasonable paraphrase?
  • What else in this chapter resonated with you?

My family has owned a Vizio TV (model V555-H11) for a while, keeping it disconnected from the internet and using as a dumb display for other devices. The picture has been fine. The audio has had some issues.

We have experienced sound cutting out during quiet parts of movies, often accompanied by a quiet-but-perceptible-and-pretty-annoying-once-you-notice-it popping sound (probably an over-enthusiastic gate without the smarts to quietly clamp the audio, instead just cutting the sound wave abruptly). We are not the only ones, and it’s not just our model.

I had tried different options in the menu to fix it, and not come up with a good fix. But I recently got annoyed enough to try harder to fix it.

What ended up working was a twofold fix:

• update firmware (from 1.11.x to 1.20.y)
• turn “Volume Limiting” off

When I connected the TV to wifi, it was not able to find a firmware update, even though Vizio’s site showed a new version existed. I ended up downloading the firmware and updating using a USB stick (instructions are on the firmware page). The user experience was awfully clunky, involved a factory reset and several TV reboots without showing status, and had me wondering several times if anything was happening—but after 10-15 minutes the TV was updated.

After the firmware update, the gating effect was much less (lower threshold on the gate, or longer hold time, or both?). The pop on either side of the quiet part was gone, which was a win by itself. But, well, we were going to watch a movie in a bit and I wanted to make sure.

So I experimented with other settings. Turning “Volume Limiting” off seemed to help a lot. With “Volume Limiting” turned on, the quiet parts still seemed to get cut out some. With it off, we had the overall volume setting at a higher number, but the sound levels were similar and we could hear (quietly) what was going on during the quiet parts.

Sure, we could get more equipment (e.g., a receiver ahead of the TV) to help with this. But that’d be overkill, when the problem we are really trying to solve is “I’d like to hear the audio without interruptions, please.”

(For those curious, the “test track” was the scene in Encanto in which Mirabel interrupts the party to tell her Abuela that the house is cracking and the magic is in trouble. I had noted that as a troublesome spot before. The noisy party, the hush punctuated by dialogue as they inspect the now-perfectly-fine house, and the sound levels coming up as Abuela announces “The magic is strong, and so are the drinks!” gave me all the conditions I needed to hear to tell whether things were improving or not. And yes, after I got it working, I did give in to the requests to jump to “We Don’t Talk About Bruno“.)

Beginnings, like this one, are important. A good beginning draws you in, puts you in the right frame of mind to engage with upcoming ideas or events, and makes you want more. A good beginning might be funny, action-packed¹, raise burning questions, make you feel a need, or enrage you.

A bad beginning doesn’t do any of those things. A bad beginning is there because a thing has to start somewhere. If it didn’t start with Chapter 1, suddenly Chapter 2 would become Chapter 1. A bad beginning is not interesting, not useful, not repellent. It might be a bit clunky or it might not. It just starts the thing, but only because it’s the first part of the thing it’s starting.

To be sure, good things can start with bad beginnings, and good beginnings can lead to bad things.

The beginning of a new job is often called “onboarding”. But that term means different things at different companies, or even in different departments within the same company. A few months ago, I started at WP Engine. As a new hire, I went through a process called “New Employee Onboarding” or “NEO”. It’s a couple of days of getting all the new-hire stuff handled, communicating culture, and making sure that you can at least communicate within the company. It’s done pretty well. But after NEO is done, there’s more onboarding to be done: the new hire reports to their team to “onboard” there. That’s where it moves beyond understanding the company, and into getting up to speed on the actual work.

Neither of the onboardings I’m going to write about involved WP Engine.

Both of the onboardings I’m going to write about strongly influenced my view of the company I was joining. Think about that for a second, by the way. I had interviewed. I’d met people. I’d read about the company. I’d asked my questions. I’d already formed a positive view of the company and said “yes” to an offer. And even so, the onboarding process did far more to influence my view of the company.

Two onboardings means two companies. I won’t name them, though it’s probably possible to recognize them if you worked with me at either. Let’s call them “DC, Inc.” and “Marvel, LLC”. Both were remote jobs, and in both I had a senior role in software development.

Got that first-day-of-school feeling? A little nervous, a little excited, hoping that nobody will steal your lunch money, and hoping that the school lunch is good this year? Great!

DC, Inc.

I logged into Slack on my first day, a few minutes before the agreed-upon time and took a quick look around. People I didn’t know, talking about projects I didn’t know about. No surprises. Great. I messaged my new boss. Good morning, Bruce. I’m ready to work. Where to start?

We hopped onto a Slack call and talked for a bit. Bruce was excited to have me on the team. I could tell, not just from the words he used but from how he talked and what he talked about.

I’d be working on a project that had a team very much in transition. It turned out that I was replacing someone who had recently left. In the meantime, Bruce was filling in. The other member of the team, Diana, was moving to a different team soon. We had about a week of overlap. But it also turned out Diana had something going on that week (maybe PTO? I don’t recall exactly). So we pulled Diana into the call and made a plan for my first week.

Bruce had meetings or PTO or other teams that needed some attention that week. So on Monday, Diana would get me oriented in the code, make sure I could access all the cloud platform resources I needed, walk me through the project, and hoped to have me set up to do something productive-ish that week while she was out. Bruce would check in every so often throughout the week, and then week two would be more normal. Me, I had no context to argue. It felt a bit chaotic, and a bit like I was working without a net. But it also made sense given our current constraints.

Great! We’d agreed. Diana helped me get my editor set up, got me into Github, gave me some access in our cloud platform and helped me request a bit of additional access, explained the project context and architecture to me, suggested my first commit (I believe we’d found a typo in some of the Getting Started instructions in the developer’s readme), and helped me deploy following my change (spoiler: changing the developer’s readme makes for a pretty safe deployment). I noted how nice it was that the development and deployment docs were in the project repository, and that the docs were genuinely useful. Somewhere in that day, I got to meet the rest of the DC, Inc. crew. They all seemed pretty nice and happy to meet me.

Cool.

Tuesday through Thursday or Friday, I was pretty much on my own. As planned, Bruce would check in every so often. As I recall, I was working on some throwaway Python scripts to prove out an approach to data extraction from a third-party API that we’d later re-implement into the core product in the project language (which was definitely not Python). I worked, and I talked with Bruce when he was free, and I asked a few questions here and there, and I ended up with a few hundred lines of mostly-tested Python that would get and transform the data we needed. Bruce seemed happy about what I’d gotten done.

Also cool.

The next week, Diana had moved to her new project. Bruce’s calendar had cleared up. We were able to get into the new normal rhythm. The DC, Inc. normal way of working was regular pairing. For my first week, I really hadn’t paired much. So in week two I learned what it was to pair remotely for most of the day.

Loved it. It was hard work, but I loved it. Questions never lingered more than a few minutes. “Hey Bruce, why are we building it like this?” I never got too far away from the existing design—or, when I did, it was because Bruce and I had already talked about how the design needed to change. Any confusion about the test framework got cleared up pretty quick.

Though, come to think of it, I do remember writing a half-dozen functions to transform some data that were pretty darn clever while Bruce was in a boss-type meeting. And when he came back, he observed that those didn’t need to be functions. They could be done inline using existing language features. And they’d be more readable.

Darn. He was right. But that was cool too. I learned. And got to delete a few dozen lines of code.

One of the interesting things about the whole onboarding experience was that I was included. Part of the team. Expected to make mistakes at the same time I was expected to contribute. My questions were answered (repeatedly), my jokes were tolerated (sometimes even laughed at), my contributions noted, and it felt great.

A few months later, another team needed Bruce. We’d hired Kara and onboarded her about a month after I’d joined. So Kara and I, with maybe six months of total experience at DC, Inc., were the new team. Bruce was available for questions, sure (so was Diana, for that matter).

And it worked out. Sure, I was senior and of course I’m pretty awesome. But that’s not why it worked out. It worked out because we had really been onboarded. We had enough knowledge about the product we were building, the problems it was trying to solve, the business context we were operating in, and the cloud platform we were building on. We could be responsible for the product. We could have in-depth discussions with other stakeholders. And we did. Like I said, it worked out. Nicely.

Marvel, LLC.

I logged into Slack on my first day, a few minutes before the agreed-upon time and took a quick look around. People I didn’t know, talking about projects I didn’t know about. No surprises. Great. I messaged the CTO, the guy that had hired me and the one person I knew there. Good morning, Tony. I’m ready to work. Where to start?

It turned out I needed to start with some machine set-up, HR-type training videos, and other “getting started” things. Solo work at Marvel, LLC. So I headed off and got started.

Somewhere during all that, my team lead, Clint, gave me a call on Slack to say hi. He pointed me to some of the repositories I’d be working with, said he’d get me invited to stand-ups and other meetings, explained what our team was working on, and said he was glad I was part of the team.

And I got to meet my manager, Scott. He gave me some introduction to the company and talked a bit about some areas he was interested in picking my brain on.

Cool.

The next few days were frustrating. I met my team. That was nice. But I felt awfully lonely at work, even though I remember noting how friendly and willing-to-help everyone was. It seemed like a lot of things were difficult. Some of the training videos seemed to assume context that I didn’t have. I needed to get set up on a certain system to access my softphone. But when I asked IT for help, they kept calling my softphone to try to help me. They left a message, though, so all I needed to do was access my softphone and listen to the message. In other areas, documentation was just a bit out of date, and several of the links I got handed in Slack contradicted each other. Stand-ups were tough because people gave their quick summary to Clint and that was that.

And I couldn’t even access the team’s board for a couple of days.

Frustrating.

But finally, I got my access squared away. One of our DevOps guys, Steve, was great and patient at working through my access issues. Where he couldn’t directly set things up, he knew who to ask. And by Wednesday or Thursday of my first week, I was ready to do something more productive than chase down access!

Clint assigned me my first task. We had customers who were using a new brand of arc reactor management software (ARMS). I needed to add it to the list of ARMS types.

Cool.

So I dived in. I looked through the code. Yep, there were some ARMS packages in that list there. How did that affect how our system interacted with the customer? More digging. More looking. Not really clear. Finally, I asked Clint. It turned out I just needed to add that new ARMS package to the list, and that’d be it. Just add the string.

That was, again, frustrating. There was no way I could have known that. And maybe I could have asked questions ahead of time to figure that out, but until I dug in I don’t think I even had enough context to understand that there might be implications of adding a new ARMS package. I definitely didn’t have enough context to understand how adding a single string to a list that populates a drop-down could produce any value whatsoever for Marvel, LLC., or for

But after getting a few more access issues resolved, I finally, about two weeks in, got my first commit deployed.

Several weeks after I started, I got to go to my first all-engineering meeting. A lot of folks had their cameras off. There wasn’t a welcome for those of us that were new. That felt kind of sad.

I remember musing that help at Marvel, LLC., seemed to fall into one of two categories: “here’s a fish” or “here’s a fishing rod with a disassembled reel, and a map.” I don’t recall what prompted this—probably a culmination of time I’d spent tracking down access and help, and the difficulties I was having in getting to an understanding of the product and its context. But I do remember how it all felt.

Frustrating.

DC vs Marvel

Two onboarding experiences with two companies where I got to work with smart people from whom I learned a lot. Even allowing for the different company factors (e.g., DC, Inc. was much smaller than Marvel, LLC.), the two experiences were hugely different.

There are two questions I want to explore.

First, why did I feel frustrated and lonely at Marvel, LLC. even though there were people there to help—but not at DC, Inc. where I was mostly solo that first week? I think the answer may have something to do with what I worked on my first week at each.

Second, was I simply oversensitive to the Marvel, LLC. onboarding experience? How could we tell if that’s so? And if so, is there a way to construct an onboarding experience that doesn’t make Matt² grumpy, that still works for other people who might have enjoyed their Marvel, LLC. onboarding experiences?

By way of wrapping this up, I will mention that I have made a practice of taking notes during my onboarding experiences. Those initial days with fresh eyes are unusually valuable to a company or a manager. It’s easy to forget how difficult or frustrating an experience was³. Writing it down ensures the thought can be retrieved in its original form. If you’re starting with a company, or even switching roles, take notes.

At Marvel, LLC., I used my notes to offer suggestions to Tony, the CTO. I was able to share my experiences, both good and bad, and was given the project of improving the onboarding experience for future hires. Because I had notes, I was able to come up with specific items to improve.

But improvements to onboarding are a future post. This was the tale of two onboardings, and that tale has now reached its end!

1. The opening sentence of Seveneves by Neal Stephenson is “The moon blew up with no warning and for no apparent reason.” Action builds from there. [↩]
2. I gave everyone else superhero alter-ego names, so I feel like I should do the same for myself. But I won’t. Since, y’know, Matt happens to be my name and that might get confusing. [↩]
3. This quirk of memory may be responsible for the continuation of humanity, as the “let’s never ever do this again” feeling of waking up to deal with a crying baby fades into the feeling of “we can totally do that again, and y’know, I think we should!”. [↩]

I’m married and have three kids in the elementary and middle school age ranges. They’re doing things that normal kids do: growing, learning, being active, participating in sports and other activities. My wife and I both work. Our baseline level of “busy” is relatively high, as it is for any family with three kids at home.

Like most families, we experienced some changes starting around March 2020. Some of those changes reduced our level of “busy”—like activities and gatherings being canceled. But other changes increased our level of “busy”—like long quests to find rare and valuable items such as toilet paper. Still other changes shifted where the “busy” was applied—like virtual school for the kids and adapting to virtual teaching for my wife.

Somewhere in all of that, we had an unrelated medical issue come up. It mildly increased our level of “busy” for a while, but we did what we had to do and kept on with life.

A few months into the pandemic, Cedar Rapids (and much of Iowa) experienced a derecho. Much of the city, including my house, experienced sustained 100+ mph wind. We had substantial roof damage and substantial interior damage from water that got in through the roof and (closed, locked) windows. It was an amazing, terrifying storm. Neighbors had tree losses. Power was out for a week. So we did what we could. We helped neighbors clear trees. We thanked friends that did our laundry, provided a generator, and helped us move things out of bedrooms. We filed an insurance claim. We found contractors to fix things up.

With insurance difficulties, supply shortages, and busy contractors, repairs took a long time. That’s calendar time, sure, but also time spent coordinating.

We moved nearly everything in our house at least once during the process, if not more. First-world problems, to be sure, having so many things and so much space.

Meanwhile, the 2020-2021 school year started with all of its COVID precautions and stress and finally ended with most of the precautions lifted or ignored, with all the stress of stress of adaptation smeared through the entire school year. Summer came.

And in the summer my employer was acquired and eliminated my department, and I found another job, feeling fortunate to have landed on my feet.

And as I transitioned jobs, we finally finished the last of the interior work on the house.

And as the school year started, my wife headed back to the classroom with an increased workload because of the difficulty hiring teachers after a full school year of COVID. (There’s a whole separate case study there about how workload is shed onto those still present, and a whole slew of musing that can be (and already has been) done about how important schools are to our society and how that became obvious during the pandemic and how unrecognized and unthanked teachers are in all of this.)

During all of this, life hadn’t stopped. I’m happy about that! The kids kept growing, learning, spending time with friends, being involved in sports and other activities. We stayed involved in our community. As best we could, we kept connected with friends and family. Fall turned to winter.

Not all that long ago, winter, with all its snowpants and boots and coats and hats and gloves and snow shovels, finally gave up. As we started on the normal process of checking sizes and packing things away for the summer, we realized there was just too much. Not in the affluent Western “we have so much and there are those in the world that have barely anything” way. In the “wait, why are there so many pairs of boots and shoes here, and what’s in this box, and why do we still have these toddler-sized mittens?” way.

What happened, in short, is that our level of “busy” crept up (and occasionally spiked) so we didn’t have the time to do some of the normal maintenance we do: discarding things that are worn out, doing “spring cleaning”, having a garage sale every now and then, or just donating stuff that’s too small. We were—are—two or three years behind on basic life-and-stuff-maintenance.

Being that far behind cascades in interesting ways. We’re not just a normal “maintenance pass” away from being caught up. Normal maintenance is incremental, gradual, and part of the normal rhythm of life. We need something a bit bigger than that.

But even if a normal maintenance was enough, we’re not quite able to do normal maintenance right now. Some of the resources we need are gone. We have a storage area in our basement. We keep Christmas decorations, Thanksgiving decorations, Halloween buckets, a supply of plastic Easter eggs (for reasons that always seem good at the time), tools, rarely-used sports equipment, and other typical storage area stuff. It’s great workspace for sorting, or temporary storage space for donation boxes—under normal circumstances. Right now, it’s hugely overstuffed and disorganized. Things that should have been garage-saled in 2020 are piled up. So are hastily-packed boxes that came out of bedrooms after the derecho that we still need to sort. Things have been pulled out by people besides me that lack the reach or strength or inclination to put them back, and other things have been piled on those things. It is simply unusable as workspace or storage space right now.

Besides lacking resources to do some things, even what can be done now takes longer. Finding the bin for snowpants isn’t as simple as walking up to the shelf and pulling it down. It’s finding a pile that looks like it might have the right bin, disassembling the pile, opening the candidate bin, sighing, reassembling the pile, and repeating until found. Getting to my circular saw involves weaving through piles and squeezing past a few pieces of furniture.

Simple maintenance won’t cut it anymore. We need a project to get back to a good state. Actually, we need several. The storage area, sure. Also, the kitchen (the kids are big enough that we don’t need a cupboard for their sippy cups and toddler spoons anymore). The family room in the basement that has some bins stored in it. And other places around the house need attention for similar reasons.

We started in the garage, and got a quick and delightful win. Then we moved into the laundry room that does double-duty as our the entry room off the garage. That’s where the true depth of this problem hit me. Remember earlier when I referred back to snowpants and boots and shoes? We filled a contractor garbage bag with snowpants, boots, and shoes that are some combination of worn out, too small, and too damaged to be donated. We had a bunch of other junk we cleaned out of there too. And that’s in a room that’s a shade less than 9’x6′ and mostly occupied by a washing machine and dryer.

It’s no wonder I felt stressed every time I was in there, and no wonder I felt like it just wouldn’t stay clean. It was way beyond what normal maintenance, light sorting, and tidying could handle.

So what’s the point?

First of all, the point is not “oh, poor Matt and his family.” Yeah, stuff happened and things have been rough but we have amazing friends and everything is getting back to good.

The point is also not “stuff is bad” or “too much stuff is bad”. Stuff certainly can be a problem, but in this story, “stuff” and “things” are just a nice visible manifestation of something that happens all the time in other situations, but less visibly. So in this case, “stuff” is helpful in making the point.

So here’s the point.

A few years ago, my family was, like most families with multiple kids, operating close to capacity. Our baseline level of “busy” was high, and we certainly couldn’t have doubled our “busy” in any reasonable way. Operating at that state, we were keeping up okay with normal changes in our environment, like kids getting older and seasons changing. But then we had major disruptions: a pandemic, a weather disaster, trying to recover, and a job change. We naturally stopped doing “little things” to handle the new “big things” we had to deal with. Some of our normal life maintenance fit into the “little things” bucket that got dropped. Recently, we noticed that some of our systems had failed completely—and we hadn’t even realized they had failed years ago!

I’m going to take a second here to reframe this all in the world of work.

Companies almost always have systems and processes. That’s how stuff gets done. Maintenance systems are built around that. The software guys write the software and the TPS reports ensure that the software meets the requirements.

Usually, those systems and processes, including maintenance, respond to changes in the business environment. We realize we need cover sheets on the TPS reports. Bob gets promoted out of the department so Peter needs to take over stapler compliance checks. We double our headcount (yay growth!) or have a layoff (boo economy!). We adapt.

But sometimes, disruptions happen and we need to handle them. There’s an acquisition, or a key employee leaves, or a fire, or an important integration needs to be built quick to land a partnership. Sometimes multiple levels of disruption happen at the same time. The acquisition is delayed a bit and both Ross and Rachel go out on parental leave, and we need to move up the priority of that big project because our competitor just made an announcement.

In the moment, we feel some stress, but we know we’ll get through it. And when we do, it’ll all be okay because things will be back to normal and all our normal maintenance systems will clean up the loose ends.

And sometimes it’s true.

And sometimes it’s not.

Sometimes, the environment changes so much that our maintenance systems completely fail without anyone even noticing. Or, to put it more precisely, our maintenance systems continue to operate but no longer have the capacity to handle the necessary work; or they continue to operate but have not been adapted to the new reality and so give us a false sense of security. Or, to re-state it in terms of its effect, our maintenance systems completely fail without anyone noticing.

So there we are, with failed maintenance systems and only a vague awareness that badness is somehow piling up but nobody else quite seems to be worried, or at least nobody outside of the department is worried, or at least not worried enough to do anything about it, and anyway there’s work to be done so I should really go get some real work done instead of worrying I guess.

And that puts us in the situation where it doesn’t even take a crisis to break things. It just takes an unexpected-but-normally-okay increase in workload, or one more little change in the process, and everything grinds to a halt. The failures finally have cascaded to the point of breaking something noticeable, and often big.

So how do we stay out of that situation?

Well, I don’t think you should ask me. I ended up in this situation, remember? But since you’re still reading this, I’ll share a couple of ideas.

Start by paying attention to your feelings. Does it feel clunky or convoluted or like it’s just not working? I had that feeling when I looked at the piles of shoes on the laundry room floor next to the unused shoe baskets. Trust yourself.

If you feel like something’s not working, pay attention to the little details. What’s difficult about this thing that you’re feeling isn’t quite right? That difficulty might be a clue about what’s changed. One thing that changed in my laundry room was that there were more shoes. That could have clued me in that we hadn’t purged outgrown shoes in a while. Realizing that could have gotten me to the point of realizing that we had other maintenance tasks we hadn’t done in a while.

Be willing to start a project to fix it. There are probably two parts to the project. Most obvious is getting through the backlog (of shoes, or Trello cards, or Jira tickets, or paperwork…). Less obvious is adapting the maintenance process to the current environment. As a really simple example, my kids’ feet are bigger, so the shoe baskets we had weren’t working anymore. We needed a different way of organizing everyone’s shoes; just sticking with the old baskets would have at best re-started the clock on the problem.

Whether it’s a feeling of discontent every time you step into your laundry room, or feeling like your software deployment process just isn’t quite as good as it should be, or that nagging uncertainty about how much longer it takes to get those reports from accounting lately, or just a suspicion that your level of “busy” has been too high for too long—pay attention to it. It’s a signal that maybe, just maybe, a system that was close to capacity has been overwhelmed and failed. Your vague feeling may be the closest you’ll get to a warning before it cascades to something bigger.

This started as a blog post for the HI Digital Solutions blog, and never got posted for some reason. I’m pretty sure it had to do with it being drafted in April 2020. I think some things may have been happening around then. Anyway, I’m (re?) posting it here. Enjoy!

---

H.I. Digital Solutions (DS) has been working through improving our employee evaluation and growth processes. As part of exploring how we could improve, I scribbled a few questions on a sheet of paper. Then I started to answer them.

This somewhat unusual blog post is a very lightly-edited, typed-up version of five pages of longhand, in which the original few questions quickly turned into a written Socratic dialogue (in a non-strict sense). The topic being explored is “what should we pay for?” Put another way, what are the desired outcomes DS employees produce?

The question to address: How do I know whether I am providing value exceeding my cost?

What is value?

Systems that make or save money for the business.

Just those?

Well, also systems that give the business new capabilities, which are strategic or make or save money.

That’s vague, can you give me an example?

Let’s say there’s a market report that contains useful insight for your customer, and it takes two weeks to assemble. If you could deliver it daily (cutting lead time from two weeks to a few hours), that might be a new revenue stream – OR – it might establish your company as better-informed (therefore a market advantage) – OR – it might open up a new niche somehow.

Okay. So outcomes are tied to value?

Yes, explicitly, because we pay for outcomes. We pay for products from companies, and outcomes are the “products” of our employees.

That leads to the logical next question: What are the valuable outcomes to DS?

Like I said, systems that make, save money for, or give the business brand new capabilities.

What does that look like for software folks?

They’re the ones that construct the systems. They also design, test, and install them. And so they have to ask a lot of questions about what the business processes and outcomes are that are affected. That puts them in a good spot to suggest alternative approaches.

What do you mean by “alternative approaches”?

Sometimes the right thing to do is to write software. But sometimes, the right thing might just be to put up a sign, or cc Accounting on a regular report, or to use Excel.

What you’re saying is there are times when a software person might better serve the business by delivering a process recommendation, using existing tools?

Right! Or even recommending NOT doing the work.

Tell me more about what that means.

Let’s take a contrived example. Let’s say you have a monthly financial report that someone in the business wants daily. Your team explores the issue. It’s technically feasible. It’ll take M months and cost D dollars. In the course of investigating, you discern that the source data behind it only gets updated weekly, and the business already reviews and reacts to indications in the source data weekly. Your team estimates the business will only get 1/100 D value per year if you build the system to deliver the report daily. So the sensible thing is to recommend NOT doing the work.

If I knew Latin, I’d tell you that a contrived example can prove anything, in profound-sounding Latin. But you’re quite convincing. Let’s tie it back to outcomes, though. What are the valuable outcomes a software person might achieve for DS?

Running through what we’ve already said: (1) saving time and money by recommending NOT doing a project. (2) Delivering a system that saves money for the business. (3) Delivering a system that makes money for the business. (4) Delivering a system that gives the business a strategic capability.

Are 2-4 all the same? Delivering a system?

Maybe. Maybe they’re really (5) Delivering valuable software or systems and (6) Identifying business value.

Is “identifying business value” really an outcome?

Maybe not. You could identify but never act on it. Probably not.

Are there outcomes besides recommending (1) and delivering (2, 3, 4 and maybe 5)?

Oh! Sure! That’s growing the team, and growing the team.

Those sound remarkably similar to me.

Sorry. (7) Expanding the capability of team members to achieve valuable business outcomes (such as delivering software), which can be summarized as – but is different than (8) Expanding the team by recruiting, interviewing, and hiring new team members when capacity needs to be increased.

I see the difference. So, delivery, non-delivery, growth? What else?

“Delivery” might be slicing things a bit narrowly. It’s not just about handing over a floppy disk. It’s about delivering, installing, operating, monitoring. Or if your team isn’t responsible for all of that, keeping the teams that are happy. If the software is hard to monitor, your Ops counterparts will end up grumpy.

We should explore that more. But before we do, you mentioned other teams. Is it just other teams, or is there something more general here?

Yes! You’re right. There are other stakeholders too! There’s probably something about satisfied stakeholders. Let me think for a minute here.

While you’re thinking, can you also consider the tension between satisfying stakeholders and (1) NOT doing a project?

The outcome we’re probably after for satisfied stakeholders isn’t 100% rainbows and unicorns, is it? Sometimes things don’t go someone’s way, or a pet project gets squashed, or someone has to subordinate their desires to the bigger needs of the business. How about we start with a simple outcome that we can revisit? (9) Satisfying stakeholders so they want to work with you again.

Sure, I suppose. There might be a way to improve that, but as a starting point, I’ll go with it. Let’s go back to what you said earlier about other teams.

Other teams, like Ops or a help desk, or a downstream team, should be kept happy. The software needs to be valuable — do something valuable — but it also needs to be low-friction so people don’t dread it.

What do you mean by low-friction?

Think of what I already mentioned about the things like installing and monitoring. Plus there are things like usability — does using the software simplify life, or just move pain around? And for future software teams taking on the project, they will want to be able to modify it and deploy it with confidence.

There’s a lot there! Some of it—like users able to use it—feels like it falls into the category of satisfied stakeholders. The last part, about future teams, seems like there are two ways to go. You could tie it into delivery (2, 3, 4) somehow, or keep the focus on future teams. What do you think the best approach is?

To me this feels distinct from delivering a system, somehow. The “future” part of it makes me think the outcome we want here is related to sustained ability to keep producing results and being productive. Hmm. So there’s an aspect of sustainability for the work delivered, but probably also for the user.

Let’s try to tie off the “work delivered” part of this. What do you mean, and what’s the outcome?

It’s like I said, “sustainability for the work delivered.” After you ship 1.0, you should be able to keep working toward 2.0 without being bogged down in regression testing and struggles with deployments and all that.

So what’s the outcome you’re looking for?

Keeping it simple here, (10) Develops software that can be modified and deployed with confidence that it works.

Simple works. How about sustainability for the worker? What do you mean there? Rest breaks?

No, not that. I’ve heard it said that one mark of a good leader is that when they leave their team, it doesn’t fall apart without them. It shows they’ve invested, developed, improved things, and created a culture that does not depend on their involvement.

This sounds like a special case of (7)?

It does, with that lead-in. But that’s not what I was after. It ties in, though. You see, that good leader has some sort of system, not just luck. Or personality. So the team can continue without them, AND the leader can go on to a new team and replicate (or sustain!) their success. We want that sort of thing here too—people who can sustain good outcomes.

You might be getting into a meta-outcome?

Maybe. What if we keep it meta and try to phrase it (11) Consistently delivers desirable outcomes to their team and customers?

Is that too squishy? What you’re trying to get at is they sustain their work, but the “desirable” and the “and” make that harder to evaluate.

True enough. I want to say something about reducing entropy, but I just realized that doesn’t necessarily imply delivery, or satisfied stakeholders. Cleaner software and slightly less grumpy stakeholders isn’t a great outcome. So let’s simplify and go with (11a) Continues to deliver good outcomes—OR MAYBE—if we want self-growth (11b) Consistently delivers good and continually improving outcomes.

We can probably work with that. One of those options, anyway. That “entropy” idea is interesting. What do you mean by “reducing entropy”, and what does that look like?

Well, entropy is disorder in a system (“between gravity and entropy, we don’t stand a chance / everything goes downhill while it falls apart”). A good software engineer—or really, most employees except maybe demolitions technicians—reduce entropy by putting things in order: structures, systems, namespaces, useful vocabulary—and by solving problems that create disorder.

So you’re talking technical entropy, code and build systems and the like?

Huh. I was. But now that you mention it, there’s non-technical entropy. Assumptions that don’t hold, processes that could be unified and simplified but aren’t, disagreements among business areas. And maybe it’s not technically entropy, but ossified processes that slow work down and might cause work disruption. I’m thinking projects that take longer because nobody wants to interface with a particular group—so they route around it and there’s turbulence, like a rock in a stream.

Reducing this non-technical entropy, what does that look like? Can you articulate an outcome that’s not subsumed in, say, (2, 3, 4, 5)? Or (9, 10, 11)?

Yeah, I’m not sure. The combination of delivering value plus satisfying a broad range of stakeholders probably captures it. Maybe it needs to be made explicitly, but I think it’s hard to sustainably deliver valuable software, satisfying your stakeholders, if you’re not reducing non-technical entropy—both inside and outside your team.

Does that same argument apply to technical entropy?

It might! Let’s take, say, TDD or build systems. You can deliver software without these. Maybe even repeatedly. But without these kinds of things—things that solve problems of correctness, repeatability, modifiability—you start to reduce velocity. I assume.

You mentioned solving problems just now, and you’d mentioned it earlier. Is there something there we should pull out?

I’m pretty sure some people would argue that “solving problems” is what engineers do. Some do it with computers, some with roads. Building software, debugging, and even deciding NOT to do a project all fit under problem solving. Is that too big to be an outcome?

Flip that around. What might it look like if it were an outcome? What does problem solving look like as an outcome?

Just saying “solve problems” isn’t enough, probably. How about (12) Creatively solves business problems with the long-term interest of the business in mind?

That sounds promising. Does it overlap with our earlier outcomes?

Probably, but it takes a different approach. It seems distinct, or at least different.

Could the earlier ones fit into (12)?

I’m going to argue there’s enough of a difference that we should keep it separate for now. Besides, if you look at it broadly enough, it might be the only outcome, with even (7) and (8) falling into it.

It seems we’re wrapping up. What else? Any other outcomes we’re missing?

Is there anything around working transparently? Or might that be a value, or a method for achieving outcomes? Probably, I suppose. Same for collaboration. It supports outcomes. Same for things like ethics—that’s stakeholders and sustainability. And leadership probably is also a tool or method.

So we went from wanting to know how a software engineer can “deliver value exceeding my cost” to a short list of outcomes. Is that satisfying?

I think so. I think there’s enough granularity in the outcomes to make distinctions between skillsets and skill levels. And they’re able to be decomposed so we can help someone build the skills to achieve those outcomes. That’s probably more work—future work. Thanks for your help!

You do realize you’ve just been talking to yourself, right?

Yep. Well, writing to myself. Thank you anyway, self!

---

Outcomes “Discussed” Throughout

For convenience, I’ve gathered all the items discussed throughout this post here. Recall that as the “discussion” went on, several items were crossed off the list, decomposed, or rephrased.

• (1) Saving time and money by recommending NOT doing a project.
• (2) Delivering a system that saves money for the business.
• (3) Delivering a system that makes money for the business.
• (4) Delivering a system that gives the business a strategic capability.
• (5) Delivering valuable software or systems.
• (6) Identifying business value.
• (7) Expanding the capability of team members to achieve valuable business outcomes (such as delivering software).
• (8) Expanding the team by recruiting, interviewing, and hiring new team members when capacity needs to be increased.
• (9) Satisfying stakeholders so they want to work with you again.
• (10) Develops software that can be modified and deployed with confidence that it works.
• (11) Consistently delivers desirable outcomes to their team and customers.
• (11a) Continues to deliver good outcomes.
• (11b) Consistently delivers good and continually improving outcomes.
• (12) Creatively solves business problems with the long-term interest of the business in mind.

My team uses Postman a lot.  We store our collections in git.  But we don’t want to store our secrets there.  Postman is kind enough to allow users to create environments to hold those secrets.

Even with environments, the need to keep secrets secret gives us a configuration problem.  Setting up a new environment involves some clicking around into Azure keyvaults and making sure we’ve given the right secret the right name.  It can be tedious.  And that’s if we remembered to document where each key lives.  If not, it can be downright frustrating!

Not my favorite way to spend my time.

So I put together a little tool to do the hard work.  The idea is to document the environment in our markdown documentation, point a tool at it, and have it create an environment suitable for pasting into Postman.

Pretty great, huh?

Well, that’s really just the intro to what I want to write about, which is the piece of code that pulls the environment table out of a markdown file.  Excited yet? 

It’s a story of two refactorings (and an initial assumption that turned out to be pretty good), of turning spaghetti logic into a nice tight piece of code, and trying to make the code as communicative as possible.

Act One:  An initial assumption, and some spaghetti.

One of my design goals for this project was that it would be self-contained.  I didn’t want to pull in external libraries to parse markdown.  That was okay, because as a specialized tool, the code to extract data from markdown could be fairly dumb.

Based on past experience, I made an assumption:  I would need to keep track of named states, as opposed to a basic approach of comparing to previous lines, or the status of a handful of variables and flags.  It’s easy to get into trouble when you’re trying to understand state based on multiple conditions at the same time:

if len(rows) == 0 and found_label_header and is_table_row(current_line) and not finished_target_table:  # Even with good variable names, what does this even mean?

Compare that to: Easy to get from there to “Therefore, we’re starting to process our starting table!”

if state == STATES.TARGET_LABEL_FOUND and is_table_row(current_line):  # Oh.  If we've found a header containing the label we're looking for, but not started in on the table, and the current line is a table row

Easy to get from there to “Therefore, we’re starting to process our starting table!”

(Note:  it’s not how I wrote the code, so I can’t be certain, but I’m pretty sure there’s a bug in the first (non-state) example.  What if we’d already been through a table so rows was non-empty?  And that’s how it’s easy to get in trouble.)

So one of the very first things I did was to sketch out a state diagram.

Lovely, if barely legible.  The scribbled-out parts make it even lovelier.  (Then again, how much time did I save by realizing I was modeling things wrong while it was still on paper, as compared to a few dozen lines of code in?)

Armed with a pretty good idea of what the states were, what caused each change in state, and an editor, I started to test-drive some code.  After a while, and a little bit of refinement of the state list, the code looked a little something like this:https://github.com/mattschoutends/postman-environment-builder/commit/ea0cedb49d68947c01beb46631d57d839dd38e71 (Actually, it looked exactly like that.)

That’s pretty clean.  A defined list of states, a couple of well-named helper functions (ignore the fact that is_table_line is actually wrong, which I realized embarrassingly late in the development process), and a few well-defined nested conditionals.  Not bad.

Then again, looking at the “well-defined nested conditionals” was starting to feel like looking at spaghetti logic. 

Act Two:  A refactor!

Not only was it starting to feel a little like spaghetti, I felt like I was seeing a repetitive structure.  Based on the starting state, the code checked some condition.  If that condition was met, three things might happen:

1. The state might change
2. The table_text list might be cleared
3. The table_text list might have the current line appended

(If #2 happened, #3 would also happen.  It happened all on one line, but keeping those in mind as separate operations could be useful.)

I asked myself if it would be possible to represent the repetitive structure as a data structure.  It turned out to be pretty easy.  I was already thinking about rows and columns, since I was extracting tables.  I made a list of column labels, and a bunch of state transition “rows”.  The rows contained:

1. Current state
2. Condition (a function name)
3. State to transition to
4. Should the program clear the table_text list?
5. Should the program append the current line to the table_text list?

A little bookkeeping work turned the relatively human-readable table into a useful data structure for the program to use (a dictionary keyed by starting states, where the values  are lists of transition “rows” converted to nicely-labeled dictionaries that the program can check).

What had been about 35 lines of nested conditionals turned into a half-dozen lines of code.  (With one unfortunate bug that, with the set of state transitions at the time, never got triggered.  Spoiler:  It’s using continue instead of breakon line 7) https://github.com/mattschoutends/postman-environment-builder/commit/5a76ce7b58ae1a84d7b212c6af8823c4c65c7fa9

for transition in possible_transitions:      
    if (transition[TRANSITION_LABELS.CONDITION_FUNCTION](line)):      
        state = transition[TRANSITION_LABELS.NEXT_STATE]      
        if transition[TRANSITION_LABELS.SHOULD_START_NEW_TEXT]: table_text = []      
        if transition[TRANSITION_LABELS.SHOULD_APPEND_TEXT]: table_text.append(line)      

        continue          
 
    if state not in TRANSITIONS:      
        break

Not bad!  The bookkeeping code makes it a bit longer than I’d like, but still.  Not too shabby!

Well, mostly.  Those bare True and False items in the table bothered me.  They’re not exactly hard to follow, but they’re also not very expressive.  It wasn’t bad, I thought.  Not with so few state transitions.

Remember when I said is_table_line was actually wrong?  Well, I’d started writing my tests with some bad test data.  I’d separated my table headers from the table body with a line of dashes only (------), not dashes and pipes (| --- | --- |).  As I fixed my test data and started fixing code, I needed to add a few state transitions.

And suddenly, that little bit of smell with the bare True and False got much harder to look at.

Act Three:  Another Refactor!

With a few more rows, the raggedy trailing edge of the line felt like it was hiding behavior.  This was especially true when it took a while to figure out a bug.  There was a copy/pasted line with a False that should have been `True`, and it was really hard to spot, hiding in the middle of a line.

I wanted to change the transition “table” to be even more expressive, and to highlight the places where things really happened.  It seemed like it would be useful to distinguish between lines that just marked time and lines that did something to the list.

So that’s what I did.  Instead of leaving the table as a list of lists, I turned it into a list of dictionaries generated by function calls.

Which, reading it, sounds pretty unhelpful.

But when looking at the code, it’s the difference between

[STATE.IN_TABLE, is_table_dash_row, STATE.ON_TABLE_DASH_ROW, False, True],
[STATE.IN_TABLE, not_table_line, STATE.FINISHED_UNLABELED_TABLE, False, False]

and

transition(STATE.IN_TABLE, is_table_dash_row, STATE.ON_TABLE_DASH_ROW, append_line=True),
transition(STATE.IN_TABLE, not_table_line, STATE.FINISHED_UNLABELED_TABLE)

Using default parameters in the function meant that all the False values could be left out.

The function also had the nice benefit of simplifying some of the bookkeeping code.  Dictionary keys could be set, so the list of labels (used for keys) could go away. 

Conclusion

I’m pretty happy with how this chunk of code turned out, relative to the design goals I had for it.  It’s relatively small, concise, and makes the intent easy to extract. 

The key refactor was Act Two, making state processing a deliberate abstraction, instead of something implicit and accidental in a mass of spaghetti.  Once the state processing was separated from the state transitions, it was suddenly really easy to see (1) what the state transitions were and (2) what the program might do in each state.

Was it worth the extra time for the refactoring?  Yes.  The first refactoring made it much easier to add states when needed.  The second would have made it much easier to find the copy/paste bug that forgot to clear the list.

In the spirit of surveys like the DORA survey or Stack Overflow’s Developer Survey, it’s time for the next round of the State of Work survey.

A lot of things have changed since June: kids have gone back to school, sports leagues have gone back to doing sports, offices have re-opened, and people have adapted.

But broad generalizations and observations about things that have happened don’t help much with understanding how people are truly doing.

I want to hear a about your experiences so I can get a better idea of how life really looks for everyone. I want to hear from you: employed or not, affected a lot or a little, in the U.S. or not, urban or rural, I’d love to have your input! I anticipate it taking about 10 minutes to fill out the survey, unless you’re going to write a lot.

The second round of the survey focuses on work. It also asks about things that have changed. If you didn’t take the previous version, that’s just fine. Feel free to start with this one!

Your individual responses are 100% confidential.

If you are interested in finding out what I learn, I’ll send a report to anyone who has filled out the survey after I’ve gone through all the responses. I’m planning on finishing analysis in November 2020.

One final thing: to get the broadest possible reach, would you please share the survey with your friends, colleagues, and the rest of your network?

The State of Work Survey for June 2020 will be open for responses through Saturday, June 13.