I don’t think it’s wildly off base to assume that at least some of us in DevOps are attracted to the work and the methodology behind it for reasons relating to a partular mindset and problem-solving approach associated with doing what we do.
You might recognize the title of this post as an allusion to the expression, “couldn’t see the forest for the trees” – a situation in which one is so bogged down in the details of a particular problem that one misses or ignores the problem context in whole or in part, leading to an inelegant or incomplete solution.
A lot of what we do, or at least what I do, is “seeing the forest,” i.e., how all the pieces of a software system interact, identifying pain points and bottlenecks, breaking down the problem into chunks that can be solved step-by-step and then putting it all back together. Across my career I’ve encountered a diverse set of problem-solving strategies. Some strategies produce solutions that are generalizable and elegant. Others lead to solutions that are very problem-specific and hacky, barely squeaking past the minimum viable goal post while getting the job done in record time. While all of these strategies have come in handy at some point, I think DevOps work has a clear preference for the former as an intrinsic part of the methodology.
Maybe this is too much of a generalization and a near-perfect example of the fundamental attribution error, but I find myself occasioanlly associating problem-solving strategies with personality types. If I may make a binary distinction, I propose that there exist two poles of a spectrum: “tree people” and “forest people” (with the obvious caveat that most real people lie somewhere in between). Tree people like to approach the particular, and are very good at e.g. enumerating a series of sequential tasks. Think imperative programming, algorithms engineers, data scientists. Tree people tend to build one-off solutions that cannot easily be adapted or customized to other similar problems, but they can iterate quickly with little to no up-front planning cost. Tree people roll up their sleeves and start hacking away at writing code, often rewriting the whole thing several times before arriving at a working solution. Tree people like REPLs, Jupyter notebooks, and code frameworks that support hot reloading. Copy-pasting code doesn’t bother them too much, as long as it works.
On the other extreme are forest people, who easily get lost in the specifics of any particular problem. Without having a solid working knowledge of a system overview, forest people struggle to find an entry point from where to begin. On the other hand, forest people excel at seeing the big-picture view of a system’s architecture. They prefer to invest time in building a one-size-fits-all Grand Unified Solution that takes more time to implement, but the investment often pays for itself when the inevitable similar-but-slightly-different-problem arises down the line. Forest people like sequence diagrams, architecture schematics, and flow charts. Forest people like working with declarative configuration management and infrastructure-as-code tools modelled after object-oriented programming concepts that encourage code reuse. Writing any single line of code more than once is likely to result in some grimacing, at the very least.
As an engineering student (mechanical), I often struggled working through problem sets when I was missing the conceptual framework. Whenever I worked with peers, and they were powering through some problem set, I would occasionally stop them and ask something along the lines of, “how did you know to plug this value x into that formula y?” They would respond with a blank stare, accompanied by “uh, that’s just what you do to get the answer.” This sort of rote memorization of a sequence of steps drove me insane. I could not work through a problem set without understanding the concepts involved, and I could never understand those who had no issue doing so. Students who studied by cramming problem sets and hoping that at least some of the problems they saw last night would be on the exam the next morning tended to do pretty poorly when that wasn’t the case; on the other hand, the converse was also true. I felt like I was part of a tiny minority of students who was actually interested in why the formulas worked, not just as a learning exercise but to do engineering.
I ended up dropping out after my third year.
Of course, any good, experienced engineer has acquired a lot of tools over their career, and also knows how and when (and, perhaps more importantly, how and when not) to use each one. Sometimes you have to meet deadlines, and there’s just no time to implement Grand Unified Solution before the end of the sprint. Sometimes, the exam really is tomorrow morning, and you have to cram problem sets hoping something sticks for lack of any better strategy. But in the long run, you’re going to forget which numbers got plugged into which formulas, and if you ever encounter a similar problem in real life, you may miss the forest for the trees, and re-invent the wheel once again, this time with a slightly different diameter. In the software world, your repo of 93 undocumented shell scripts in the root directory with names like
cleanup_indexes-3.sh that only run on that one Ubuntu 16.04 server is eventually going to give the person who replaces you a heart attack.
I think every one of us who works in software has a preference for doing things the Tree way or the Forest way. I’m making the case that DevOps practitioners are much more inclined to be forest people - not just in our professional work, but also in our general approach to life. We prefer concepts and ideas over step-by-step howtos. We tend to remember narratives and personal stories in terms of motivating factors and circumstances rather than a temporal sequence of events. We wince at the thought of rote task completion, always opting for automation whenever and wherever feasible, not out of laziness but out of a desire for predictability, consistency and reproducibility. But we are also susceptible to ignore outliers, even when they are statistically significant. We run the risk of over-engineering, spending time and effort building perfect mousetraps that could (and should) be spent elsewhere. Unlike tree people who are busy counting leaves, we tend to spend so much time thinking in terms of forests that we forget that forests themselves are comprised of trees. The end result is often a generalized solution that in theory should work for every case, but in practice works for no case in particular.
My advice for fellow forest people: embrace your systematizing tendencies to build beautiful, resilient systems, but don’t overdo it. Be cognizant of the fact that the tree people in your life probably see things very differently from the way you do. Make concerted effort to provide relatable, concrete examples for why doing and seeing things your way sometimes pays off in the long run. Also, sometimes you might be wrong, and in some particular instances, just grit your teeth and do it the Tree way, and be done with it.