I was inspired by an article on Management README’s and thought it was high time I do something similar. This is mostly for folks I work with, but I hope it’s useful to anyone who interacts with me. I try to keep it up to date, but it may be six-plus months out of date. Hopefully I don’t change that much and it’s only just me gaining more insight about myself! [UPDATE: edited in March 2021 after moving to AWS].
My name is Greg Herlein. I am an Engineering Leader and Maker based in San Francisco California. I am active on LinkedIn and sometimes on Twitter. I keep my open source work on Github. I don’t use Facebook anymore - too many trolls, and I’m not a fan of how algorithms have turned us all against each other to make a buck. The genii is out of that bottle but I don’t have the energy to deal with it anymore. I can barely handle twitter these days.
I’m a linux enthusiast and have been since before the 1.0 kernel. I’ve contributed some small things to the linux kernel, including some very early RS-485 bug fixes and co-designing the Linux Telephony API. That effort never actually amounted to much, but we tried. I also do electronics and am decent building a variety of things with my hands. I’ll talk more about my geeky efforts below.
I work at Amazon AWS managing teams of Solution Architects. We work to help customers understand how to best use AWS to solve their business problems. I’m not actively building a product anymore - rather I am helping many customers build their products and use the cloud to run their business. I love it, but I do miss building a product.
My Management Style
It’s hard to summarize my style, but these three points nearly capture it:
- Hire people smarter than me and empower them to do great things, removing obstacles along the way
- Ensure that my teams understand the bigger picture and that we are doing the “right work” (as opposed to “doing the work right”)
- Carefully compose teams, reinforce them, and craft a leadership pipeline of technical and management talent
I believe in hiring people smarter than me and then supporting them. This leads me to what I look for most in folks: good judgment. This is very hard to detect in an interview, but critical if you are going to actually delegate and let folks make their own decisions. Keeping decision-making too centralized and you slow down. You have to delegate. The Amazon interview is heavily focused on behavioral questions - the “tell me about a time” approach. I strongly favor this. A history of making good decisions is a pretty good indicator you’ll make more good ones in the future.
A key part of leadership is making sure everyone understands what’s been delegated, who has what roles and where their responsibility lies. This isn’t usually very technical but it might be the most important thing I do. This requires that I have good judgment too. I need to have the ability to dive deep into a problem or solution and contribute my experience and wisdom, but not get lost into the details. It’s critical for me to not “miss the forest for the trees.” This is the balancing act that is the hardest at my level, because usually the technical details are far more fun than the bigger picture. My most important job is to see what’s coming, and plan to eliminate roadblocks or glitches that would slow down the team. I can’t see around corners if I’m head-deep in the code. But it’s also important that I not get too far from the code. I believe that “trust but verify” is good policy all around, especially in complex systems. Admiral Rickover, the father of the Nuclear Navy, is reputed to have said “you get what you inspect, not what you merely expect.” If a leader blindly assumes that everything is going just fine then the reality may be drifting and you don’t know it. Even if you don’t directly inspect it, you can have a process that can ring an alarm when something is not right. Delegation of authority does not mean I’m no longer responsible. It just means I am responsible to find a way to inspect the results. Because at the end of the day, the leader is responsible for the results.
Another major principle I lead by is to make sure we know what we are building and why it’s important - and that we have all that we need to do it. It’s so easy to go down a side street, or get caught up in work that is interesting but not strategically important to where we want to go as an organization. Build the right thing. And if the teams know why it’s right then they make better decisions. This means that as a leader I need to also understand, deeply, what the problem really is. That usually means directly talking to customers. I don’t believe that only Product Management should be talking to customers. Engineering needs to as well or you get what I call “tree swing results.”
This changes a bit now that I am not leading a team directly building a product. It’s harder. It reminds me of pointers to pointers in C: the actual work product is being done by AWS customers and my team is helping them. Anything I do that tries to improve my team has an indirect effect on the end customer. This is OK, of course. I just have to think that way and realize that I am not aiming at one product (or product line) but instead at many, many product and businesses. It’s a big, exciting challenge.
Building teams is more complicated than building software because humans are complicated. Good Engineers are often very complicated. That’s usuall what makes them good! A former boss said that I “make teams that don’t break.” That’s the nicest thing anyone has ever said about my management skills. I take the time and effort to identify talent and nurture it. I am proud that many folks who worked with me over the years are now either key technical leaders or Directors or VP’s in the industry. My leadership pipeline works. I am proud to still get to work with some of those people several companies later. And many of that group gather a few times per year for a pint and some laughs. I’m so proud and happy about that. But this takes work and effort. I strongly believe that at my level my job is to craft the team that makes the product, not just crafting the product. That’s important. My most important deliverable is the overall team. A lot of folks miss that distinction. It’s important to think about team building like you think about software building. I’ve blogged about this.
My Operational Style
I try to adhere to DevOps principles - read some about it here. I believe that you have to know two important things:
- What does “normal” looks like?
- How do you know when you are “done?” - what is your “definition of done?”
If you fail to define those you will have more work and lots more stress. And quite often the “pefect is the enemy of the good enough to ship.” It’s usually better to ship something and get feedback - that is, if you can fix it live. Embedded systems are a whole different beast. Ship something and measure it. Get customer feedback. Instrument it so that you know how it’s behaving. Understand how it works intimately. And that leads to the ability to decide when it’s “done.” Software is never done - it’s like cleaning dishes: it’s only done if you never plan on using it again. But “done enough” means having the right features (which your customers will tell you) and having it work well enough (which your observability will tell you). You should be measuring a graphing all your key signals.
One really interesting thing in my new role is thinking about what signals matter. Like I said above, our work product is indirect. What can we measure to know if we are having the desired impact? Expect a lot of blog posts this year about that.
Now, it’s easy to expect too much. It’s important to find a balance. I expect a LOT of myself and those who work with and around me. I think teams can, in general, do more than they think they can. But I strongly believe that there’s a huge risk of diminishing returns if you work too much. Humans need time away from hard work. Making software is as much a creative effort as is it a thinking effort. Tired Engineers lose the creativity. Worse, tired leads to mistakes and bad judgment. I try really hard to make sure my teams are not over-worked. It’s bad for the people and bad for the product we are building. And bad for everyone’s real life. We should all work to live more than live to work. This just reinforces my belief that we should be making software do the heavy lifting. Systems should self-heal. I work towards the “reduction of toil” that the Google SRE books talk about. Most good Engineering teams will happy work even harder to build solutions that eliminate toil. And in Solution Architecture, there’s a lot of toil. What do we do for every customer that we can automate? AWS Solution Architects are builders: we can automate things. We just need to realize which things are the right ones to automate!
My Communication Style
I’m very direct. If you don’t know me you might think I am too direct. If it bothers you please talk to me. I try to learn from how I come across to people. But the flip side is you can be very direct with me. Just say it. We’re adults. I value open and honest communication. AWS is an environment that values this too, and I definitely feel more comfortable. It’s all about data, and making good decisions, and focusing on doing the right thing for the customer.
I also place huge value on personal integrity and honesty. Facts matter. It’s Engineering, not fictional literature. The fastest way to lose me is to play loose with the truth. Especially bad news. It’s like cheap wine: it does not age well. Honest, direct communication. Probably what I value most. We all make mistakes. When we make them it’s most important that we admit it and learn from them. I got my start in Engineering in the US Navy Nuclear Power program. The founder of that program, the father of the Nuclear Navy Admiral Rickover said that “success teaches us nothing, only failure teaches.” I embrace that, and try not to make the same mistakes twice!
I am also heavily biased towards action. A less nice way to say that is that I can be impatient. Time is the one thing you can never get more of. Money yes, resources yes. Time No. Once time slips past it’s gone. But please say something if you think I’m too impatient. As I said above, facts matter. Sometimes I don’t have all the facts I need and I might make a decision too early. I am very good at making decisions and moving foward with lots of ambiguity and only part of the facts. Most of the time that’s a strength. Sometimes not. It’s a learning process. But I do generally bias towards action rather than waiting for more information.
One of the side effects of this is that if you work with me you need to speak up! If you don’t you may keep me from having facts we need to make good decisions. I find that Senior Engineers have no problem with this. Many less experienced Engineers can be intimidated, or shy, or just careful and don’t speak up. You should speak up though! I am no different than a lot of experienced Engineers: I have a strong personality. Don’t let that quiet you. I was just like you not all that long ago. Speak up, even if it’s just to ask questions. Conversation is how we all learn. The only stupid question is the one you didn’t ask. Genuine curiosity and inquisitiveness are the bedrock of a good team. And I can tell you some stories about folks asking “wait, what about…” and saving the team. Questions are the essence of Engineering.
My Working Style
I use a lot of async communication (especially email) because I can multi-task well (mostly) and my life often has more meetings than I wish I had. Some people like my async style. Some people prefer to talk directly. Please ask for what you need from me. Otherwise I am likely to revert to emails thinking that everything is working fine.
I believe in stack-ranked lists. Yes, I tend towards kanban as a workflow management tool. But I will use whatever works. Often the best approach depends on the team and their dynamics. But regardless of tool/methodology/approach I strongly believe in knowing your priorities. I borrowed a tool from Rackspace (where I worked for a while) called the “3P Reports” and added Priorities and call it the “4P Report.” The P’s are Priorites, Progress, Plans, and Problems. These are usually one liners. Plans should map to Priorities and Progress should map to last week’s Plans. Problems are not excuses for failing to meet your Plans - they are things that you need help with. These are public. My whole team can see everyone elses 4P Reports. Transparency leads to sharing of facts. Facts Matter. See above.
I use email alot. I often do email at odd hours and on weekends. I do NOT expect anyone else to do this, and I do NOT expect instant replies. Email is for async comms. If a fast reply is expected, IM or Text Messages are better. Those generate interupts and can be expected to be serviced fairly quickly. Email? A day or so, generally. That’s what I expect, and that’s what you can expect from me. If it’s on fire, call me. That’s what phones were invented for.
I use IM a lot as well, in bursts. My teams have used Slack a lot, of course. Cisco used WebEx Teams, Amazon uses Chime. I’ve also used WhatsApp, and Viber, and iMessage, and… you get the point. IM is for things you need a faster reply to, email is for expected async answers.
Monocultures have blind spots. Diversity leads to better results for many reasons, but one of them is that different people bring different viewpoints to a problem and solutions tend to be better on diverse teams. I’ve seen this for my whole career. Beware: diversity means more than skin color or gender or country of origin. It also means age, culture, socioeconomic background and more. A whole team of about the same age all from the same college may LOOK diverse but might not be. Beware monocultures. I embrace diversity and actively seek different perspectives.
I am especially looking to increase the number of qualified female Engineers on my teams. This is not just because it’s the right thing to do (it is) but because in my years of experience teams with women get more done and are happier than teams without. Somehow those teams communicate better, understand better, work together better. It’s good business. And it’s the right thing to do. I have a daughter. I’ve seen how biases work against girls in the educational system. I’ve seen it in the workplace. So yes, I do all I can to be a positive force for change. But flat out: teams with qualified women on them perform better. ‘Nuff said. Besides, some of the best managers I have ever known are women. Percentage-wise, far more than men. Gotta bet on success, you know?
Plus,it’s the right thing to do. And frankly, the tech industry has a lot of work to do in this area. There’s a ton of terrible stories about what women in tech have to deal with just to do their job. Similar problem for people of color. I don’t tolerate discriminatory behavior and neither should any of you. It’s not just a “management problem” to solve. It takes everyone to stand up for equality and diversity.
Praise and Criticism
I strongly believe that proposed solutions deserve intense scrutiny and valid critcism. People, not so much. Behavior yes. People no. Too often I see criticism of a person actually be direct or biased discrimination. I don’t tolerate discrimination cloaked as criticism. I’ve seen teams where ideas are shared and shot down because of who proposed it rather than on the merits of the idea. I don’t tolerate that. The best ideas win. That said, if you propose something that you have not thought through you can expect my teams and I to be pretty vocal about it. Steel is not forged in temperate heat: the best steel comes from a very hot fire. Ideas and designs are no different. Good Engineering comes about because the designs are looked at from all angles and rigorously evaluated. It’s not for the faint of heart.
I believe in the old adage “Praise in public and criticize in private.” I strongly celebrate successes and I love to do that publically. I actually think that most Engineers really like the admiration of their peers. That’s why many contribute to open source. Showcasing what people build is incredibly powerful. Criticism is public is the fastest way to destroy a team. This is a balance with the statement I made last paragraph. It’s vital to ask questions, to challenge, to inspect. But if it comes across as criticism that can undermine everything you are trying to accomplish. I’m thinking about this a lot these days, so that I am mindful. Maya Angelou said “I’ve learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel.”
Traits I Consider Most Important
Good Judgment. Hands down. We assume a certain level of intelligence in Engineers. Skills can be acquired. Knowledge can be gained. Good Judgment is something you have, or you don’t have. As I said above, it’s hard to test for in interviews. Some folks have OK judgment normally that totally vanishes under stress, or when tired. Some folks just seem to make good decisions no matter what. Those are keepers. Folks don’t need to always be right. It’s not about being right. It’s “did you make the best decision you could with the information you had at the time.” More importantly: did you make a decision at the right time?
After judgement comes curiosity. If you are a learner, you probably have the right mindset to be adaptable. The only constant is change. Learners adapt. Tomorrow’s technology is about to hit us in the face. Can you learn how to use it today?
After that comes being a maker. Makers make things. It does not have to be software. It might be sewing. Or metalsmithing. Or pottery. Or robots. Or glassblowing. Or cooking. Or poetry. In my experience the best Engineers create things. It’s a trait that goes well beyond fingers on keyboard with an open editor. Try it. It will make you a better Engineer.
One thing that does not work for me is a sense of “that isn’t my job.” There’s no “I” in team, as they say, and sometimes everyone has to do things they don’t like. I try to align the work to the skills and desires, but a highly functioning team is one where everyone pitches in and gets it done. The most important thing is to “ship it.”
Lastly, while extraordinary effort is always appreciated, I don’t value heroes in the workplace. CMM Level 1 organizations succeed “based on the heroics of individuals.” I try to build organizations that don’t need heroes to succeed. Team effort. Mutually reinforcing team members, prior planning, clean execution. Software is hard enough already. No need to sprinkle the need for heroics into it. My job is to build an organization that can delver the product. If I need heroes to deliver then I have not done my job. Period.
A common AWS interview question is “what is your favorite Amazon Leadership Principle? While there are many I like a lot (Invent and Simplify, Think Big, Learn and Be Curious, Hire and Develop the Best) I have to say that “Insist on the Highest Standards” is my favorite. It’s also key to understanding me. I’m not a perfectionist (really!). I just have high standards. I do credit the US Navy Submarine/Navy Nuclear Power experience I had in my early career for that. If you lower your standards, you get even lower than that. I hold myself and those around me to very high standards. This is a cognitive dissonance because I also beleive in “good enough to ship.” But that’s OK. Good enough and iterating is how you get to the highest quality product in the long run.
If you interact with me you are likely to sense me pushing. Better. Faster. Clearer. Simpler. Stronger. Always. This is a mentality as much as anything. It’s not personal, and if I bother you with it I apologize. Well, not really. I’m sorry it bothers you. It’s the trait about myself that has led to most of my success. I can’t apologize for that.
My Personality - Real and Apparent
I am told I come across as an extrovert. Nothing could be farther from the truth. I am very much an introvert. I love to work alone, curl up with a good book or cuddle on the couch with my wife and watch a good movie. I generally don’t like to go out, or to parties, or even walk down a crowded street.
That said, when I decided to manage teams I had to learn how humans worked. And that meant learning to be social, learning to go to parties, to talk, to have a drink or a meal together, to do public speaking, to show personality. But all that takes a huge amount of energy from me.
Even though I am an introvert I’ve learned that I learn so much from spending time with smart people. I love to hang out with my teams, especially teams from former companies who still get together often. I especially love to be with customers and partners. I’ll trade energy for learning, anytime. So while I am very much an introvert, I’m a learner more. It’s an energy investment that I happily choose to make.
My Interests and Projects
Managing at my level means I don’t get to do code on a regular basis which makes my side projects all the more important to my sanity. I can’t imagine not coding. The fact that my son is now asking me for help on coding homework for college is a dream come true!
I do geeky stuff on my own time. I wich I had more time to build circuits and robots, and especially to write more code. I do home repair on my very old house. I am slowly moving towards some home automation using MQTT-based electric switches using Sonoff and Shelly devices reflashed with Tasmota. And, getting ready to connect all that with AWS Greengrass IoT. I consider myself a maker. I just wish I had more time!
I’m an emacs guy. Sorry if you’re a vi person. My favorite language to code in is C. It’s got a purity I cannot stop loving. I’m learning golang, still. A few years back I wrote a network use visualization tool called gonetmon that works with prometheus and a CLI tool to lookup MAC address vendors called ouilookup. I was working on a robotics project that uses MQTT but it’s not done. It used an XBox™ controller module and controller and a forked version of pi-blaster. I’ve also been playing with some low cost radios. I also was working on a robotics platform called RBOT, but that’s also not finished. I just got my PineCube and now just need some time to play with it. Which I should put on hold until I play more with my RPi4 Zero but I’m wondering if the PineCube will be a better solution.
You see, I’m better at starting projects than finishing them. I have a dozen partly done things on my workbench at any given time. This is because I get the value out of LEARNING by doing, not out of FINISHING things and then using them. This is not universally true, but more true than not. This is also probably why I push so hard at work to SHIP IT. I know that my personal tendency would be to keep tweaking it. In a commercial setting, I know I have to balance that out.
Personal Stuff, Background, Etc.
I live in San Francisco in a really old house in Cow Hollow. I’ve been here for 23 years which makes me a native to some. My wife is really a native. She grew up here and went to the same school her Mom did. My kids went to the same high school she did (my youngest is still there). My son is majoring in Aeronautical Engineering at CU Boulder. Pre-COVID my daughter played competetive club soccer which meant we drove lots of places every weekend for games. We have two dogs - both rescue mutts.
I met my wife through a great friend (actually his girlfriend, whom he later married). She thought we’d be a fit because we both talked about someday owning a bookstore with a coffeeshop in it. She was right. We just celebrated our 21st Anniversary. We love to travel, and it seems that my daughter caught that bug. She wants to go to University in Europe, maybe in Italy. My family is the most important thing in the world to me. This is us on a magical trip to Italy a few years ago.
I’ve been really lucky to have a lot of really good jobs. Once upon a time I was in the US Navy and served in Submarines. I was on the USS Richard B. Russell (SSN-687). I won some awards and the ship won some major awards for things I cannot talk about. Sorry. But I can tell you those years were incredibly formative. Engineering really matters when failure sinks a billion dollar ship and the reactor pollutes the earth for 50K years. But if you do it right, all the time, it’s game changing. I was an Operator and then a Plant Supervisor. It’s where I really fell in love with complex systems. It’s also where my core leadership values were instilled in me.
After the Navy I worked for PG&E doing research and development, mostly on measuring the power system. I wrote an article for Linux Journal about that. I also got to work with Woods Hole Oceanographic Institute (WHOI) on the Research Vessel Atlantis. We carried the mini-submarine Alvin. That was a really fun job. I even got to do this:
I’ve had the pleasure of writing code and building systems and teams across several generations of technology. The more it changes the more it stays the same. It’s still what I want to do every day. If you really are interested in all that I’ve done you should check out my LinkedIn profile.
Now I have a dream job: leading Solution Architecture teams at AWS. I was around for the PC revolution, the web boom, the begining of mobile phones, Arduino and Raspberry Pi. I first installed linux on a VM on Slicehost sometime before 2009. I worked at Rackspace. I moved several companies SaaS products to the cloud. The way I see it AWS is increasing the pace of global innovation by introducing so many amazing offerings. In my long career I have never seen this much raw potential in our industry. I could not be happier to finally be at AWS.
OK, some stuff that’s a little more personal:
- Favorite SF Restaurant: tied between Kokkari and Zuni - can’t wait for post-COVID to go back!
- Favorite meal: properly grilled steak, medium rare, arugula salad with lemon dressing, roasted potatoes, a good red wine
- Favorite cocktail: vodka martini, up, dry, olives, slightly dirty, shaken, ice flecks on top - Harris’s Steak House may make the best in SF
- Favorite author: CJ Cherryh, especially her Alliance Universe, especially RimRunners
- Favorite sport: Baseball. I live and die with the SF Giants. Hard to imagine going to a game now though
- Favorite city: San Francisco, even though she’s in a rough spot right now
- Favorite small town: La Cruz de Huanacaxtle - yep, we bought a house there.
- Favorite vacation spot: Mexico - see favorite small town
Seems silly, but this document is Copyright © 2018-2021 Gregory C Herlein and released under the MIT License