Being an Apprentice is better than a Scrum Master (Tools do not make thou work with Agility)

Buzzwords. They seem to make their way into the vernacular via the process of regurgitation from folks farther and farther removed from the source. They start out as simple ways for tech folks to quickly communicate a concept to other tech folk and eventually become shorthand for explaining something to the business. The process of using analogy to describe something is not a bad thing, but when a shorthand term itself creates a cottage industry, one might want to dig deeper. In particular, let’s talk about the now Hackneyed term “Agile.” With the growing drumbeat of organizations trying to squeeze more efficiency out of teams, they have turned to the “magical pixie dust” delivered through the now buzzword of Agile. Heck, even people that helped create the Agile Manifesto are now having to respond to it (see Dave Thomas post here). All these certified scrum-master courses that are popping up like weeds in a bed of mulch after a rain are perhaps the tipping point for the overuse of a shorthand term.
Look, the guys that put together the Agile manifesto were just trying to cut through all the bullshit that happens in traditional corporate spaces to just ship some decent software fast. Taking simple concepts and creating a certification (Scrum Master) for it seem counterintuitive to me since, if it were actually simple, then why would there be a need for a certification rigor?  It makes more sense to me to have a mindset of being an apprentice where you are constantly reminding yourself how to work efficiently by….

  • Valuing Individuals and interactions over processes and tools
  • Preferring Working software over comprehensive documentation
  • Valuing Customer collaboration over contract negotiation
  • Responding to change over following a plan

The reason the guys behind the Agile Manifesto came up with these simple terms, was precisely because all the damn project managers at large organizations were so locked into the doing massive “waterfall” style project plans and Gantt charts and dedicated people to do technical writing in bloated 6MB Word documents that software developers just couldn’t work quickly and without friction. Dave Thomas made the point that people should stop using the term Agile as perhaps a noun, and go back to using it as a action statement more akin to an adjective.
Dave says the following brilliant statement: “It’s easy to tack the word “agile” onto just about anything. Agility is harder to misappropriate.” So for him you aren’t and agile programmer – you are a programmer that programs with AGILITY. You don’t work on an agile team, you work on a team that exhibits AGILITY. You don’t use “agile tools”, you use tools that enhance your AGILITY.

Since I myself have been been trying to make people on my team act and work with more “agility”, I try to keep myself tuned into ideas and people that “get it.”  So when someone tries to help convert traditional IT project managers and technical managers into working with more “agility”, it is worth taking note. Kamal Manglani’s book is trying to do just that. He is trying to get people to work with more agility. Having worked with Kamal in the past, I can tell you that he does get it. Kamal starts shaking his head when standups are taking too long and people turn them into bitchfests or psuedo spikes. He is known for being a bulldog that will search people out and sit with them at their desk in person to solve a problem and won’t leave them alone until the problem gets attention or resolution.  He’s the guy that frown’s on large bloated Word Docs in favor of a wiki or a readme file in Git. His book at least doesn’t misuse the term agile, it’s called ‘The Apprentice and the Project Manager‘.  So, while the devs hopefully get it, at least someone is trying to get those pesky project managers thinking and acting with more AGILTY, not being “agile”.  If this helps teams ship things faster and gets teams more productive by reducing the number of folks in their path, I’m cool with it.  Since we have to keep reminding ourselves how to do work with agility, perhaps being permanent apprentices is better than a scrum “master” of anything.  Is anyone ever a master?


Thoughts on having an opinion on tech

regurgitate != knowledge 
explore+test = have_an_opinion 
requires_work = 1 
sed -i 's/no_time/make_time/g' /my/schedule

What the Heck is an Architect?

I was never one of those little kids that told mommy and daddy what they wanted to be when they grew up.  I didn’t even start officially working in the “IT” industry until 1996.  I just kind of fell into the career because I enjoyed fixing other people’s computers all the time.  The closest I ever came to wanting a particular job as a teenager was being an architect.  You know the traditional kind like the dad the “Brady Bunch”?  In my mind an architect was always an engineer with a big desk, rulers, calculators, knowledge of complex math, physics, and materials science.  They also were visionary creative types that could come up with amazing designs like the Chrysler building, the TransAmerica Tower, or Falling Water.  In my mind these architects could do designs in an office, but also go out in the field and do some of the engineering work themselves.

The Architect - Mr. Brady

Architect-Mr-Brady

Fallingwater-Frank Lloyd-Wright

Fallingwater-Frank Lloyd-Wright

Fast forward to today when the job title of architect can take on a variety of constructs:  software, infrastructure, political, health care, network, security, traffic, etc.  The job title has morphed into any type of senior design position in some fields.  I myself am part of this trend with the job title “Infrastructure Architect” currently.  However, due to my admiration for the “traditional” architect job, I feel an obligation to at least try and transfer my ideas of what skills translate from the traditional architect into the modern IT infrastructure engineering space.  This has been the cause of some stress for me over the past three years.  I will freely admit that today I worry about what it takes to be proficient in this modern “infrastructure as code” world.  After having a few beers, I sat on my porch and tried to jot down some notes on what troubled me personally and what the causes might be.  I think I have narrowed them down:  personal expectations versus market expectations, time management, and ambition.

Personal Expectations vs Market Expectations

I would be lying to you if I did not admit that I struggle every day personally to try and meet what my own expectations are for an architect in the infrastructure space.  Infrastructure has several “silos/towers” that represent some traditional capability areas.  These are things like servers and virtualization, storage, networking, security, databases, service buses, application hosting, as well as monitoring and performance metrics.  Having deep knowledge in more than one of these towers is what I would say an architect should have.  Due to the amount of knowledge needed to know multiple silos this can take years of work.  In my case this journey has spanned over 17 years since 1997.  There is no formal curriculum so to speak for “architect” so it kind of materializes via technical jobs you take and a little serendipity.
My architect journey kind of materialized like so…

  • Hardware phase (servers/PCs/peripherals)
  • DOS phase (batch programming, emm386 fun days)
  • AS400 and VAX/VMS phase
  • Token ring and ethernet (10base2-10000) phase (jiggle it just a little bit)
  • Novell phase (3 to NDS)
  • Windows phase (over a decade of windows troubleshooting madness)
  • The whole certification industry treadmill phase (mcse/cne/ccnp/blah thank goodness this is dead)
  • The DBA phase (SQL server and Oracle)
  • Network engineering phase (ethernet to satellites to sonet rings to mpls and vpn)
  • Unix/linux phase (Solaris, Slackware, mandrake and RHEL)
  • The Perl/CPAN/Sourceforge phase
  • The php/html/css everything phase
  • The slashdot addiction phase
  • The AWS and Rackspace IaaS phase (ongoing)
  • The OSS and Github revolution phase (ongoing)
  • The ruby and cm transformation (ongoing)
  • The Hacker News addiction phase (ongoing)

The presumed belief is that the more you stuff you learn the more valuable you become from a holistic experience lens.  This is likely true in that you learn over time how multi-level dependencies and other design issues can affect resiliency, performance and scale.  Eventually the need for this kind of holistic knowledge was enough that the modern job titles of “architect” in the IT space arose.  In some ways, the architect position is seen as a more senior role (even though many organizations don’t have an HR job code with architect in it.  To be sure, I’m ambivalent on whether the job title even needs to be official).  The flip side is that going deep in n+3 or more capability silos becomes increasingly difficult to do well given the other responsibilities that often fall on the shoulders of an “architect”.  The funny thing is that these “other responsibilities” often begin to take upwards of 70% or more of the time of an architect type person.  There is a constant balancing act going on where one foot is on the hands on (development/testing/research) side of the fence, and the other foot is in things I call CCE like (collaboration, communication, and evangelism) or leadership side of the fence.  The latter half inevitably results in meeting hell.

JBPadgett_Success_Formula

DO > TALK

Why do these other things increasingly take over the time of an architect type person?  I have thought a lot about this, and I believe it likely boils down in part to the holistic experience thing (others may only have a single silo view of the world and hence need help with understanding about why something could be better another way) and partly because the architect (even though the role may not even be official on paper) is saddled with the overall responsibility that things work cohesively within a given capability area.  When someone has a job responsibility to “make things work” together, it implies that they ideally understand and have proficiency in the technologies involved.  This gets increasingly difficult the more technology areas you are expected to know.  I will be the first to admit that this knowledge responsibility only works when a federated model of architectural ownership is in place.  By this I mean that having multiple architect roles within infrastructure is important and many more within an enterprise is absolutely a necessity.  The old mindset of someone having the job title of ‘Enterprise Architect” is a farce.  It is a feat of impossibility if you expect to have actual architecture efficacy in an IT organization.  The archaic “ivory tower” approach to managing architecture in a top down model is just not feasible.  To be sure, I don’t think it ever was actually effective; and so in my mind the “Enterprise Architect” is dead and has been for a while.  The point I want to make here is that there is just too much to know to be truly effective outside the limited scope of a major capability area.  The ability to be hands on and also do requisite CCE I feel is best within a more manageable size capability area.

“The ‘Enterprise Architect’ is Dead”

Pretty much since 2007 the infrastructure as a service revolution inspired by AWS has been a massive snowball of change.  This change has and continues to be awesome.  I have wholeheartedly embraced it and have been taking part in the transition for years now.  The changes have in part re-invigorated the popularity of the open source movement and with it has come unparalleled opportunities for startups.  It has inspired the creation and success what has become a literal tidal wave of open-source projects on Github.   All one has to do is look at Hacker News on any given day and see that some new project was announced that “fixes this” or “closes this gap in functionality”.  One could argue that a business could start and run without much if any proprietary commercial products today (short of the client devices).  It is an amazing time that we are living in that maybe 30 years from now will be mentioned in history books as the tipping point for mainstream open-source adoption and innovation.  All of this is super cool, especially if you are a small to medium sized startup, and can get everything you need infrastructure-wise from something like AWS or Rackspace or Digital Ocean using an API.  Even better, modern configuration management and containerization sits on top of these IaaS platforms and allows you to install packages and manage build outs rapidly using “infrastructure code” as well.  If you are a software developer and need to be productive in the infrastructure space (even if you don’t know it well) you can do so effectively with just the knowledge of a given API.

Marc Andressen’s famous callout in 2011 that “software is eating the world” has never been more true than in the IT infrastructure space.  I have felt it every day for the past 5 years now.  The challenge, specifically for me, is staying productive, efficient, and helpful in a space that is increasingly more software.  Software development to be precise.  Let’s be honest, even with the cloud NaaS revolution, most enterprises still run the bulk of their infrastructure in house.  Part of this is dealing with legacy apps and the baggage they bring, part of it is a pure cost play where the bandwidth and storage used by an enterprise can still be less in house than using AWS or Rackspace, and finally, part of this is security and performance concerns.  So with many enterprises trying to take part in this “infrastructure as code” revolution while still maintaining significant internal hardware, the need for someone to know both traditional infrastructure management and new code based approaches is critical.  To be sure, a software based management abstraction layer (api) is increasingly meaning that the work an an infrastructure architect does is more dev and less ops.  Since I want to be an architect that does both hands on coding and traditional architecture work (to remain relevant), I need time to focus on development.  Getting the time to do the work and focus is a real challenge.  But dude, the thing is I have to do this to enable infrastructure to provide any value for the app dev teams in the enterprise.  If I can’t help make internal infrastructure as easy to consume and programmable as Amazon AWS, then the Devs increasingly see me (vis-a-vi infrastructure) as a blocker and I don’t blame them.  At least for now, because of the need for supporting legacy and all the painfully inflexible COTS packages (hello Oracle), Infrastructure ends up also having to play a type of luxury one-on-one concierge service for app teams.  I don’t know how long this “concierge” style need will last, but I’m gonna be pushing as much self-service infrastructure via API as fast as I can make it happen.  Things like Openstack and SDN are making this possible for me, but doesn’t address everything needed.

Pressure on Myself  in the Valley

If you buy into the software will eat the world argument (and I feel like I do) then the question becomes:  How can I be the best at my job as an infrastructure architect?
More precisely:

  • What is valuable work?
  • What are valuable skills?

If the answers are only writing code, then there are lots of questions I have the keep me stressed.  First off, is this thing endemic to Silicon Valley only or is more widespread?  I feel like the mass transition is already well underway.  Right now the change is already prevalent in Silicon Valley.  There may be a three to five year delay before it hits middle america, but its probably coming.  But how will these changes (for all their merits) scale?  Are all the old sysadmins that can’t code pretty much SOL?  I think perhaps so.  I said the traditional sysadmin was dead in a paper published last year, but I wonder if middle america can keep up.  I mean this stuff is not exactly easy.  Is this an example of the robotization of everything?  If everyone is writing code, then who is spending time communicating and collaborating (remember this takes a buttload of time) and who maintains the code down the road?  What about all these small IT consulting companies that handle outsourced IT for the space between mom and pop businesses to the enterprise?  Is the “Geek Squad” of the near future going to need staff that are polyglots understanding CI pipelines and Git?   Writing code takes a singular level of focus and time that is already a challenge for even seasoned developers.  The needs for the people (often architects) that sit at the event horizon of software development and collaborative design and communications won’t simply go away.  The difference is now I think they need to be doing both.  Architects, as I have mentioned, often have a lack of time due to the immense gravitational pull of CCE (collaboration, communications, and evangelism) that needs to be done with all the stakeholders/customers of infrastructure.  If I were to surmise that my ideal job time split between “hands on keyboard” time coding and CCE were 50/50, then I’m gonna have to change the way I manage my time.  The balancing act of hands on work and meetings is just way fracking hard.  I may have to start being more selfish with my calendar and time.  I’m just not sure how best to solve this yet.

“The balancing act of hands on work and meetings is just way fracking hard.”  

balancing-hands-on-work-with-leadership

Another issue I struggle with is the perception that the ability to do both hands on development and CCE or leadership well is commonplace in the valley and elsewhere.  This is buttressed by the examples of exceptional people like Elon Musk, Sergey Brin, Larry Page, Mark Zuckerberg, and Max Levchin as bellwethers for how to work.  I feel like these guys and others like them are truly exceptional people.  They garner media attention because they are success stories, but the translation in the valley is perhaps that their skills are as obtainable as riding a bike.  I feel like this is just not the case.  I don’t necessarily want to lend too much credence to the Malcolm Gladwell “10k hours” argument, but this shit is just not easy.  The competition here is high, especially if you truly want to make a difference and want to think of yourself as good at your job.  Perhaps I am just too hard on myself, but here lately I have gotten to where my blood pressure rises, I get hot, and feel overwhelmed when I read HackerNews.  Sometimes I feel like all the article titles are flying by my eyes like data in the matrix making me feel more inadequate by the minute (the Oracle said I was not “the one”).  Is this is the bane of type A personalities who want to be the best and are competitive?  I rarely think of myself as an “expert” at anything even if I am kicking ass at something.  Sometimes this personality trait of mine helps me and drives me to be better, while other times it exhausts me and gets me down.  Am I a little out of balance with incessantly wanting to be a better programmer, a great speaker and leader, a great skateboarder, up to date on all the latest geopolitics, and be able to write eloquent comments on HN?  Maybe.  I might need to wean myself off HN for a little bit for my own mental health.  I think when you start to get jealous of the gas station cashier for having a simple and less stressful job, it might be time to level set.  Than again, I just want to do a good job in this rapidly changing infrastructure as code landscape.  So what is a modern infrastructure architect?  Is it a software developer, a shoe salesman, or a crisis negotiator?

Wait, sorry I have another meeting to attend.  Let me know if you approve and merge what I perceive to be a market pull request.


Introducing SFSC

I decided to take my many years of skateboarding love and pour it into my own new company.  So, after a year of hard work, product development, and testing, I am happy to announce the launch of San Francisco Skateboard Company (SFSC).  There are so many things I want to do for the sport of skateboarding and this company and brand will provide me a way to make things actually happen.  I plan to do a lot and I will be sharing it with you here.  I would really appreciate your support for the company.  Try out my products and let me know what you think.

Check it out here: San Francisco Skateboard Company

Image


Skateboarding and Open Source Communities

I have been skateboarding since 1986.  I love it.  I cannot get it out of my blood.  I still skate today regularly even after years of injuries: broken toe, broken thumb, broken clavicle, multiple ripped ligaments…the list goes on.  This is more than dedication, skating it is a life-morphing adrenaline rush that can forever change you.  I also love computers and software development and have been doing that since 1990.  Rodney Mullen is one of the most influential skateboarders in skateboarding.  You may not have heard about him, but this guy is amazing.  He won literally every competition in freestyle skateboarding he ever competed in.  He basically invented the entire category of freestyle skateboarding.  He defined a singular work ethic in skateboarding that many in skateboarding couldn’t groc at the time skating upwards of 9 hours a day.  But he didn’t stop there.  He started a company and made it hugely successful and then sold it.

Sound familiar?  This is what software developers and folks like us do in Silicon Valley all the time.  Imagine my joy when one of my idols was able to articulate the similarities between the skateboarding community and the open source tech community.  Recently, Rodney Mullen was a speaker at Ted Talk at the USC campus in LA.  At the talk Mullen pointed out the importance of creativity and innovation in skateboarding.  He says the greater the contribution to the community, the greater the response and extension by the community.  Man does this ever hit home.  He makes the example of the Ollie and how that singular trick development and innovation spawned an entire new set of skateboarding tricks that have come to define the sport.

The comparison to the open source community within the tech world has the same multiplier effect like in skateboarding.  Look at the amazing decisions to make software like Linux, Apache, Hadoop, and OpenStack available to the community.  The community responds in kind with continuous advancements and ongoing maintenance.  We all benefit.  So while you may want to do a startup to be successful (and to be sure skateboarders invent tricks to be successul too) the joy and benefit of your creativity to the open source community  is also a worthy reward.  I say this also to draw your attention to the incredible work ethic of many skateboarders and the innovative community they foster.  Our two communities have many similarities.

See the Ted Talk here.


OpenStack Quantum Network Design

I can never seem to get enough OpenStack networking information.  I have been collecting various notes in Evernote for 2 years now on OpenStack networking.  Having built and destroyed OpenStack Diablo, Essex, Folsom, and now Grizzly at Gap with much pain, I decided to share my notes in a consolidated format.  My hope is this will help make OpenStack Quantum network design more tenable for folks trying to be planful with their network design and OpenStack.  You can see the full presentation embedded from scribd below or you can access it here.

View this document on Scribd

Becoming Coruscant?

The June issue of The Economist has a lead article titled “Towards the end of poverty.”  Among the key points made in the article is that gradually the world is making progress toward raising a defined minimum level of living wages to something more in line with developed countries.  As more countries join the global community and take part in the benefits of developed economies, everyone rises.  Now, there are many more granular, micro-level arguments that can be made about widening gaps between the more wealthy and middle-class, but the point of the article was that those in the world with living conditions that were extreme poverty are decreasing.

This got me thinking:  how long will it be before “developing” countries become “developed” and overall living conditions begin to approach those of current and long-time developed ones?  What would this new fully developed world look like?  How would a global citizenry all wanting and needing similar “developed” things function?  Think about it.  In the past 15 years China has grown in power and stature incredibly and (though opinions rightfully vary per issue) its citizens have benefitted along with this growth.  More middle-class, less poverty (overall), increased numbers of educated, and increased life expectancy levels in general (baring pollution issues) .  Similar exploding growth is happening in India with Africa likely to be next.  So, how many places on earth in 2013 are left that are considered “developing”?  Interestingly, there is not an official index from the UN (need to validate this), but there are some established viewpoints that get published from the World Bank and the IMF.  I found some here from Oxford University that says there are ~35-60 depending on who is reporting.  So, if all these countries took say twice as long as China (perhaps~ 30+ years) we might have a large global middle class by maybe 2050?

Combine this with the incredible advances in technology and medicine we have made in the past 30 years and you have the making of some of these Hollywood visions of the future like the self driving cars in Minority Report and ever larger cities that form interstate sized connecting metropolises.  Fast forward even farther and something like the full planet city of Coruscant in the Star Wars series doesn’t seem all that much of a fantasy.  The interesting thing to ponder is not so much the technology advancements but the globally touching life we all might lead in such a future.  I guess the question remains:  what does this future world look like when everyone becomes developed?  What type of interactions will we all have?  How will trade be done?  How will businesses remain competitive when wages are no longer arbitraged by going to a developing nation?  Could earth become one big developed planet?

Coruscant


Follow

Get every new post delivered to your Inbox.