The scribe stirs

It has been a while dear reader, far too long! And that without an explanatory word as to my absence. For this, my apologies. Much has happened in the meantime, both personally and professionally. Not to be making excuses but I had to stop some activities to achieve a sustainable pace. The blog suffered. The fact that all the images disappeared suddenly for some strange reason, and that there was no one-click way to restore them, didn't help counter the inertia when I started to feel uncomfortable about how little log my blog was becoming...

Since I last posted, I have been involved in a Solvency II project which, though ultimately very satisfying, was a challenge like none I had ever faced at work before. For a narrative of this project, from the perspective of change management, see our presentation at Agile on the Beach.

Otherwise, I also became a father for the first time, of twins!

During the same period of radio silence, I also attended a Cognitive Edge Cynefin foundations course, read some good books (including Continuous Delivery and The Lean Startup) and continued to build the agile consulting service at Qualogy.

My current contract is a step into the unknown for me, I am coaching (Kanban) at a company that provides platforms and infrastructure as a service. Exciting virtualised cloudy kinda stuff. This will certainly provide me plenty to blog about as I turn more and more to Lean to get me through the day. Topics I intend to blog on in the near future will include teams as complex adaptive systems (based on a definition of CAS as offered by Paul Cilliers in "Complexity & Postmodernism"), scrumban as solution for the enterprise and (automated) testing as a merciless and pervasive spotlight on quality.

To fresh beginnings and instructive ruminations!

Calmly going where no (wo)man has gone before (or have they?)

My colleague and I are setting up an agile consultancy within an existing consulting firm, Qualogy. Our mission is twofold – spread the agile word, increase agile expertise among our own consultants and build and promote an agile services portfolio. I've been involved for about a year and although we have a long way to go, things are really beginning to take shape.

Because of the nature of our mission we are always looking for ways of killing two birds with one stone. For instance, we offer (agile) training to our partners and clients, but also to our own consultants, which has the double function of increasing agile expertise all round while hopefully making a name for Qualogy as an agile provider. This is easier said than done however and we're not the first to think along these lines; the market is nearing saturation point. We try and set ourselves apart by concentrating on technique over process and by getting the very best to come and offer courses not easily available elsewhere (in the Netherlands). For example, Ron Jeffries & Chet Hendrickson have been to Qualogy twice to deliver hands-on agile development training.

Recently we added Gojko Adzic's Specification by Example training to our portfolio. As is our policy, Wouter & I attended this first instance of the course at Qualogy. Due to some late cancellations from our own consultants we could only claim a tarnished victory – good service to our clients, not much penetration among our own consultants. This slight disappointment did not impinge (much, after a while :-)) on my enjoyment of the course though. As is always the case with when attending these courses, I'm not just interested in the material but also in how the course is run. Adzic was a joy to watch.

This post is not however about the course itself, or even Gojko's admirable delivery of same. As Adzic himself writes, and as was clear from how he delivered the course, Specification by Example is method/framework/etc. agnostic. He tended to use lean terminology during the course, speaking as he did of bottlenecks and/or the theory of constraints as opposed to impediments for instance, while at the same time, his explicit insistence on cross-functional collaboration smacked of agile. What I would like to share from my learning is the realisation that Specification by Example (or BDD or A-TDD) might well be the M (mashup) we so badly missed at CALM alpha!

At CALM alpha, Dave Snowden named three heuristics that seemed to offer common ground between (social) complexity, agile and lean; fine-grained objects, cognitive spread and dis-intermediation. Fine-grained objects meaning; small well-defined stories, short clearly-demarcated iterations etc, cognitive spread meaning; input and reasoning from multiple perspectives & interests, set-based design etc and dis-intermediation meaning; visualisation, no separation of design & implementation, narrowing the gap between information and meaning etc.

Back to our Specification by Example course – Gojko was prompted a number of times during the course to state that any specification which required an ever-growing list of examples to illustrate it should be split. While working on turning 'classic' requirements into specifications with examples, merge&diverge was explicitly employed to ensure the best results. But it was while examining a bunch of specifications with examples for how useful they might be as respectively, a tool for shared understanding, acceptance tests and/or documentation, that I was struck by why I had found this idea of living documentation so powerful all along.

Requirements that are tests that are documentation – no handovers, no translations, no interpretations, no esoteric jargon pitting one guild against another – that's lean, that's agile. Specification by Example is the ultimate in dis-intermediation!

Project managers as scrum masters? (Know your attractors!)

Inspired by a flattering response to a couple of tweets of mine at the weekend, which were a condensed paraphrasing of the Cynefin model's approach to complex problems, and keeping in mind @0x17h's (via @RonJeffries) paraphrasing of Arthur C. Clarke, “Any sufficiently advanced catchphrase is indistinguishable from wisdom”, I decided to try and illustrate my statement with an example problem and how it might be dealt with.

The (combined) tweets:

Similarly constrained self-organising systems will tend to organise towards one of a few attractors (patterns), the trick is getting the constraints right & then damping, or encouraging, patterns as they emerge. Know your attractors!

Problem statement

What to do with the project manager when transitioning to scrum?

Constraints

• There are only three roles in scrum; PO, scrum master & team
• Team should plan it's own work for a sprint and meet daily during the sprint to take stock and make any adjustments necessary to stay on track

Project managers often have former lives, meaning that many see a transition to scrum as a natural way for them to return to the work they actually love without losing face, i.e. they become team members. Others are naturally inclined to assume the Product Owner role. Some consider themselves good candidates for scrum master, or are cajoled into it, as the scrum master role is often, erroneously, seen as equivalent to the project management role.

Whether or not a PM will be a good scrum master is a problem in the complex or chaotic domain because, even if it were possible to identify all the variables involved, they are usually fuzzily defined and (therefore) difficult to measure – e.g. scrum master competencies – which means that causality is often discernible only in retrospect, if at all (retrospective coherence).

According to the Cynefin model's probe-sense-respond approach to problems in the complex domain it should be possible to observe the scrum for one of a few known attractors (this is what differentiates action in the chaotic & complex domains – act vs probe – probe implies that you have some idea of the range of possible consequences of your actions) and amplify positive developments and/or dampen undesirable ones appropriately.

For the sake of brevity we will specify two (possibly overly?) simplified attractors and a number of patterns by which you might identify which attractor your situation is headed for, and how you might intervene.

Attractors

• Command-&-Control (alpha male/the way things were)
• Subsidiarity/Distributed cognition (agility; the path you want to follow)

Probe: PM transitioning to Scrum Master

• Pattern: Scrum master as power broker, boss (Attractor – Command-&-Control)
  • Sense
    • Scrum master determines sprint scope (in committee with PO)
    • Scrum master dictates tasking
    • Scrum master assigns tasks
    • Scrum board not a source of truth
    • Scrum master unilaterally decides on action in the event of interruptions
    • Little regard for sustainable pace
    • Estimates as commitments, regular overtime
  • Respond
    • Dampen
  • How
    • Insist that PO prioritises and Team forecasts
    • Insist that scrum teams must organise their own work to be successful
    • Insist that tasks be pulled, during the sprint, and not assigned (pushed) during planning
    • Ensure that only work represented on the scrum board gets done
    • Ensure that interruptions are triaged with team input, and resultant outcomes agreed with the team
    • Ensure that variance in velocity is recognised as normal, coach the team and the PO to create stories small enough (~10 per sprint) such that (occasional) failure to fully meet the sprint forecast should not cause any serious business issue
• Pattern: Ingrained habits and perspectives difficult to displace (Attractor – Command-&-Control)
  • Sense
    • Scrum master plays an active role in task planning
    • Daily stand-ups always convened by scrum master
    • During stand-ups, team members look to scrum master when 'reporting status'
    • Burndown updated by scrum master
    • Burndown 'flatlines' till last day or two of sprint
    • Stakeholders ask scrum master for status updates
    • Scrum master coordinates effort to be able to demo during review
    • Scrum master demonstrates new functionality during review
    • Team communicates with other teams through scrum master
  • Respond
    • Dampen
  • How
    • Scrum master leaves the room while story is being tasked
    • Schedule stand-ups for the same place & time daily, ask the scrum master to hang back until a few team members have approached the board
    • Have team members pass a token during stand-up, whomsoever has the token, has the floor. Ask team members to update the scrum board as they talk. Have scrum master stand laterally to the group.
    • Ask team members to rotate this duty, agree that the last to speak updates the burndown
    • Encourage swarming, introduce explicit WIP limits
    • Ensure scrum board is big, visible and kept current
    • Make it clear that the scrum master's role is to facilitate the team's demo, not run it
    • Introduce subject matter experts and leave the conversation, facilitate setting up communities of practice etc.
• Pattern: Scrum master facilitating & delegating (Attractor – Distributed Cognition)
  • Sense
    • Planning meetings not dominated by scrum master
    • Retrospectives not about the scrum master
    • Impediment list ordered, current & highly visible
    • Team decides on sprint scope
    • Tasking completed by the team
    • Tasks pulled during the sprint
    • Number of open stories less than number of team members
    • Stand-up happens of its own accord, at the same time every day, in front of the scrum board
    • Trust between PO and team evident
    • Communities of practice emerging
  • Respond
    • Amplify
  • How
    • Ensure the scrum master does intervene when necessary (e.g. PO pressuring team)
    • Have scrum master change format of retrospective on occasion to prevent stasis, ensure that PO allows for (a minimum of) one improvement story per sprint
    • Ensure impediments are being cleared (list isn't just getting longer and longer)
    • Ensure the team continues to challenge itself – organise it such that that variance in velocity is recognised as normal, coach the team and the PO to create stories small enough (~10 per sprint) such that (occasional) failure to fully meet the sprint forecast should not cause any serious business issue
    • Challenge tasking outcomes; are we missing anything? etc.
    • Remain alert to power distributions within the team
    • Continue to try and minimise the number of concurrent 'open' stories
    • Have the scrum master be the first to approach the board on occasion
    • Continue actively encouraging transparency
    • Bring people together, don't get in their way

This is anything but an exhaustive list dear reader, and in a real-life situation you will probably be both dampening and amplifying on different tracks simultaneously, but I'm sure you get the picture.

Princely agile

Whether running an agile project or a Prince 2 project, the principles at work are the same, in both cases a project is broken up into small pieces and regular formal inspection is applied to ensure alignment. It is in the detail of the how & why of that decomposition, and the interpretation & exercise of control, that important differences can be found. Another important difference is the level of prescription, Prince 2 attempts to describe the complete set of possible project contexts and states that the method should be pared to fit. Agile methods, like scrum, tend to prescribe very little and insist that context will determine how the method or framework should be extended for any given situation.

Ironically enough the majority of so-called Prince 2 projects are PINO (prince in name only), have few or no (non-project management) products defined and never include stages (which should not be confused with phases), nor the associated formal decision moments on how to proceed, or not. This means that stopping a project often only becomes an option when original deadlines and/or budgets have been thoroughly and spectacularly violated, more often than not without the delivery of any usable product whatsoever. The PID (project initiation document) often remains live deep into the project lifecycle, sometimes being finalised only when the project draws to a close (talk about gaming the numbers!).

Product based planning is a cornerstone of Prince2 and, if applied appropriately, can provide a link to agile methodologies. Appropriate application would involve hi-level specification of products during the Initiation phase, whereafter detail is added incrementally and iteratively, just-in-time and through collaboration with the team. Minimum viable product (or feature) should be the guiding principle in designing a product breakdown structure (PBS) - not process flow like requirements document, analysis document etc.

But if scrum is a framework for project management, why would you need Prince 2 at all?

Well, scrum is indeed sparsely prescriptive and basically says nothing about (financial) governance other than that it is up to the Product Owner. In fact the role of Product Owner is one of the most problematic aspects of scrum. Scrum says that the PO should have vision & authority and be (always) available to the team thereby ensuring maximum value delivery. This is a difficult assignment, and if the stakeholder and/or user landscape is complex enough, it requires nearly superhuman abilities to execute satisfactorily. A sparse version of Prince 2 could provide a solution here. Specifically, a steeringboard could provide a workable escalation channel in priority disputes (as long as it is still employed according to the principle of management by exception, i.e. at the discretion of the PO), and the maintenance of a formal business case might provide an explicit policy for the PO to work from from day to day.

I have used Gojko Adzic's (most excellent!) Specification by Example to create a mapping between Prince 2 phases (always mandatory != stages) and agile execution:

Project Start

• State goals (Brief/Mandate)
• Appoint Steering Board
• Appoint Project Manager or PO
• Assign existing team(s) to the project

Project Initiation

• Collaboratively determine scope (hi-level PBS e.g. in the form of Story Mapping/Initial creation of Product Backlog, consisting necessarily of primarily rough-grained stories)
• Create business case
• Collaboratively elucidate hi-level risks

An initiation phase could easily consist of a small number of sprints, this would ensure a much better understanding of scope & risk, and allow for using actual velocity in provisional (release) planning etc

Project Execution

• Managing stages & stage boundaries: Each iteration is closed with the demonstration of working software, actual progress is tangible and concrete which means that the validity of the business case can be continually reviewed and assessed with confidence. The definition of formal stages to this end may be overkill.
• Work package creation & acceptance (and associated wasteful hand-offs) are replaced by the ongoing collaborative activities of backlog grooming (“refining the specification”) & sprint planning. A project manager (if there is one) therefore has no role in activity planning.

Closing a project

• Lessons learned – continually generated during the project (review & retrospective)
• Product handover – not necessarily applicable if every sprint realises (potentially) shippable product
• Review goals

Et voila – princely scrum!

Of course, there are those who suggest that management generally (Radical Management, Management 3.0) and financial governance specifically (Beyond Budgeting) would also benefit hugely from an agile approach. I agree, princely scrum (if and when contextually appropriate) should be but a step on the road to imperious agility.

What's eating Bob Marshall? (A polemic)

Today, dear reader I present you with a polemic. I feel obliged to warn you before hand because polemics as a rule are not my favourite fodder, and you might prefer to be spared my petulant outburst, as indeed I tried to resist penning it. But I can't get no satisfaction, so here goes:

Alright! OK @flowchainsensei, we get it! The scribes and the Pharisees have taken over. And they're in bed with the money lenders. Agile is not the true gospel.

Bob Marshall (@flowchainsensei) is concerned for the state of our “broken industry”. Do we inhabit the same digital space? My world is turned upside-down & inside-out with increasing regularity. Are not many of the problems in fact a measure of the industry's success?

Yes, it's important to have people around who see the glass as half-empty but it's really tiresome to have people running around screaming “Fire!” all day because the toast was burnt at breakfast.

In his prolific railing against all that is wrong in the world (of software development) Marshall consistently pits Deming, Buckminster Fuller and Ackoff, the truly wise, against those pretenders from Snowbird (and especially their disciples), while simultaneously, incongruously, lambasting all that is agile as old-fashioned and even outright evil. Don Reinertsen rightly points out however that the implications of systems thinking are potentially “terrifying” and concrete manifestations of Buckminster Fuller's sociological ideas sundered quickly under the crushing weight of their naiveté.

Any utopia is an El Dorado, and anybody claiming to know the way there is necessarily a charlatan.

But let's just assume that Marshall has seen the light, and that if we follow him we will arrive in the promised land, what will it be like? Well, like all visions of paradise, it's pretty enticing, as per Marshall's own description. But how do we get there? According to Marshall - by way of an instantaneous mindset shift effected simultaneously throughout the entire organisation.

There are leaps of faith and leaps of faith.

Luckily, Marshall also offers more practical tips, like ditching CVs. I find that an attractive idea, but the associated technical advice is weak - “hire mindset, not experience; where someone wants to go is what counts, not where they've been”. That's anti-empirical and probably the best way ever devised for ensuring the hiring of the slithery of tongue.

Another good idea that he espouses is subsidiarity. Marshall's version however has something unsettlingly absolute to it and is tinged with begrudgery: Leave it all up to the ordinary people, the folks on the front lines, and everything will be fine. Workers good, managers bad. To me, it smacks of the worst kind of spiteful socialism.

Dialogue is another favourite of Marshall's in his search for a better world, although an exchange has to be of a very specific type before it can qualify as dialogue. When I posted a link to my response to Marshall's “Agile Coaching is Evil”as a comment to said blog, he apparently felt that it wasn't in the spirit of dialogue and did not publish it. Repeated fruitless efforts at eliciting a reason for this have helped germinate some very uncharitable thoughts in my mind, thoughts through which sycophants and yes-men flit. I may just be paranoid.

So, sensei or coach? Whomsoever is willing to pick and choose from systems thinking, agile, lean and/or (social) complexity. Whomsoever eschews dogma and mystery. Whomsoever strives to help you improve your effectiveness through mindful practice & considered reflection.

Beware the false prophet – identifiable by his penchant for citing personally vetted Ancient Wisdom from August Thinkers, while hard-selling a puritanically proprietary Utopia.

To estimate or not to estimate (that is the question)

Not too long ago I was introduced by Jim Benson to the concept of dropping estimation in favour of measuring cycle time. To me, this was a radical departure. My very being seemed to rebel against the notion (yes, I have been a project manager). And yet the argument against estimation seemed so powerful, so in keeping with my own experience (humans are notoriously bad at estimation, estimates are wilfully interpreted as concrete commitments) that I could not easily dismiss it. The idea of measuring cycle time also appealed strongly to my empirical bent. I was thrown into a state of confusion, of profound cognitive dissonance.

How to facilitate essentials for governance like ROI projections, prioritization etc. if not with estimation?

Shortly thereafter, while reading David Anderson's excellent Kanban, and the account contained therein of how Dragos Dumitriu (@netheart) turned a struggling Ops Dev team around, many of these concerns were addressed. In fact, I wished I had been as smart as Dumitriu in a recent similar engagement of my own. Basically he agreed with his customers that the team would abandon estimation and instead would undertake to complete any piece of work they committed to within a certain time frame. This was possible as existing agreements stated that the team involved should not take on work beyond a certain scope – work of that nature was managed and executed via different channels. Historical data indicated that only 2% of requests made to the team exceeded the agreed-upon scope and it was decided that the team could query work items (by estimating them) if they felt the item in question might exceed the limits specified. Upshot of all this was that the team went from estimating every job that they did, or eventually did not do, to estimating only in exceptional cases. In combination with a couple of other very astute changes this led to a fairly spectacular improvement in the team's performance. I was convinced, nearly...

Part of my ongoing issue with dropping estimation arose from the benefits I had seen Planning Poker deliver - if the team was not estimating, when would the opportunity for high-bandwidth communication on the content of upcoming work arise? What if the team involved had no agreed cap on the scope of work it should take on, very large chunks of work might arrive and clog up their queue. What to do with outliers? How to identify them? How much variance can be tolerated in a queue without detrimentally affecting throughput? And what about batch size, if we are not estimating, how do we know when to split?

Well dear reader, I'm sure you will be glad to know that, thanks to Siddharta's recent post on variance in velocity (and what not to do about it), the alpha waves are humming away harmoniously inside my cranium once again .

I suggest, in the spirit of CALM Alpha, that this seeming conflict between multiple agile methodologies (or their proponents) can be resolved as various approaches to the same problem. The trick is to think in ranges and limits and not absolute numbers. In the Dumitriu example above, estimation was implicit in the agreed scope limit (we estimate that we can get this done in less than X days), Mark Levinson has blogged on related insights (stories < 8 to 13 pts), and Ron Jeffries is proposing a similar approach of late (we estimate story to be sufficiently small to complete in a few days). This is congruent with the complexity heuristic of fine-grained objects.

We can conclude, in agreement with Esther Derby, that estimation can be useful because of the communication that it engenders, but that estimates more accurate than the likes of “needs to be split” and “small enough” have questionable worth. Such aggregated estimates are used for managing batch size and variance (and thus throughput) and not as a stick for beating developers. Thereafter, whether you use velocity or cycle time matters little - prediction remains precarious.

Fighting the good fight (A personal parole)

Recently, Bob Marshall (@flowchainsensei) has raised his lashing of agile and all its manifestations to a frenzy. His language has become ever more dramatic and sensationalist, culminating in a post entitled 'Agile Coaching is Evil'. To understand what the point of all this might be it is essential to read another post of his; 'On the Morality of Dissent'.

If you can peel back the layers of irony and provocation - all this talk of morals and ethics and Good and Evil – then there is certainly value to be discovered in what is being said. I continue however to disagree fundamentally with much of the detail of what Marshall is promulgating, even as I find his stated goals honourable and worthwhile. Yes, I am an agile coach.

I view the three main points of Marshall's argument in 'Agile Coaching is Evil', points that he makes repeatedly over many channels, as:

1. Tackling partial problems leads only to more problems – systems must be addressed holistically otherwise any change will eventually be swallowed up by the system.
2. Effectiveness being a function of mindset implies that significant change can only be realised once a radical and wholesale philosophical shift has been effected
3. Anyone purporting to help an organisation improve its effectiveness by way of local optimisations is at best naïve, probably disingenuous and possibly plain corrupt

Let's start with the holistic systems-thinking argument. It is in fact, easily dismissed:

There is no point in addressing problems in your software development organisation because the organisation around it has not yet changed its mindset (had its mindset changed? brainwashing, hypnosis, how does this happen?), therefore any optimisation achieved within your sw development organisation will eventually be nullified or even reversed by the immovable object of the mindset of the departments adjacent to it in the value chain. Okay. Say we effect mindset shifts in product management, sw development and operations? Not enough - finance, sales, customer service etc, still have not moved so eventually the local optimisation will be swamped by the inertia of the rest of the organisation. And so on until the company has somehow instantaneously leaped the mindset chasm as a single organism. But wait. Then there is the system that is the markets in which the company operates. If said markets haven't changed their mindsets in the meantime the company we have been talking about is doomed, after all, it will eventually succumb to the mindset of the greater system in which it, sub-system, is embedded. Etc., ad infinitum.

There is much of use in systems-thinking, and its focus on people and their empowerment can be found in agile, lean and the Cynefin model (complexity). In all these other cases however there is either the implicit or explicit recognition that everything progresses in small steps. In the case of Cynefin, which specifically addresses complexity and our inability to use reductionism to solve certain problems (initially), it's interesting to note that a heuristic like 'fine-grained objects' is prevalent. Wicked problems are untangled strand by strand. Possibly also interesting to note at this juncture is the fact that much of what one might do in the complex domain in the Cynefin model is aimed at moving problems into the complicated domain. Black-box observation with the specific aim of making reductionist analysis possible!

Regarding mindset shifts as a pre-requisite for increased effectiveness:

I think that Marshall has this exactly wrong. Strangely enough this would seem to be a position he has arrived at since he first published his Marshall Model, sub-titled as it is “Dreyfus for the organisation”. The Dreyfus model of skill acquisition, as with other learning models, implies that insight (mindset) follows practice and not the other way round. In other words, Marshall's insistence that effectiveness follows mindset is in my opinion, the misattribution of cause and effect to an observable correlation. I would posit that mindset is an emergent property of practice. More correctly, practice informs mindset which in turn informs practice – a positive (meaning re-enforcing) feedback loop. There is no conflict here with the observed mindset leaps at the boundaries of the organisation types in Marshall's model. Punctuated equilibrium is not an alternative to gradual change, it is its manifestation in the real world; phase shifts are the result of cumulative change in combination with tipping points.

If Effectiveness = f(Mindeset)
         and Mindest = f(practice)
then Effectiveness = f(practice)

Rightshifting phases - copyright 2010 FallingBlossoms.com

On local optimisations and the nefariousness of their champions:

It is possible to learn without a teacher, in some ways it might even be better (conjecture; one might continue to nourish a beginner's mind more easily), but it is almost certainly the slowest way to learn and autodidacts are often hampered by strategic weak spots in their conceptual armoury. There are of course always bad teachers, in every domain.

In my role as coach, I pursue small modifications in local practice and understanding, while keeping an eye on the bigger picture. Does this make me naïve, disingenuous or corrupt? I can't be accused of naivete precisely because I realise that change, if achieved at all, happens in small steps. Because I communicate exactly this to (prospective) clients, and always start any discussion with an enquiry as to why agile, I am not disingenuous. If my only motivation were my pay cheque I would be corrupt. I am however motivated primarily by the conviction that work for most people is unnecessarily miserable, a drudgery to be survived rather than a fulfilling and rewarding part of life. I feel that the agile manifesto can help and that it is as relevant today as it was ten years ago. The only thing required to make it universally applicable is to replace 'working software' with 'concrete product' (or some such).

I shall continue to try and execute workplace coaching transparently, on the basis of high bandwidth communication, and regularly vetted by concrete results, as a force for good in the world.