The history of a failure (Where's the value in that?)

Yes, this post represents a quest (nay an odyssey!) for that silver lining, because even though as I hope to show; failure has value, it still hurts!

It all began on the 16^th of June of this past year 2011 when an ex-colleague of mine contacted me regarding project management at her new place of work. Introductions ensued and the first contacts were so bright that by the 23^rd I was led to tweet:

One hot afternoon another two weeks down the line, we “pitched” by delivering a Scrum Basics workshop, on the framework of a retrospective, to a collection of interested individuals. One month and a few meetings later, my colleague and I were given the opportunity to propose a model for helping the development organisation of some 7 teams move to scrum. It seemed logical to all concerned to grow scrum in the organisation, team by team. This naturally generated the possibility of a pilot to start.

We reasoned as follows:

We had 3 workshops (Basics, Backlog Management and Planning & Estimating) and reckoned we needed 2 to 4 sprints to get a team off to a solid start. Where possible we would graft our workshops onto actual scrum ceremonies(planning, grooming, review & retrospective), using 'live' input, so that on top of having imparted whatever theory was necessary for the activity in question, the workshops would produce actionable output, in the form of the familiar scrum artefacts (realease & sprint backlogs etc). In this way we hoped to do more showing, though doing, than telling and to be one step ahead of anyone inclined to feel that our learning was getting in the way of the 'real' work.

We also wanted to just hang out a bit, on the one hand to explain and discuss and help out where possible with new ideas like swarming, pairing, WIP etc, should the need arise, but also to get to know the team and experience the physical environment in which the work took place, attend stand-ups, help with setting up other ceremonies etc., etc but we did not know how much time this would represent (we had no data from previous efforts). Wouter and I were also keen to pair throughout the pilot transition; we knew we had an awful lot to learn too

So, to remain honest and open in the relationship, and further sweeten the deal – we offered all our consulting hours for the pilot at zero euro. In this way all parties would have at least some basis to predict costs/effort before committing to anything truly substantial and Wouter & I could both hang around as we saw fit during the pilot.

It was a good deal. Our return, besides 2 paid workshops (we agreed that we would only charge for workshops if they had been 'tested' at least once before), was the opportunity to learn, structured feedback and some word of mouth advertisement. Of course, with a bit of luck, the whole thing would be the start of a mutually fruitful, and profitable, collaboration.

We got the go-ahead, the only decision that remained to be taken then, in early August, was with which team we would cooperate on that pilot. We were invited back in early September to discuss particulars and possible start dates. Little had evolved in the holiday month in between but the establishing of certain communication patterns. I was champing at the bit at this meeting, and it was here, is my own analysis, that the seeds of future failure were sown - if it had been chess this would have been my fatal error:

The team we were to work with were busy with the UAT phase of the current release of their product and their next development cycle was to start sometime in November. For various reasons, it was difficult to say when we could get the whole team together in the meantime such that we could discover of them what it was they did and how approximately (we had decided this should always be our first step when getting involved with a new team).

In the face of all this uncertainty about dates and states and next moves, I boldly suggested that we “just start!”. Wouter and I would hang our Basics workshop on a planning meeting and we could get learning in this UAT period, which for everybody but Test, wasn't actually that busy a period. We could do this at the drop of a hat, so why not? The team was unstable they warned us! My riposte was that we could get to know the team in (informal, relaxed) bilateral meetings after the first session, as the opportunities presented themselves. If we acted now, I asserted, the basic scrum cadence could be well established by the time the next development cycle started! (Privately, I also congratulated myself on how easy it was going to be in the UAT period to get the whole team thinking about testing as a whole-team activity)

How unstable could the team configuration be? All agreed.

As in chess, as in life - in chess too my downfall is often impatience, or hubris, depending on your perspective.

When we met (most of) the team on the morning of the Scrum Basics workshop, some of them for the first time, it was immediately obvious that the decision described above had caused some damage. The team felt crept up upon, and in some cases, put upon, the PO-elect among them.

And yet everybody pitched in positively, we had made no secret of the fact that we were new to the coaching side of things and the team generously allowed us some early fumbling, the workshop was a success. A sized sprint backlog (comprising outstanding UAT work as the team knew it at that moment) was produced and the scrum board setup and prepared. At the same time we had delivered most of our Scrum Basics slides. The perfection game feedback for the session averaged 8...

Lengthy sessions with the PO, the testers and the developers respectively took place during that first sprint. From our point of view it was essential to gain a better understanding of the domain, and the general technical and organisational parameters. But we also used these sessions to follow up on any questions or other input the team might have regarding specifically the adoption of scrum. After the fact it transpired that we had in some cases achieved the exact opposite of our aims, coming across at best, as hard of hearing, and at worst, as cerebrally challenged.

We had agreed to try and keep the number of open stories to lower than half the number of team members. This forced the team to search for new ways to cooperate during this first sprint and things seemed to go quite well initially, but eventually a lot of worked piled up in test anyway. All the skills necessary to deliver a story were available within the team but historically roles had been clearly demarcated so the team needed time to get to know one another's work. The sprint was successful though, twice as many story points were eventually delivered than that the team had committed to in the planning meeting!

On day n-2 of the sprint however, there had in fact only been one single story point actually delivered, DONE. Essentially the team had 'flatlined' all the way through the sprint and on the last day or two managed to get everything done and dusted. Some points were basically delivered through negotiation.

I had introduced the scrum master to Derby & Larsen's “Agile Retrospectives” and we discussed setting up the review and especially the retro along the lines as laid out by the “retrospective goddesses”. The stated goal of the retrospective, having dwelt briefly on the successful delivery, would be the production of one “improvement story”. In our case, a story that would start the team on the road to finishing with starting and starting to finish and give us the opportunity to discuss WIP limits and all that good stuff.

Surprisingly (or not) there was some resistance from the developers when it came to the demo. There wasn't much GUI work involved and everybody had already worked so closely together the whole sprint, telling each other, every day, what they were doing, on a mature project, etc. We insisted anyway and were quietly delighted when almost every single story demonstrated resulted in lively discussion.

The planning meeting for the second sprint under our guidance was less of a success. There was confusion concerning who should have done what to be prepared for the meeting and some of the team were in London while the meeting was being held. It was a difficult session but it clearly surfaced the necessity of excellent communication infrastructure when not co-located and more importantly; latent problems in understanding the scrum roles, and so was very useful. Again we produced a sized sprint backlog, including the improvement story that the retro of the previous day had yielded. But it was also becoming apparent just how badly I had underestimated what people had meant when they talked of team instability in the project phase (UAT) the team found itself in - during that meeting one of the team was called away such that he could put his development skills to good use on another project. And then there was our impending absence...

The week this 2^nd sprint was to end, Wouter and I would be away, firstly at LKBE11, and then CSD training with Ron Jeffries & Chet Hendrickson, so we tried to schedule dates with the PO and the Scrum Master for the two remaining workshops we wanted to give such that we wouldn't exceed the 4 sprint maximum we had agreed with management. We were also convinced of the value that they would deliver. That planning meeting would not have been so problematic if we had had the chance to cover backlog management.

It proved difficult to set a date for this (or any other) workshop over time. Not only was it tricky to get everyone together (as we had been warned it would be) there was also a difference of opinion as to what material could be used as input for these workshops, the team was suspicious of any information that had not been signed off and the PO was wary of re-work, it seemed our hands were tied. Wouter and I had long and involved, but ultimately unproductive, discussions about how to get which management to give what signal. The last day at the client's just prior to our planned absence, having failed again to set a date for either workshop, I looked back for the first time at my insistence that we start immediately with serious trepidation. The team clearly felt that there was no room for mistakes.

On the upside though, at that point in time there were three days to go in the second sprint (sprints ran from Thursday to Wednesday) and the scrum board had been looking okay. The following Wednesday evening I mailed the scrum master and asked how the sprint had run down, how she had got on with the review & retro and if she was ready for the planning meeting the next day, or if there were any problems, questions etc... I received an evasive answer explaining how the sprint had been completed successfully, and that the team had delivered many more stories than anticipated.

I surmised that the improvement story had not been done. That there had been neither a review nor a retrospective and that there would probably be no planning meeting the next day.

When I (Wouter was at the Scrum Gathering in London, the lucky lucky...) turned up at the office the following Monday my worst fears were confirmed. The scrum board was closed and abandoned. When I opened it I saw the remains of the previous sprint (with indeed many many post-its in DONE). The improvement story however had not been touched...

I asked the member of the team sitting closest by, “What's going on, how come we don't have a sprint backlog, weren't there any ceremonies run?” They had collectively decided that there was no point for the moment, the workload was too variable, more people had been moved off the team (that was also why the improvement story wasn't picked up in the end, the guy who was to pick it up had been moved...) and so on.

I sought and found our sponsor. She explained that there had been some delay in signing off the next release with their client and that they had therefore re-assigned as many people from the team as they could. (It was clearly not the right moment to re-open the discussion on the advantages of stable teams.) There was however no doubt that the release/project would go ahead. It was just going to start a little later, probably the end of November (it was the middle of October at that point). We decided to leave things for a while until there was more clarity. Then, we could come back and finish off our transition with the hopefully largely re-grouped team. Precisely how we would start up again at a later date, we would leave up to the circumstances and how much material we still needed to cover.

In the mean time, the next day, we (our sponsor and I) ran an impromptu retrospective of the pilot transition. We wanted to gather feedback both on scrum, and the coaching. I was beating myself up about how badly we must have done our job - the first chance that had presented itself, everything had been ditched, by everybody!

Upward and onward, the retrospective itself went very well. Using the timeline activity we gathered data and emotions. We then dot-voted to bring some structure to our reflections. Some difficult discussions ensued but we all stuck to the task and reviewed events and each other honestly. The upshot of it all was that we coaches needed to be much more aware of our customer's narrative, where they were coming from if you will, and that the team needed to be more open to change, and dare to challenge their managers to change. I left that day much happier than when I had arrived, as far as I could tell, despite the problems, the team were still onside:

I was all full of the value of failure, I tweeted as follows:

And then fate struck. It's not just bad decisions that are required for failure, you must also have some bad luck along the way. A herniated disc in my lower back laid me low as of late October.

Wouter returned to our client's on his own in early November and they agreed that we would get a call the week before the re-scheduled project was due to launch (as of that moment, the week of the 21^st).

In so far as I was able (treatment for herniated disk is mind-bending painkillers & rest) I helped Wouter to turn our Backlog Management and Planning & Estimating workshops into a sort of project kick-off built on the framework of Rasmusson's Inception Deck so that we were ready for any eventuality.

Meanwhile, time ticked on, and on. And the promised phone call never came.

When contact eventually was established, we heard that “everything was a go. Somebody just needed to check back with somebody else...” Now I know my project manager's instinct would have had me knocking down the door at the our customer's long before their deadlines for calling us had passed, but would it have made any difference? Ultimately? I don't think so, as I said, I think a little patience on my behalf, way back in September, would have served us all well!

Anyway, another 4 days went by, and then we were told that our services were no longer required. The project was up and running, what was the point? The rest of the teams had also decided to transition under their own steam. We had missed the boat.

And so, the balance.

On the negative side:

• We did not successfully sell agile and I am concerned for our customer's adoption
• We did not complete the pilot transition
• We ran only one billable workshop
• We did not test our new workshops
• Direct return on investment was negative

And on the positive side, the value:

• We now have a thoroughly tested Scrum Basics workshops, which we can execute on various frameworks (retrospective, planning meeting)
• We have working experience as (external) coaches and have sent an invoice
• We completed our Planning & Estimating workshop
• We added the bones of a Project Kick-off workshop (framework inception deck) to our portfolio
• Directly or indirectly, our work there was the stuff of a number of  blog posts!
• Learning, learning, learning...