Staging the Backlog

The team and the Product Owner usually collaborates around the top of the backlog: around the "doable line" (as defined by the Definition of Ready), and the coming sprints (3 is a good number).

Even though we often envision the backlog as a refinery where large and fuzzy ideas gradually converts into doable User Stories, that is in reality never the case. This is because the items will never refine themselves, it is us that do it by working with one item at a time, dividing them into smaller pieces, answering specific questions, etc. The items will be refined in steps, or stages.

One of the benefits with the sprint-focused method of Scrum is that it defines very clearly one of the last of those stages: where features actually get implemented in usable code. Thanks to having a defined sprint-length, a Definition of Ready and a Definition of Done, we get a better overview of our actual capability to generate business value, and thus a better ability to plan within the scope of a couple of sprints. The well defined stage brings clarity.

There is a method to get that benefit and planning ability for larger scopes than sprints. The trick is to look at the backlog below the doable line and divide that into stages, just as Scrum defines the sprint stage, and try to define those other stages.

Cadences: Sprints and Features

A cadence is a rythm. Having retrospectives once in a month is a cadence, releasing software every third month is another, and sprint planning every second month is also a cadence. In By-The-Book-Scrum there is only one main cadence: the sprint, and everything (planning, demo, retro) is aligned to that (the other Scrum cadence being the 24-hour cycle between stand-up-meetings).

When adapting agile to your business, you often end up with a more polyrythmic approach. There can be weekly product-board meetings, retros every second sprint, releases every six sprints, and replanning of urgent items twice a week. That is completely in order and something that drives agility: the cadences will often reduce Mura and increase predictability, while still allowing for necessary variation.

Defining a stage before the sprint stage is about defining an outer cadence, a meaningful rythm that encompasses a number of sprints. The sprint is meaningful in that it is a time-frame large enough to implement a User Story in, but small enough for us to manage risk.

However, a single User Story in isolation won't be very useful to a real user. We need to group several User Stories into Features in order to realize real business value, something that a customer would be happy to actually pay us for.

So the next level of meaningful cadences, above the Sprint, would be the Feature Frame: The calendar time needed for our organization to realize something that the customer would pay us for.

Now, the time needed for us to realize a feature differs a lot. Some features are exactly one User Story, while others might span several. Being able to save in a certain format could be a selling-point targeting a whole new market segment, while the effort might equal a developer-tester-pair working together for three days. Other features, like a special mobile app of the GUI, might take whole teams working together for months to realize.

Small features aren't a problem. They fit into one User Story that will be realized in one particular sprint (or what you like to call your development cadences). It is everything that spans many sprints that pose a problem.

So the trick is to find a cadence that is large enough to cover the bulk (like 80%) of any new feature. An observation in many organizations is that most often, a feature will be realized within the timeframe of 6-12 weeks, with some tricky, but important features requiring double that time. Using sprints that are 2-4 weeks long, that would give us a meaningful cadence around 3 to 6 sprints.

The agile value "Measure progress in working software" where "working" is defined as "useful and in the hands of the user" implies that everything unreleased actually is waste, and that releases that consists of less than one feature is rather useless. Taken together, this would imply that there would potentially be 3 to 6 meaningful feature releases per year, each containing 1 or more features.

(Note that this reasoning doesn't imply that continuous releases is a bad thing. The actual release cycle might be much shorter than a sprint. Releasing improvements in your system nine times a day doesn't conflict with the idea of feature releases.)

So it would be a good idea to think of a Feature Stage in the backlog, before the Sprint Stage: each Feature Stage spanning 3 to 6 sprints. And if you have just a couple of releases of your software per year, think of aligning the releases containing the result of 1 to 3 Feature Stages (3-18 sprints of 2-4 weeks each, with a maximum of 52 weeks).

(In one organization I worked with, the Feature Stage was called a "Block" of 3-5 sprints with 1-2 blocks per release, 2-4 releases per year. In another organization, it was called an "Increment" and defined as 3-6 months worth of work.)

Precondition, Postcondition

When you have established one or two levels above the sprint (Feature/Release fx), you need to look into what would constitute the precondition and postcondition of those stages.

The postcondition is always determined by the following step. When the teams implement software, they need to state their Definition of Done so that the organization understands what is to be done in order to get the implemented increment released into the hands of the user. The Definition of Done must match the deployment organization.

The postcondition of the stage before the sprint should in the same manner match the precondition of the Sprint Stage, ie the Definition of Ready of the teams, or the "doable"-line as I like to call it.

If you use the 3-sprints-ahead method for grooming User Stories, you could have a postcondition of a feature-stage stating that you are done with your analysis when the feature is broken down so it will be fixable within 3 sprints. A precondition for could be that the user, the main use-case, and the business values are clearly defined. Given that, a group of architects and business analysts could collaborate into breaking the feature down into doable sub-goals at the User Story level.

The level above the feature could be a vision increasing the business value for a whole market segment. Here the precondition could be the vision, and the postcondition a number of features described as use-cases (or happy-path User Stories), that the Feature Stage could work with.

As always: your mileage may vary, and you need to find your own stages by considering what a reasonable cadences, post- and preconditions you have in your organization. The important realization is that you need to think through the stages you need for the transition from idea into executable business value.

Grooming 3 Sprints Ahead

Items on the backlog is often too large and/or too fuzzy. The process called "Grooming" is about the team helping the Product Owner to clarify and divide such items into doable User Stories, so that there won't be a lack of them in the sprint planning sessions.

But how do we do it? First of all you should know that it depends on your situation. Are there many urgent large or fuzzy items? The team need to assign more resources into grooming. Are they very large? Groomers need to have an architect approach slicing them. Are they very fuzzy? Groomers need to act as business analysts, seeking out answers from the stakeholders in the business.

Now, situations like those, where stuff comes in bulk, should generally be avoided, since it is examples of Mura, creating waste in the organization. It is better to have a situation where the amount of ungroomed issues is predictable and handled smoothly.

This is achieved by not grooming much more than you plan to implement, and by grooming ahead. I will give you a practical example of how that can be accomplished, in a way that I have found effective in many organizations.

First: set aside a number of grooming sessions during the sprint where they will hurt as little as possible. One hour once a week could be good for starters, and schedule the sessions so that they won't interrupt things. This gives everybody a sense of predictability.

Second: during sprint planning you briefly look ahead at the items that will need to be groomed. Identify what kind of issues they are (maintenance, new development, component A or component B etc), so that the team can decide who is best suited to groom them. Normally, you put a cap on the grooming saying that no more than 10% of the team's capacity will go into grooming.

Third: at the grooming sessions you look at the issues in three blocks:

• Issues 3 sprints ahead: Product Owner explains the issues, and receives a lot of questions and criticism from the team. Until the next occasion, these are questions that the Product Owner need to find answers to, preferably in the form of a live stakeholder. This is about reducing fuzziness.
• Issues 2 sprints ahead: Product Owner has found out answers to all the open questions, so that the team can start dividing issues into doable User Stories. This is about reducing size.
• Issues doable in the next sprint planning: The team and the Product Owner knows what stories there are, and preferable there should be an early estimate so that the Product Owner can prioritize together with the stakeholders.

Either in that order, or the other way around.

The method presented above gives you a structure to the grooming process. It will need to be adapted so that it fits with your other cadences: Product Owner meetings with the stakeholders, sprint planning, etc.

Have you drawn your "doable line"?

I often like the backlog to a refinery. In the bottom, you feed the refinery with the crude "oil": vague ideas, big proposals, great plans. As you and your organization work with the items (the goals) the items will eventually turn into useful increments in your product, slowly bubbling to the top.

From an agile point of view almost all backlog items suffer from two things: size and fuzziness. Size, because it is easy to come up with great ideas that also will take a great amount of time to implement. Fuzziness, because it is hard to know what you want.

The refining process for our requirements and wishes (the grooming) is basically about reducing this size and fuzziness. Can we take a slice of the requested functionality and implement into something meaningful during the next sprint? Can we get clear answers to the open questions so that we will know more exactly what we are supposed to do?

In Scrum we never let the Product Owner come to a sprint planning session with an ungroomed backlog: a backlog where the top stories (items) are too big, too fuzzy, or both. This is because such stories aren't really doable. Or: yes of course we can implement them. But not in any predictable manner. We will get stuck in trying to figure out the precise meaning of the requirement, and since the item is so big we won't be able to finish it within the limited time-frame of the sprint.

It will ruin our predictability and thus the ability to plan, as well as it will ruin our morale. The organization will start to feel uncomfortable with our ability to deliver.

It is always better to know early on that an item isn't really doable. We'd rather refrain from trying than fail from trying. Therefore many teams has come up with the Definition of Ready. If the Definition of Done is the team's post-condition (this is what we can promise about anything we have done), the DoR is the team's pre-condition (this is what all items should look like before we accept them into our sprint).

Typical Definition of Ready criteria (or "doable" criteria) includes: no outstanding open questions, and: can be implemented by two persons within the scope of one sprint. These two criteria targets the fuzziness and size directly. Other criteria could include: have an explicit way of testing the item, have the solution reviewed by the operations team, be a proper who/what/why-story (I'll explain later).

The doable criteria serves as a quality assurance of the requirements themselves. We know that we increase the risk if we don't follow them, so therefore we don't accept them into our sprint. We handle them at the grooming sessions instead, until they are refined enough to be considered.

At the top of your backlog, draw a line. This is the "doable" line. All items that fulfil our Definition of Ready may pass that line from below. Everything we plan to include in our sprint is taken from the items above the line. Smoothness, predictability, information generation and distribution, and high quality requirements will be the result.

Cashing In

As both the Agile Manifesto and its underlying principles states: it is shipping things that work that matters! The manifesto specifically mentions software, but that is merely an example. Making valuable stuff available for people is what justifies our existence no matter what we are delivering be it a physical device, a painting, or a theatre production.

You should regard the very release moment as the point in time where you actually accomplish things. Everything up to that point is merely a cost: things have been thought out, planned for, implemented, tested, documented. But no real value has been created, until someone starts to use the "product" (agilistas use the term "product" in the broadest possible sense).

This means that everything you do with your product between releases is creating debt: you invest time and effort in it, but you don't get any benefits. Doing things creates a cost, but every day not being able to ship is a day with no gain. Especially not gaining insights on how your product is received by its audience. You won't get no inspection, and thus have no ability to proper adaption.

Both factors: high debt and slow feedback loop, leads to high risks and a costs. If you want agility in your organization, no matter what role you have, you have to focus on releasing! Focus on cashing in!

While there are some organizations that are able to release many times a day, it is not uncommon that the release process is expensive, due to structural factors in your own organization, and may not even be wanted due to structural factors in the receiving organizations.

The good news is that even if we won't be able to deliver (and no one will be able to receive) that often, we will still benefit from making everything we do easy to release. Hard to release means expensive to release, and that will be a cost even when there is no pressing need to release early. And when there is a pressing need to release, the cost associated with having the release take so much calendar time will skyrocket.

So take a critical look on your release processes. Could you be able to release more often, at least internally? What would it take to bring down the costs associated with releasing? What can you do to minimize the calendar time, so that you won't lose being slow to market, or slow to release that critical bug fix? Focus on cashing in.

Product Ownership is Goal Oriented

According to Scrum, the Product Owner is the person responsible for the ROI of the "product" (whatever a "product" is in your world). Being responsible for the ROI, the business case, simply means that you are to make sure that the value brought to you by the product over time is higher than the cost of taking you there.

The cost comes from having people do things to the product. Those things we call "tasks". Goals is mapped to the tasks needed to achieve them.

Now, if we set a goal for the product that should be met once a year, they will probably be mapped to a huge amount of tasks (or else we should be able to meet the goals much earlier). The problem with that is that we will never know where we are in relation to our goal until it is too late. Yes, we can follow up on the tasks during the year, but we will not be sure whether the tasks ultimately will make us achieve the goal. Halfway through the task-list: are we really halfway to the goal?

This is a symptom of having feedback-loop that is way to slow. And we won't move faster in the loop if we just follow up on the tasks more often and check whether they are completed in due time. The completion of tasks alone doesn't tell us where we are in relationship to our goal.

This is why successful product ownership is goal-oriented in all stages. The interesting question is always what the product's capabilities are at a certain point in time. When detailing what we are trying to achieve, we find lots of sub-goals that need to be met in order to reach there. When we specify and follow up on those sub-goals instead of tasks, we will always have a clear indication on how far we are from meeting the greater goal.

User Stories are Goals

The concept of user stories supports this. A user story is the most fine-grained unit that exists in the backlog, and specifies a small feature that is useful for a certain kind of user (which is specified, and because of a business reason, which is also specified). In order to be doable in a predictable manner by the developers, it should also be small enough to fit into a predefined time-box (such as a sprint in Scrum). A user story doesn't specify a task, doesn't tell anyone what to do. It declares a small goal that the product should be able to meet. Goal oriented.

So as product owners we need to find a way to work with the goals, from the targets that may be set yearly by a product management, down to the doable user stories that are to be implemented in the following sprint. What tasks need to be done is up to the teams to figure out, we should focus on goals.

There are a couple of strategies one can use for both goal refinement and goal break-down, and I will return to them in time. I recommend for example a classification where a Business Goal is broken down into one or many Features, where each feature is implemented through a set of User Stories, of which some of them are Epics (= too big stories that need to be broken down further).

But no matter what strategies you employ, make sure that all your backlog items - from the largest to the smallest - are expressed as goals.

The Pressure Owner

Since one frequent source of problems for a software developing organization is when conflicting goals are communicated from the business to the developers; one success factor, common for many different agile methods, is the introduction of a Product Owner.

The PO is a person with basically two responsibilities:

1. Keeping all wishes from the business sorted so that you always can see what is more important than anything else.
2. Being able to clarify exactly what those wishes - requirements - are.

In other words, the developers say: "Dear PO, you should represent the whole business to us, now go and figure out how!"

It is no wonder that Product Owner burn-out is an expensive and painful reality.

The problem is that the role is very clearly defined, but the tools for fulfilling the role is almost non-existent. As a PO, you know from the Scrum Guide that you should come to the Sprint Planning with a well-groomed backlog, but you get no advice on how to put a backlog together in the first place.

The PO has no well defined method or process to stick to. That creates an awful amount of stress.

From my experience in acting as a PO in both large and small organizations, and coaching groups of PO:s in very large organizations, I have seen that one key is the ability to translate what works in Scrum, into something that does it for PO-work.

The thing behind sprints for instance, is the introduction of a common inspect and adapt loop between the askers and the doers, at regular intervals. Points in time where everybody knows it is time for some serious collaboration across organizational borders. As a PO, you need to identify the need for other cross-organizational inspect-and-adapt-loops in your neighbourhood, and then make them happen.

I will follow up with some examples of how that can be done, you could even call it a concrete method for how PO-work could be done, no matter what method the developers use.

But for now: realize that the PO-stress is real, common, and just a consequence of the developers asking the business a very normal question: Exactly what should be done now? You need to bring that question out to the business in a structured way, collect the answers, and go back to the developers. I will soon show you how.

A Field Manual

My name is Ola Berg.

For quite a number of years now, I have been working as an agile developer, coach, teacher, and project manager. Based on my experience I have come to see that there is a need for a collection of tools and tricks that you can refer to when you try to make agile rock in your world.

Agile methods, such as Scrum or XP, aren't in fact agile. They are quite rigid. But they have the ability to make your organization agile, if you implement their practises well. I think that a collection of good ideas would be helpful for that. This blog is the sketchpad for such a collection, a collection that I call "A Field Manual for Agilistas".