The Importance of a Healthy Backlog in Product Development: Patterns and Anti-Patterns

When developing a data product using Agile methodologies, Product Backlog is a determining factor. The Product Backlog is the conceptual representation of the product under development and therefore, must be taken care of, always well ordered, changed and constantly reviewed because a failure can trigger costs and generate unexpected problems.

It seems that everyone knows how to define and build a product. However, the reality is that when working with Agile, many errors appear due to old habits that cause more definition problems than expected, some of them becoming anti-patterns of the product.  Here are some examples of where this happens.

1. PRIORITIZATION BY A PROXY PRODUCT OWNER

One of the mistakes that can be observed in companies that decide to develop their products with an Agile framework, is that there is no empowered Product Owner (PO) in the decision making process. The role of PO is often associated with that of Business Analyst and this makes many people think that all that is required is to define requirements and as a result, the role is allocated to a person without the ability to make decisions autonomously. This is the famous Proxy Product Owner. Usually decision-making is delayed because when problems arise, this type of PO must consult another person who is more empowered, and then communicate decisions back to the team and so on and so forth, thus wasting valuable time.

2. THINKING 100% IN ADVANCE

Another common mistake is the tendency we carry over from previous ways of working to define everything in advance. Usually, billing models are prepared to deal with closed projects rather than Agile projects. When companies talk about working with Agile methodologies, their priority is getting the job done faster, not changing their internal processes of ordering, invoicing, payment, etc. 

These processes mean that companies often need to have everything “laid out” in advance in order to be able to initiate certain necessary bureaucratic processes: purchases, orders, bids, invoices, etc., which is where the problem arises. As this approach is taken in advance, an implicit contract is generated that is difficult to change. As the decisions seem to have been already made already, subsequent changes will be difficult to make.

3. OVER-DIMENSIONING

Another consequence of thinking everything through in advance is that it generates a gigantic backlog of things to do. This often leads to a focus on things that don’t necessarily end up being developed, with the consequent loss of focus on current development. There is also a feeling of lack of progress due to this excessive backlog. The larger the backlog, the more static it becomes and the harder it is to review completely.

4. OUTDATED PBIS

Another consequence of having an oversized backlog or trying to detail everything that needs to be done in the project, is that many PBIs (Product Backlog Items) become outdated and obsolete. An Agile approach indicates that older items should disappear from the backlog. They may no longer make sense in the way originally intended or the need they covered is now extinct. The market is constantly changing and what is necessary today will no longer be necessary in a matter of a few months. So how long should an item be in the backlog? It depends, but if a GDP has not been updated in two months, we must consider whether it makes sense or is still important.

5. ESTIMATING EVERYTHING IN ADVANCE

Having an estimated backlog can be a great help when planning or also prioritizing some GNPs against others, based on effort, complexity, etc. However, we must be very careful because the investment we make when estimating is not always recoverable. A backlog must always be estimated lightly, so as not to invest too much effort. If you are working in an Agile methodology, the environment and the market may be very changeable. In this case, making a very large estimation effort can be counterproductive, because it can change its scope, and therefore its complexity in just a few days.

Another problem is thinking of this effort valuation as inalterable. Experience says that surely everything will have changed since the time it was estimated. In addition, if the backlog is very large, that is to say, over-dimensioned, this can result in a lot of effort only for many of the GNPs to have to be discarded.

6. WORK BASED ON COMPONENTS OR PARTS

This is another classic example of a poorly conceived product. Sometimes products are broken down into parts, layers or components that make it easy to see the architecture, design, etc.

A good backlog must be based on the needs of the customer, user, market etc. In this way, the potential of working with Agile methodologies can be fully exploited. When the backlog is based on parts, layers or components, this results in a well-known problem: dependencies. Working in component mode generates unnecessary dependencies that cannot be solved within the team and so development slows down: you have to define APIs, wait for another team to finish, integrate the solution, etc.

In addition, when working with components, another issue arises that attacks the Agile Manifesto, and that is that the architecture tends to stop being emergent. Furthermore, once dependencies exist, self-organisation is no longer so easy.

7. POORLY DESIGNED PBIS

One of the obsessions of many POs is to have stories in a user story format. We must not forget that the concept of User Stories is only one of the possible formats, but not the only one. Many people believe that an advantage of Agile is that things don’t have to be defined, that everything emerges; some even go so far as to say that a simple sentence is enough to define a need. Sometimes what is forgotten is that for a basic definition to be really useful, there is a minimum you must define. What you expect to achieve with that functionality has to be made very clear, regardless of the format it is written in: user history, requirement, feature, etc.

The bare minimum required when defining an element of the backlog are the acceptance criteria. What do I expect to achieve and how do I expect something to work? It seems unbelievable, but most of the time this is enough and from here things do emerge.

8. OVERLY DEFINED PBIS

In contrast to the previous point, we can go to the other extreme: over-defining a GNP. This leads to the concept of paralysis by analysis, that is, not moving forward due to having defined absolutely everything. The key is to find a balance between information that needs to be written down and information that needs to be understood. A PBI must generate and be confirmed by a conversation, not replace it. When we have a very defined PBI it seems that there is no need to talk or ask questions and this can create disconnection between the PO and the development team.

9. LACK OF A SINGLE INITIAL VISION

The consequence when business and development do not work together from the start is lack of a single shared vision. Sometimes it is believed that by having everything defined, vision will be clear. However, discussion of the needs and vision of the product can help avoid many problems. In order to talk about that unique vision, it is important to clarify the objectives of the product. If these are not well defined and shared by all, the system ends up failing. Decision-making is affected and wrong decisions are made. In addition, if not aligned with the objectives, refinements will not work properly. They tend to be more ambiguous and to drift without ever working towards the shared objectives.

GOOD PRACTICES FOR A GOOD BACKLOG

Thus if we want to have a good quality, healthy product we must try to follow these guidelines:

• There must be a real Product Owner, with the ability to prioritise, with all information necessary as well as total power and authority over what happens in the product. Only in this way can the right decisions be made, from a business perspective.
• Working from the current need and not expanding all possibilities allows you to provide the greatest value at all times.
• Focus on the short term. The only elements in the backlog should be the elements that you plan to execute in the next 3-4 sprints. You can use a Roadmap, User Story Mapping etc, but these should not be mixed with the short term.
• Comply with the previous point and also run many refinements to clean up the backlog frequently.
• Use basic estimations only as a decision tool. Estimates should not be seen as unchangeable, but should be reviewed as part of the refinements.
• Conceptualise the product around full functionality to avoid unnecessary dependencies. The time invested in solving dependencies is an avoidable waste of time.
• Clearly define acceptance criteria of the elements of the backlog in way that can be validated and observed.
• Strike a balance between defining too much and too little.
• Always work the backlog with the business objectives in mind. This will ensure that our backlog is always correctly prioritised and targeted to provide the greatest possible value according to the objectives we wish to achieve.
• Always keep the Agile manifest in mind and make use of it throughout the product lifespan.

Image: unsplash | @irfansimsar