When you have a team that is creating some output, delivers something on a regular basis, but doesn’t have sprints, story sizes, planning or velocity for any of its history, how can you say if the team is doing better, worse or the when comparing any period to a previous period?
Going one step further, there are more useful and valuable questions to be answered.
When should we expect delivery of any specific [story, bug, epic, release, version, MVP, prototype]?
How much should we expect someone’s time off to affect a future delivery?
What can we delivery next week, next month or this quarter?
(…)
Remember that no number on its own will ever paint a minimally complete picture. That’s true even for the best devised metrics, but it’s even more insidious when looking back at things produced without organized and intentional measurement/recordkeeping.
If you don’t have story points or other pre or post size record, consider counting completed stories, bugs, tasks, epics, etc.
Count Pull Requests and/or Code Review rounds. As much as possible, try to qualify and add context to the data. Consider average, median, minimum, maximum, outliers and special cases. In some cases it will be useful to give each Pull Request some dimension, so you may want to also count number of files added/removed/edited, number of comments, number of reviewers, number of commits, etc.
Count files changed, repositories touched, commits, rollbacks (if you can identify them), lines of code modified, etc.
Count deployments, separate development from different acceptance levels, from production. If manual work is involved, count hours spent on them.
Count test execution results/reports! And if the count is near zero, you have some other parallel initiative to work on!
There may be more than one angles to this idea. Maybe the more obvious one is to be careful to not fall into the trap of trying to measure too much, too far back in time. You may end up with a lot of data, but it may not be useful or even usable. Another angle is to try as much as practical to keep a moving time window, so that you don’t have to guess or estimate the impact of external factors, such as holidays, vacations, sick days, delivery crunch, extra work hours, etc.
Under control doesn’t always mean short, but it often do, and how short is a very contextual question. When you shorten the time framing, you get more accurate data points, but if you shorten it too much, you will accurately measure something that might not be true or significant. There is no magic number, sorry.
Use spreadsheets, create pivot tables, create charts, graphs and dashboards, try your hand at some new tools or go further with some familiar ones. Find obvious patterns, outliers, trends, correlations, etc., but don’t make them answer final questions just yet. You want to explore the data you can get before you refine the answers you want to get.
Ask for opinions, too, but don’t overwhelm anyone else with the raw ideas that you have. You want to have a few good ideas to present, but leave plenty of room for others to contribute. You may be surprised by what others can come up with. You most certainly overlooked something, and others might point that out early enough for you to adopt, change course or even abandon ideas.