Notes / Lead ranking

A new lead order is only useful if you can test it.

Lead ranking is about making sure the team calls the right people first. At a US consumer app I built a playground so the business could compare rankings through A/B experiments, rather than changing the order without a comparison.

Lead ranking has one job: make sure the team calls the right people first. A new order of leads is only useful if you can test whether it improves the outcome that matters.

At a US consumer app, a lead-ranking change needed to be evaluated against its effect on the business. Without a comparison, a new order is just a different sequence of calls, and you cannot tell whether the team is spending its time better than before.

I built a lead-ranking playground for comparing alternatives through A/B experimentation. That made ranking behavior something the business could examine and test. The playground put alternative orderings of leads side by side and measured what happened.

I cannot publish the ranking method, the experiment allocation, or the stack. Those details remain unconfirmed, so this stays at that level. The public case study also does not include the experiment duration, sample size, or significance analysis. I am not going to fill those gaps here.

That limit is part of the lesson. A ranking result is easy to over-read if you cannot see how the test was run. The case study says as much: a revenue result needs the context of its experiment, and the published account does not establish how much of that result transfers to another audience, ranking approach, or business. If you cannot see the test, you should not treat the outcome as a portable recipe.

When the team's work is calling leads, a worse order means time spent on the wrong people. A playground is a way to compare orders before the whole list follows one of them.

The case study for this work is Making sure the team calls the right people first. It keeps the reported outcome on that page, along with the same limits I have described here.

If a ranking or operations problem like this is sitting in your business, I take that work through Developer Labs.

Bhavin Kamani

A small command line for a curious visitor.