In a survey of 885 venture capitalists at 681 firms, conducted between November 2015 and March 2016, the average firm reported calling 10 references per deal, within 118 hours of diligence on a deal that took 83 days to close. Firms investing only in mid- or late-stage companies reported 13 references, and firms investing only in seed or early-stage companies reported 8. These are self-reports from a sample the authors note may lean toward more successful firms, and the survey counts calls without saying who received them or who chose the names.
Ten or thirteen calls is a small sample. When the company chooses the names, it will pick satisfied customers, as any company would, and an investor learns where the product works and little about where it fails. For a growth-stage software company I would ask for a different list before scheduling any calls: every opportunity that reached a serious stage in the last few quarters, won or lost, and every renewal in the same period, sorted into expanded, flat, contracted and churned. The investor then chooses whom to call from that list.
The useful comparison is between won and lost deals that reached the same stage. Define the stage by what it cost the buyer, such as a priced proposal or an opened security review, so that every deal on the list was a real evaluation. Then test the recorded loss reasons against the wins. If the CRM blames price for most losses, look at what the company charged in the deals it won in the same segment. When those wins closed at similar prices, price alone does not explain the losses, and the buyer calls should look for what does.
That is where lost buyers earn a call. A buyer who sat through a security review is in a position to say what stopped the purchase: an implementation that would have run past their own deadline, integration work their team could not staff, a requirement the product failed. When several describe the same obstacle and it has nothing to do with price, the company has a problem a discount will not fix, and adding sellers will produce more of the same losses.
Renewals get the same treatment. If the investment case assumes expansion inside existing accounts, a customer that renewed twice without adding seats or modules is a direct test of that assumption. A blended net revenue retention figure cannot show how many such accounts exist or whether they share a segment, a region or an implementation partner. The sorted renewal list can.
The way the company produces both lists is also evidence. If loss reasons sit in an optional free-text field, or renewals cannot be sorted without days of manual work, the company is managing its losses from memory. Some companies will have good reasons to refuse to put an investor in front of lost prospects; the export and its notes can still be shared.
A fair objection is that lost buyers make poor witnesses. Some will not take the call, those who do may give a polite reason, and the handful of calls a process can spare cannot estimate a win rate. I accept all three points. The export does the counting and the calls supply the mechanism: one buyer’s account of an integration nobody could staff becomes a hypothesis the export can check across every loss in that segment.
The memo that comes out of this should name the two or three loss patterns I expect to persist as the sales team grows, the engineering or delivery time each would take to fix, and the quarter in which win rates in the affected segment should move if the fix ships. In the same survey, only about one respondent in ten said they quantitatively analyze their past investment decisions. A dated prediction gives the fund something to check against what happened when the company returns for its next round.