A Product Managers Guide to Adversarial Selection
John J. Schaub
Dec 18, 2022
One of the interesting aspects of working in a relatively newly developed field like Product Management is the fact that until quite recently there has been no formalised educational pathway. In practical terms this lack of standardised pedagogy means that you have Product Managers with widely differing backgrounds and skill sets and what might be obvious to one is a completely unknown concept to another. I previously covered one of these topics in my post on Fermi Estimation a few months ago and today I’m going to talk about another slightly more obscure topic that every Product Manager should understand - Adversarial Selection.
Adversarial Selection is certainly a more niche topic then Fermi Estimation but it is one of those things that when it matters it really matters and not being aware of it is going to essentially evaporate your chances of launching a successful product. In my early career I encountered Adversarial Selection but it wasn't until later in my career when I stepped in to run my employers insurance subsidiary that I really got a formal education in the topic. Insurance is by far the easiest way to quickly understand Adversarial Selection and how absolutely destructive it can be when building a product so I’ll use that as an example.
Imagine you are 23 years old in fantastic health and someone offered you a no questions asked life insurance policy, would you be interested? Perhaps you would but you would probably choose to spend your money somewhere else and would only take it if the price was very competitive. Now instead imagine you are 107 years old and your doctor just diagnosed you with an incredibly fatal form of cancer and someone offered you a no questions asked life insurance policy, would you be interested? Of course you would, you’d probably feel a little bad about essentially robbing the insurance company but you’d certainly sign up at almost any price. That’s it you now understand Adversarial Selection and why there is no such thing as a no questions asked life insurance policy. If such a policy existed the company offering it would go out of business almost immediately as all the riskiest buyers flocked to them or would get absolutely no business at all if they set their rates to a level that reflected the risk levels of their average client.
In simplest formal terms Adversarial Selection occurs when the buyers and sellers in a transaction have an information asymmetry and one side can selectively choose to engage in the transaction or not based on that information asymmetry. This is not a difficult concept to understand when you put it in terms of something obvious like insurance it seems almost trivial but the interesting thing is Adversarial Selection shows up in all sorts of transactions and when Product Managers fail to account for it disaster looms.
A perfect example of a Product Manager failing to account for Adversarial Selection happened last year when multiple institutional investors started using pricing algorithms to make purchase offers on residential homes in the United States. Pricing algorithms are never perfect and this situation was essentially pitting the pricing accuracy of a pricing algorithm against a homeowner who knew every detail of the house and the surrounding market. Obviously when the pricing algorithm resulted in an offer below what the homeowner felt they could get in the market the homeowner would choose to sell on the open market and when the algorithm resulted in an offer above the market the homeowner would readily accept that offer. The result of the lack of understanding of Adverse Selection was the almost immediate and spectacular failure of multiple similar home buying investment hedge funds and a significant stock retrenchment for a few major companies.
So how do you avoid or at least mitigate Adversarial Selection when designing products? Step one is simply to be aware that the concept exists and ask the question with any product launch ‘Do I need to consider Adversarial Selection?’. Having a formal checklist of basic ‘did we turn off the stove?’ style questions to consider before building a product is one of the hallmarks of an experienced Product Manager and this question needs to be on that list. If you work in Fintech as I do you should expect that the answer to this question will often be yes and you need to be prepared to address it.
Once you’ve identified an Adversarial Selection risk to your product the next step is to determine how you can reduce the information asymmetry that exists. In the case of life insurance this is typically a form that asks questions about age, health, habits ect. In the case of the home buying hedge funds this should have taken the form of a human review of any purchase to sanity check the algorithm and insure they were not over paying significantly for a given home. This step need not be complex; you just need to eliminate those situations where the other side of the transaction has vastly better information than you do.
Next up you need to put in place some sort of safety to protect you if somehow the other side of the transaction has maintained a significant information edge. The most obvious way to do this is by adjusting your price to account for a significant information asymmetry in any transaction but beware that moving your price will impact the sorts of customers that are willing to buy your offering and can in fact make your Adversarial Selection issues worse not better. That is, the only customers willing to pay through the nose for life insurance are those that are very likely to need it so by adjusting your price you are likely to make Adversarial Selection worse. (IE you'll only get the highest risk customers which you really don’t want). Rather than taking such a crude approach you need to fall back on your Product Management skills and treat your launch as an experiment. Even if you are absolutely confident that your model is solid, never do a big splash launch like the home buying hedge funds tried. Instead start small with a select few transactions, monitor the outcome as you launch and keep monitoring it as you grow. If you notice sudden changes in the population using your product or find odd patterns emerging do not be afraid to tap the brakes and investigate. As an example the home buying hedge funds ended up buying vastly more homes in some markets likely because the algorithms they were relying on skewed too high in those specific markets. If their product team had their eye on the ball they would have see that problem a mile away during a rolling launch.
Finally you need to consider risk migration tools that might be available to you. In insurance for example it is quite rare for an insurance company to actually assume all the risk in a policy they will instead sell a portion (potentially a large portion) of that policy to reinsurers. By doing so they both ensure that they are not taking on all the risk but they also force themselves to regularly sanity check their underwriting processes by confirming that other insurance companies are willing to be part of the policy they just wrote. In simple no jargon terms you get a very good sense of how much of a risk you are taking on when you ask a group of smart people if they are willing to share that risk with you. If all the smart people you ask to join in back away quickly this is a strong message that you need to re-evaluate the product you are offering.
Hopefully this primer gave you a bit of a sense of what Adversarial Selection is and why you need to account for it. If you are building a Fintech product and need to bounce thoughts off someone don't hesitate to reach out.