065 - Balancing Human Intuition and Machine Intelligence with Salesforce Director of Product Management Pavan Tumu

Experiencing Data w/ Brian T. O’Neill - Data Products, Product Management, & UX Design

I once saw a discussion on LinkedIn about a fraud detection model that had been built but never used. The model worked — it was expensive — but it just simply didn’t get used because the humans in the loop were not incentivized to use it.

It was on this very thread that I first met Salesforce Director of Product Management Pavan Tuvu, who chimed in on the thread about a similar experience he went through. When I heard about his experience, I asked him if he would share it with you and he agreed. So, today on the Experiencing Data podcast, I’m excited to have Pavan on to talk about some lessons he learned while designing ad-spend software that utilized advanced analytics — and the role of the humans in the loop. We discussed:

• Pavan's role as Director of Product Management at Salesforce and how he works to make data easier to use for teams. (0:40)
• Pavan's work protecting large-dollar advertising accounts from bad actors by designing a ML system that predicts and caps ad spending. (6:10)
• 'Human override of the machine': How Pavan addressed concerns that its advertising security system would incorrectly police legitimate large-dollar ad spends. (12:22)
• How the advertising security model Pavan worked on learned from human feedback. (24:49)
• How leading with "why" when designing data products will lead to a better understanding of what customers need to solve. (29:05)