Category Archives: Startups

Even good things come to an end – Why I’m leaving Klarna

2 Replies

I wasn’t sure I was going to write this post, but I’ve had several of these conversations in the past few weeks and putting my thoughts here is a good kind of closure.

The Analyzd team joined Klarna in 2011 to bring our way of doing Risk and Product to a company that was starting its international push. Klarna simplified payments in a way I found appealing but was easy to fraud and abuse, and what we brought to the table made a difference. It was also my first attempt at testing my philosophy about Risk completely on my own – not as a group leader in a small startup or a huge behemoth like PayPal but a company in hyper growth. Boy, was I in for a ride!

It’s been two years. I learned a ton – first, that this thing really worked. The amazing team and I managed to bring Klarna to new levels of fraud and risk prevention as well as technical prowess, introducing technical and predictive innovation that allowed for some of the lowest rejection and default rates in the industry on an annual payment volume of close to $2.5 Billion in online, real-time short term credit. Klarna Risk is not only a great place to work at and a well-oiled machine delivering results but also a great place to be from, and I expect many of the young leaders to move on to lead teams in the coming 5 years, much like the FraudSciences team did. I also learned what it means to be part of a hyper-growth company, raise money and work with some of the most impressive VCs in Financial Services and in general, work with regulators and traditional finance companies and many other things.

I also learned something else – I like building and inventing, much less so the exec type, and with the team’s maturation they needed me less. After much deliberation I decided that it’s time for me to go a build something new, and let the professional executives run the show. I’m leaving behind a big, mature and strong team led by some of the most talented people I had the fortune to work with and a company with a bright future, led by smart and capable leaders. I will continue to be bullish about Klarna’s ongoing success and help them out as much as I can, while I go back to square one and build something new, hopefully again something helpful.

I am slowly transitioning out of my role, and will stick around through the end of 2012, but come 2013 I am planning to be out and about. More info on my next steps to come in the near future. Stay tuned.

Feature companies in risk and fraud (or: enterprise and consumer startups are different)

Leave a reply

David Pakman quoted Tim Armstrong today: “I’ve seen too many feature companies get hot, raise too much $ and get way too overvalued.”

It is interesting to see that this is true in various markets. The trend is extremely obvious in payments/security as well, and is a by product of the boom in seed funcing for consumer startups. I wrote extensively about payments startups and how important it is to know where you are in the value chain. The thing is that I see the same in risk and fraud detection, where you’d expect the need for complete and complex products to be obvious.

Indeed, the concepts of consumer and lean startups trickled into enterprise; as a result, small 2-3 person teams are trying to build rudimentary detection mechanisms, mostly based on “social data” (a euphemism for opt-in Connect or scraping Facebook directly) and expect to position themselves in the market as serious providers in a short time frame. This is far from a reasonable expectation, however since money is abundant and is only looking for a way out of pure consumer plays, some of these teams get funded and end up overvalued and unable to cut losses with an acqui-hire, the most likely scenario.

While I agree consumerization of the enterprise is real, this is not a sustainable approach. The definition of an MVP (much more feature complete) and iteration (much longer) as well as what it means to do customer development is very different in enterprise. Small merchants continue to think more and more like consumers and are becoming more tech savvy, and that leads to more usage of SaaS tools and more openness to outsourcing some non-core activities (in eCommerce, fraud prevention may well be considered non-core). That doesn’t mean they are open to testing any new tool that gets put out there; the time as well as expertise to integrate and evaluate its performance may be more than they can afford. You can’t trust your tool to just get picked up at random to a reasonable scale and learn from there, unless you have a very big war chest; then we go back to the funding issue.

Case in point is device fingerprinting (DFP) companies. A few years back DFP (a lot of times a glorified javascript) was all the rage. Since it wasn’t a text or flash based cookie most fraudsters, themselves not more than script kiddies, did not have the knowledge or tools to properly resist being profiled. As a result, for a while it worked well especially in reducing short term horizontally scaled attacks. Only there were a few problems: overfunded companies built too big a team, especially heavy on the Sales side since Sales cycles with financial institutions were long and require a lot of patience, as well as multiple integration solutions. Since the teams were big and sales took time each contract had to be big, so pricing went up as much as possible rather than adapt a freemium model that could boost adoption. Moreover, once fraudsters and engineers caught on it was easy to circumvent or duplicate, either internally at retailers and banks and by competitors. As a result, most of these companies are struggling and dealing mostly with litigation against competitors for some negligent IP.

In enterprise, specifically in security, one feature isn’t enough, starting lean is more complicated, and just a feature will not do not matter how many patent you have pending. One option is to take your time to come with a holistic solution, and that is tremendously harder to build (in fact, since FraudSciences was acquired, only Signifyd and Sift Science have tried building a standalone risk-as-a-service solution). The other is to start very slow and very lean, and raise very little capital. MaxMind is a good example of the latter. It’s a whole different world out there now, especially for enterprise startups. Make sure you build a real product that can sell. Don’t built a feature.

Smart risk management: why the “factory” approach could bring you down

Leave a reply

This is a re-posting of a 2010 post.

If you deal with payments in any shape or form, you know you’re going to end up with a “risk management” team. A lot of times it creeps up on you: volume picks up and so you know you need someone to look at orders. If you’re running a small shop it’s most probably going to be you, but a lot of companies just hire one or two folks. These people use whatever tool you have to look at transactions – most times a customer service tool – and make up their technique as they go. With time, and sometimes with chargebacks coming in, you realize that your few analysts can’t review all transactions, so you turn to set up a few rules to make queue and transaction hold decisions. Since your analysts are not technology people you resort to hard coding some logic based on a product manager’s refinement of the analysts’ thoughts, again based on a few (or many) cases they’ve already seen. Not a long while passes, and you realize that the analysts are caught in a cat and mouse game where they try to create a rule to stop the latest attack that found its way to the chargeback report, and put a lot of strain on the engineers who maintain the rule-set. Even after coding some simple rule writing interface the situation isn’t better since the abundance of rules creates unpredictable results, especially if you allowed the rules to actually make automated decisions and place restrictions on transactions and accounts.

It’s at this point that you realize you need a statistician to run regressions, so you bring someone in. Hopefully, you have enough of a data set for them to create a decent regression model, and you can just get anyone off the proverbial street since regression is a very common tool. The statistician comes on board and creates a model that has industry standard false positives, let’s say 80%. Your review volume grows with transaction volume and you have to hire even more analysts and customer service folks to deal with complaints by legitimate customers – makes sense, since placing restrictions with 80% false positives will get you a lot of incoming calls. Then you discover that the regression model’s effectiveness degrades pretty quickly since they’re trying to predict what transaction will get a chargeback, but there are multiple reasons for getting a chargeback, making it harder to predict correctly. You then also discover that to create an updated regression model you need to wait for most of the chargebacks to come in so you’d have a good enough set of problematic transactions, meaning that you have at least 3 months’ lead time before a new process can be kicked off. That’s, of course, given that you have engineers on board to code the model.

Next thing you do is buy fraud prevention tools to add to your modeling power; you start creating black lists of IPs and Emails to mark problematic transactions. This improves things a bit but leads to additional false positives since people share resources. You consider buying a platform to manage rule and model deployment but decide that cost is prohibitive and generally, it looks like risk management is taking over your dev resources. So you decide to hire more analysts to do manual reviews, and a product manager to decide on the rule and model roadmap, and a risk operations manager for the growing group of analysts, and a head of risk. The rules and models you already have in place are blocking so many transactions you start to wonder if they’re not slowing down your growth instead of helping you protect your business. It looks like risk is managing you.

There’s something wrong with this model. Sure, some of what I’m describing makes sense for a company that’s just starting out, but getting caught in the factory approach to risk management is a huge burden in later stages, one that can be prevented by realizing that risk management is just one of a series of classification and inference questions a company needs to deal with, and that those require a different way of upfront investment in building a team.

I had two very interesting conversations about this very subject last week with two very inspiring people, but there’s one thing I remember a colleague telling me a few months back when they took a new job. The person insisted on being responsible not only for the new company’s risk management efforts, but also generally for their data and business intelligence. That was based on the same understanding: with the abundance of data created by organizations’ activity, all attempts to organize that data and make sense of it should be bound together. It doesn’t matter whether you’re qualifying leads, improving conversion or reducing fraud – you’re dealing with users and their actions, and how automated decisions impact them. It is the practice of making sense of data, and it transcends using data to control the experience of bad users. Once you realize that, you start demanding more of your analysts: that they be technical, know how to generalize on trends beyond targeted rules, become sources of truth. You understand that off-the-shelf regression cannot be just carried between domains without adjustment. You build a system that can correct itself. And with that, you create a team that can win risk, and do much more than that:  develop data sources, identify trends in huge data sets, and reach actionable insights that transform the way you work with your users, both fraudulent and legitimate.

What’s missing? I think there first needs to be a critical mass of people dealing with data in a way that sees beyond “intuition” but doesn’t get lost with over complicating inference using huge data sets. It takes time for these people to develop skill and want to continue solving difficult problems; my analysis group had less than 20 people in it and a good part of them have had enough of payments risk and classification for the rest of their career. I’m not even mentioning starting a company. But when you start a risk or data team, make sure you seed it well, or you’ll find that the bad start costs you a lot more money and effort than you have planned. 

Why hasn’t Venmo succeeded?

1 Reply

A question was asked on Quora regarding Venmo, the payments startup that was acquired lately by BrainTree.

I’m re-posting my answer here.

The details of the question are not completely correct. Venmo is actually a textbook example of why payments startups fail despite having a smart team.

For starters, Venmo never catered for a large enough crowd and never found product-market fit. Although it is a neatly designed mobile experience, p2p payments and purchase sharing were never big drivers for adoption, since they don’t solve a real problem (p2p solves a problem but that specific market share is tiny and consumers oppose paying fees). Other statups (blippy, swipely) demonstrated that consumers aren’t overly interested in sharing their purchases. So although some people liked Venmo, it was not enough to cross the boundary of hardcore early adopters. This is a common misconception with payment startups – they invent a new type of experience that some people like, get a group of hardcore fans, but never expand to mainstream. A new experience isn’t really reason to move in payments; just handing over your card is fine.

The second point is distribution and money. Venmo built a FB-based experience but didn’t figure out virality, contrary to what the OP states. The sign-up-and-add-credit-card setup that worked for PayPal worked because they had eBay as a distribution mechanism and still they had to pay $10-20 per user, raising a ton of money in the process (PayPal lost $209 Million before becoming profitable). Venmo didn’t have that distribution and therefore did not reach the right scale for the extra fees they charged over CC fees to cover their cost base. That was evident by their move a few months ago to charging fees for CC purchases but not for direct bank transfers.

Bottom line, Venmo has smart founders and a good team; they started a company and got acquired for an amount that I hope made investors even. That may not be a success, but I’m sure they know so much more now – and if I had to launch a payments product in the US I’d love to have some of them on my team.

Read Quote of Ohad Samet’s answer to Venmo: Why wasn’t Venmo more mainstream? on Quora

 

What’s Missing in Data Science Talks

1 Reply

On January 28th, 2008, the $169M sale of Israeli FraudSciences to eBay’s payments division PayPal was publicly announced. I was part of the 65 person crew and head of the analytics group at the time. FraudSciences became PayPal’s Israeli R&D center and is still a thriving team spanning more than 100 people and providing great value to the company. Our story has even been mentioned on StartUp Nation, in an inspired-by-a-true-story style dramatization of events.

The sale and its ramifications is not what I want to talk about, though; what I do want to talk about is the events that led to that sale, and more specifically the test that PayPal ran us through. You see, PayPal had to see whether our preposterous claims about how good our algorithms were held true, so they threw a good chunk of transactions at us to analyze and send back to them with our suggested decisions. Long story short, our results had an upside of up to 17% over PayPal’s own algorithms at the time, and the rest is history.

How did we do that, then? We must have had a ton of data. We must have used algorithm X or technique Y. We must have been masters of Hadoop. Wait – no. 2007. Nothing of the sort. Everything takes forever. To get to these results we didn’t even use the two famous patents FraudSciences viewed as huge assets since they required some sort of real time interaction with the buyer. What we did have were roughly 40,000 (indeed) well-tagged purchases, good segmentation, and great engineered features all geared at very well defined user behaviors. What we had, plain and simple, was strong domain expertise.

Domain expertise, or lack thereof, is exactly my issue with the talk about Data Science today. Here’s an example: I recently had a friend, a strong domain expert, rejected from a pretty nascent startup filled with very smart engineers since they didn’t really know where to place his non-developer profile in their team. Were they wrong to not hire him? Maybe, maybe not. I can’t judge. Were they wrong to make the decision based on coding skills? Most definitely. It’s a very common passion for data and ML geeks such as ourselves to embark on the (in my opinion) hubris-driven task of building an artificial intelligence that will solve all problems, the Generic SkyNet. We neglect to admit the need for specific knowledge. It is then when discussions of volume and structure of data sets replace keen understanding of what people are trying to achieve – when complex tools replace user research. Unsurprisingly, these attempts either fail or scale down to take domain by domain. They can still take over the world – just with a different strategy.

When I read people on Kaggle, in itself an amazing website and community, list the tools they threw at a dataset instead of how they led with a pure analysis of pattern and indicators, I cringe a little. This is a craft fueled by excess – in space, in memory, in computing power, even in data. While often times highly useful, almost as often does it  make us miss the heuristic just in front of our eyes. I think that analysis and Data Science need to incorporate this realization as well, to become a real expertise.

Fraud detection and prevention and Credit issuance, the stuff we deal with on a daily basis at Klarna, are areas where this is an obvious issue. High fragmentation in geographies, payment instruments and products creates smaller training and validation sets than you’d ideally want. The need to wait for default or a chargeback limits the time between iterations. The presence of bad signals is scarce compared to other types of classification. Operational issues and fraudsters’ strong incentives to hide (as well as abuse or “friendly” fraud) cause “dirty” performance flags. And still we have a shop that uses a number of instances per segment that Data Science teams would frown upon to make some accurate decisions. How is that? The same way FraudSciences gave PayPal’s algorithms a run for their money – we use domain expertise to distill features that capture interaction in a way that automated feature engineering methods will find hard to imitate. We use bottom up analysis of behavioral patterns. We add a sprinkle of behavioral economics (but building a purchase flow is a completely different story).

This aspect of what we do is available to any Data Scientist out there – I’ve written extensively about finding domain experts. They’re around you. Use them – and don’t get hooked on the big guns just because they’re there*.

*Well, only if you want to get better results quicker and are acting under market and product constraints. If you’re a contributor to an open source project – carry on with your great work!

In Defense of Knowledge Sharing in Fraud Prevention

4 Replies

Like any other field, there are conventions and (few) blog posts about fraud prevention specifically, and behavior management generally (be that abuse, spam or anything else). There is, though, an air of secrecy around these topics that always leaves the discussion at a very high level: no numbers around KPIs, no discussion of best practices and little sharing of information other than explicit blacklist data. This is a situation that has to change if risk management online is to become a real practice, and with the growing need for risk experts it is a must.

Secrecy in fraud prevention can be driven by multiple reasons, but mostly for competitive advantage – whether the company thinks that telling too much about its fraud prevention will tip off fraudsters and competitors, or risk staff feels that not sharing what they know gives them career advantage over others. Both considerations are true to some extent, but not to the extent to which they currently justify secrecy.

The truth about fraud prevention is simple: there is a finite set of generic tools and indicators that you can deploy. The key to differentiating in risk is not the run-of-the-mill 80% but rather in the 20%, and those are the ones that should be kept secret – interestingly enough, usually those are the ones hardest to steal and transfer anyway.

What’s in the 80% and what’s in the 20%? Let’s look at a few examples.

Tools? When I tell you that every risk team must utilize linking, velocity, rules, modeling and manual review tools (the “Basic toolbox”), that’s the 80%. Even if we discuss particulars of implementation such as the right type of edit distance for fuzzy matching an address, this is still far from the 20%. Are you really giving away competitive advantage? The type of customer and data your colleague is working with may not be the same. Shopping cart design may even impact the types of typos good customers do, which then impacts normalization and matching.

Procedures and Analytics? The way we develop and implement rules in Klarna may give us some competitive advantage, but that is not because of our procedures or the fact that we use control groups. It’s only in the mixture of hiring, training, day to day mix of detection/analysis/action that the competitive advantage is attained and maintained. Risk and fraud prevention are inherently operational practices – user behavior is constantly changing, if only for the changes in the product or macro-economic trends. In consulting assignments I sometimes found myself explaining essence and implementation to a very detailed level only to see it being missed or misinterpreted in execution.

These are two examples, but there are more. Of course some things must be confidential. Your training plan, your hiring methods, and the code for the variables used in your models are all good examples. Consider this, though: if your success depends on fraudsters not knowing that emails from yahoo.com are always suspected while emails from intel.com never are, you’re in trouble no matter WHAT is in your “secret sauce”.

Secrecy is important, but the current fear of collaboration is hurting more than helping. I’m not suggesting open sources, but I am suggesting that risk management should be more similar to standard software development. This will allow us to focus on the hard problems of implementation, operation and real innovation.

What Winning in Payments Should Mean, and How to Stay Away from the 0% Interchange Trap

4 Replies

In my previous post about this topic I touched upon why the current trends in the payments market are not sustainable, nor are they really exciting. We know that the payments market is highly fragmented, and new players are introduced at an even growing pace, making it even more fragmented than ever. Over the years, the setup for how payments are made both online and offline allowed for a very details set of roles: issuers, acquirers, gateways, POS companies and so on. There’s one level, though, at which there is very little fragmentation: the underlying one.

Card networks are huge behemoths controlling a large piece of payments in the world. Before becoming public these companies were spawned and controlled by banks; and though retail banking is somewhat divided between a few large players, it is still not as nearly so as the payments market is. What do these players have that others don’t that makes them so special? One thing: the financial relationship with the consumer. When you have that relationship, one of two things (or both) happens: consumers loan money from you; consumers give you their money in the form of deposits. It’s a very straightforward litmus test, you either have it or you don’t. That trust relationship is what counts, and everything else is just UX. Every cent you move pays interchange or another fee to a player that owns this relationship with merchants and consumers; all they have to do is sit back and enjoy the ride (and in the case of online payments, without any risk).

Asking who’s going to win payments, therefore, is the wrong question*. Visa and Mastercard are, and retail banks along with them. The rest of the companies are (arguably) making money off of peripherals, from foreign exchange fees to (again, arguably, I don’t believe it’s catching on) coupons and loyalty programs. If you want to win in payments, substantially, you need to displace that trust relationship; you need to circumvent banks and card networks. This is what’s interesting, and this is the Holy Grail.

The good news is that it’s possible, and since Visa and MC are public it is even more doable than it used to be due to the separation between their incentives and the banks’. The bad news is that this is the real difficult problem to solve. You’re basically setting off to build a new type of banking. In the next post we’ll look at a few examples of why just building another layer on top of existing partners doesn’t work, and then explore a few ways to get there.

* Unless you actually mean “who’s going to get access to consumers’ data in POS, while treating money movement as a loss leader?”. In that case, your question is valid.

Why we’re going to hit 0% interchange, and why it’s all a big spin

5 Replies

Lately I had the chance to talk with a really smart entrepreneur who’s into payments. I was baffled, frankly, about why someone would go into online card acceptance, a fragmented and crowded market. He offered a great piece of insight: payments are an obviously broken part of the shopping experience, an interesting and big market that’s not advertising or gaming which is usually where young entrepreneurs go. I respect that, and I also respect the fact that he is knowingly marching towards the unknown; hoping for his and his team’s success, I know they’re going to rightfully earn some gray hairs in the coming few years.

However that raises a question that I’ve dealt with in this blog in the past: what does it mean to succeed, to “win” in payments, and what it takes to get there. Offering a solution such as card acceptance for small merchants or starting with a niche like college payments or digital content can provide good initial traction for a budding payment service. The question is, however, whether you can expand on that early start and scale; at this point you have to deal with money movement being a commodity.

Small and medium merchants are growing more sophisticated with their webdev capabilities, and they will integrate you for a better price. It’s that simple: a better price or a slightly better integration or a slightly conversion-growing experience will get you a small crowd, enough for initial traction; some of them will just add you to the pile of services they are already using.

That’s exactly what most of the new startups in payments do, and that allows them to stick around while they fund their merchants’ business using VC money or burrow deeper into a high-margin market segment like digital. Unless you have very deep pockets or can raise money effectively (and there are few players that can), you’re in trouble: price hikes, alternative revenue streams or an obligatory move to ACH payments will ensue, probably resulting in an exodus to the next low-price service, and you will never reach scale (there are other, better options – I will discuss them in a following post). At some point, it’s convert users to low cost funding sources or get out of the game.

If you think I’m exaggerating, read this post and then these great answers on Quora; price wars are not sustainable, and the market can’t support so many payments companies trying to reach scale at the same time. Something has to give.

How do you differentiate? This is the time to ask yourself whether what you’re building is a feature or a product, and whether you’re really providing any added value, and lastly whether you have an exit plan at $50M, $200M or are really playing for the big league. Each should be planned for differently. Payments are not a pure consumer play; you rarely go viral. The really interesting problems in payments have very high barriers for entry; the type of problems currently being tackled, not so much:

  1. Have a beautiful dashboard for merchants to view their customers’ behaviors? So do others.
  2. Looking at easy integration and no need for a merchant account? Stripe and Braintree have that figured out, and they’ll have to start thinking about merchant retention and higher pricing very soon, if not already.
  3. Thinking of a coupons and loyalty plan? Good luck with adoption.
  4. Planning to use ACH or other bank payments? Gear up to challenge PayPal adoption in the US, and consider whether Dwolla’s approach to bank payments isn’t the main driver behind its slower growth compared to other services.
  5. Thinking of a new POS system? Even Square, doing so many things right, is going to be pushed on price and market share by companies replicating its model. It comes down to brand and convoluted termination clauses in long term agreements, like any other commodity business.

This is deep pockets land with some talent acquisitions around the edges, not a place for innovation.

Does this mean that there’s nothing to do in payments? No. There is a lot to do in payments, and I believe that this guy I mentioned and others will find success in their business, even if not displace PayPal and others. There is, though, a more interesting aspect of the market to attack. More about that in my next post.