Thursday, December 28, 2017

The optimal way of rewarding duplicates (and why it sucks anyway)

Disclaimer: As it's the case for all blog posts, this is a personal blog post, and I'm not talking on behalf of my employer! This came up as a result of a discussion on Twitter, thanks to all that participated!

Sometimes, when looking for vulnerabilities, two or more security researchers will find the same bug. This is what we usually call "duplicates" from the vendor world, or sometimes "bug collisions", in a slightly different context.

In the context of vulnerability reward programs, most vendors won't issue a reward for the same vulnerability more than once. There are some exceptions on some programs (Google, GitHub, Tesla, among others), such as for cases when the reporter provides new information, or prompts action, but generally the rule is that if you are first, you get money, if you are second you get nothing. There are no silver medals.

This sucks for bug hunters, and makes them unhappy. The question is:

Can we create a fair and abuse-resistant payment model for duplicates?

I think the answer is yes, but at the cost of higher complexity, and significantly changing reward amounts. At the end, I'm not sure if it's worth it, as it incurs costs to everyone involved, and no significant monetary value in return. Let me explain.

First of all, paying full amount for every bug is easy to dismiss. People will abuse it, "hackers gonna hack". While I'm sure some wouldn't, many would, and I think we need to prevent abuse somehow. The risk here is not just financial, but it's also not fair for those that play nice, and security response teams are not meant to be detectives. So we need to prevent abuse.

The second most obvious way to tackle this problem is by splitting a reward. Let's say you are the first to find the bug, and you get a $1,000.00 reward. So far, so good.

Now, your neighbor reports the same bug a day after. What we could do, is to split the reward half way. But this creates a problem, this means that you can't be paid until the bug is fixed (which might take a long time!). In order to fix that, you could then just double the reward amount and pay twice, once when the bug is found, and once when the bug is fixed.

So, you get $1,000.00 as a reward first, and then you could potentially get another $1,000.00 if nobody else reports the same bug. When your neighbor finds and reports the same bug, you split the extra $1,000 half way, and you get $1,500 and your neighbor $500. This seems OK (twice as much expensive for the vendor, but easy to understand).

The problem is then, what happens when a third person finds the bug? Do you split three ways? First finder gets an $1,333, second $333, and third the last $333? Unfortunately this doesn't work and is trivial to abuse. If you want to get more money, you just report the bug 10 times (or tell 9 of your friends to report it for you), and you will get $900, and the original finder only $100 more.

This is easy to fix at the expense of reducing how much money everybody gets by splitting the individual payout in half every time a new duplicate is reported. For example:

  • Only one finder, $1000
  • Two finders, $500 each
  • Three finders, $250 each
  • ...

  • This removes the financial incentive to share the bug, as there's no more money to get by reporting the bug many times. If any, by doing that, you are losing money (and pretty much making everyone else lose money).

    The problem with this, is that the reward amounts decrease very quickly. Essentially by the 5th finder we are down to $62 (we started at $1,000!). This is particularly problematic because for programs with large communities of bug hunters, finding the same bug multiple times is very common. Getting a micro reward isn't really solving the problem.

    One way to psychologically change the way we treat this could be by making a lottery. Here's what I mean:

  • Only one finder, 100% chance of getting $1,000
  • Two finders, 50% chance of getting $1,000
  • Three finders, 33% chance of getting $750
  • Four finders, 25% chance of getting $500
  • Five finders, 20% chance of getting $313
  • Six finders, 16% chance of getting $188
  • Seven finders, 14% chance of getting $110

  • This helps at least to keep a reasonable reward even when there are more finders, as you can see below:

    However, practically this doesn't change much, 20% chance of getting $313 is technically the same as getting $62, and makes the rewards a game of chance (which is somewhat relevant for lawyers).

    So where does that leave us?

    I think the answer is that it's not worth it to reward all duplicates, because preventing the incentive for abuse makes things complicated, and sinks the reward amounts so quickly that it ends up not really being worth the effort.

    In conclusion, I think the most effective and friendly way to balance these (and other) risks for bug hunters is to just give the best bug hunters money before they start their research, to incentivize them to look for bugs and to compensate for finding non-rewardable vulnerabilities (such as duplicates). This is done at Google, and bug hunters love it :). If you have any other ideas, please let me know on the comments or on twitter!