Tuesday, September 22, 2015

Not about the money

This applies to all posts in this blog, but just as a reminder. What I say here doesn't necessarily have to reflect the opinions of my employer. This is just my personal view on the subject.

I want to briefly discuss bug bounties, security rewards, and security research in general. Mostly because in the last five years a lot has changed, there are many new players. Some giving rewards, and some receiving them.

Background

As a brief introduction, I'll preface this by explaining why I (personally) used to (and continue to) report bugs back when there was no money involved, and then try to go from there to where we are today.

Before there were bug bounties, I (and many others) used to report bugs for free to companies, and we usually got "credit" as the highest reward, but I actually didn't do it for the credit (it was a nice prize, but that's it).

The reason I did it was because I am a user and I (selfishly) tried to make the products I used, better and safer for me. Another reason was that the challenge of finding bugs in "big" companies was like solving an unsolvable puzzle. Like the last level of a wargame that the developers thought no one could solve.

Rewards

With that said, I was super excited when companies started paying for bugs (mostly, because I had no money), but also because it felt "right". For a couple reasons but mostly because money is free for companies and it is a good way to materialize appreciation.

Rewards are a great way to say "thank you for your time and effort". It's like a gift, that just happens to come in US Dollars. Before money, T-Shirts were the equivalent, and sometimes even handwritten notes that said thank you.

It goes without saying that respect was also a big factor for me. Speedy responses, a human touch, and honest feedback always mattered a lot to me, and gave me confidence back when I was starting. Specially when I learnt something new.

Appreciation

It is my view, that we shouldn't call them "Bug Bounty Programs", I would like them to be called "Bug Hunter Appreciation Programs". I don't like the term "Bug Bounty", because bounty sounds a lot like it's money up for grabs, when the attitude is that of a gift, or a "thank you, you are awesome".

In other words, rewards are gifts, not transactions.

Rewards, in my view, are supposed to show appreciation, and they are not meant to be a price tag. There are many ways to show appreciation, and it can go from the simple "thank you!" in an email, or  in public all the way to a job offer or a contract. In fact, I got my first job this way.

For companies, I actually think that the main value reward programs provide isn't the "potential cyber-inteligence capabilities of continuous pentesting" or anything ridiculous like that, but something a lot more simple. To build and maintain a community around you, and you do that by building relationships.

Users

Think about it this way. These bug hunters are also users of the products they found bugs on, and then these users go and make these products better for everyone else. And in return, the companies "thank" these users for their help for making it better not just for them, but for everyone.

These users also, eventually, become more familiar with these products than the developers themselves, and that is without even having access to the source code. These users will know about all the quirks of the product, and will know how to use them to their advantage. These are not power users, they are a super-powered version of a user.

And reward programs were built for these users, for those that did it back when money wasn't involved. They are there as a sign of appreciation and a way to recognize the value of the research.

Recognition

The traditional way of saying "thanks" for vulnerability research has been recognition. For recognition to be valuable it doesn't necessarily have to come from the affected product or company, but rather it could come from the press, or from others in the community. Having your bug be highlighted as a Pwnie is nice, for example, even more so than a security bulletin by the affected company.

More recently, it's more common to have money be used to supplement recognition (and some companies apparently also use it to replace recognition, requiring the researcher not to talk about the bug, or talk bad about the affected company). It's quite interesting this isn't usually highlighted, but I think it's quite important. For many of us, recognition is more important than money, if any, because we depend so much on working and learning from others, that silence diminishes the value of our work.

Money

One question that I don't see asked often enough is: How to decide reward amounts?

First of all, let's consider they are gifts, not price tags. If you could come up with a gift that feels adequate as a gift for a user that is going out of they way to make (your) product better for everyone, what would it be?

It might very well be that it isn't money! Maybe it is free licenses for your product, or invitations to parties or conferences. Maybe it's just an invite to an unreleased product. Doing this at scale, though, might prove difficult.. so if you have to use money, maybe what is worth to go out one night for dinner and movies, or so. You can take that and then go from there to calculate higher rewards.

This is one of the reasons I don't really like "micro rewards" for $10 USD or so. Because while they are well-intentioned, they also convey the wrong message. I do realize they are valuable in bulk (products that give out these micro rewards usually have a lot of low hanging fruit), but rewarding individual bugs at low rewards also gives the feeling you are just being used for crowd sourcing, or as a "Bulk Bounty".

In any case, I appreciate this is mostly the wrapping of the message being delivered, but I think the message matters when you are dealing with money. The difference between you being "the user" and you being "the product" changes the perception and relationship between companies and users.

Trust

Two of the most important aspects of any relationship, are trust and respect. Coming back to the point earlier about money, I really like to see how "no reward" responses are written in different companies, because they show how much of a feeling of a transaction, and how much of a feeling of appreciation there is.

Maybe somewhat counter-intuitive for some is that trust goes both ways. Trusting bug hunters makes response teams more trustworthy, and vice-versa. Accepting the possibility that you are wrong and made a mistake reporting a bug (even if the bug seems to be valid, maybe you are wrong!) and the other way around, handling a bug (even if the bug seems to be invalid, maybe it is valid!). This is particularly important when communicating disagreements.

In any case, whatever the decision was, just saying "thanks" goes a long way. Even the slightest hint of disrespect can ruin an otherwise strong relationship, specially in a medium as cold as email. What is even better is to be transparent about the rationale for the "no reward" response. This is difficult, logistically, as "no reward" decisions are the bulk of the responses sent by any program, but doing a good job here goes a long way to foster trust.

Help

At the end of the day, a vulnerability report in a product is simply an attempt in helping the product. Such help sometimes is not welcome by some ("best bug is unknown bug") and some times it's not helpful at all (invalid reports, for example), but in any case, it's help. Accepting help when you need it, saying thanks when you don't, and then helping back when you can is essentially team work.

Another aspect is that sometimes you can reciprocate a vulnerability report with help. If a bug hunter seems lost or started off the wrong way, a little nudging and education can go a very long way. In fact, this is one of the most amazing realizations I've had in these past 4 years. The most insistent researchers that send the most reports, are also the most enthusiastic and eager to learn.

Investing some time helping via coaching and education rather than just depending on automated advice or scoring and rankings helps to create and grow a community. These bug hunters want to help, why don't you help them? It needs time and patience, but it will be worth it. Some small amount of personal feedback is all that is needed. You can't automate relationships.

Challenges

Finally, something amazing happened. Some Bug Hunters started getting a reasonable amount of money out of this! While these programs were originally created to be more like winning the lottery, some very talented individuals actually became really good at this.

The consequences are actually quite interesting. If you can have a sustainable life off white hat vulnerability research, you also become more dependent on these relationships than ever, and trust becomes a critical aspect for sustaining this relationship in the long term. Transparency and clarity on decisions and consistency suddenly become not just best practices but guiding principles.

Another interesting challenge is that of long term planning. It might make you more efficient at finding bugs to do so every day, but companies really want to eventually make finding these bugs harder and harder. This also creates some uncertainty about your future, and how to sustain this for a long time.

Then there are other challenges. These programs are quite unique and explaining them can be challenging for their legal and fiscal implications. I still remember when I got my first reward from Mozilla it was a bit awkward explaining to my bank why I had money being sent from abroad. I can just imagine having to do this every couple weeks, would be crazy.

Conclusions

It's not about the money. It is about working together and keeping us all safe.
  • If you feel your reward wasn't as high as you expected, maybe your bug's impact wasn't understood correctly, and that also might mean it was fixed incorrectly. Make sure there are no misunderstandings.
  • If there are no misunderstandings, then take the reward as what it is - a gift. It's not a bill for your time, it's not a price for your silence, it's not a bid for your bug's market price. It's just a thank you.
  • If you don't get credit for the bug you found, but you think it is a cool bug, then get credit for yourself! Blog about it, and share it with your peers. Make sure the bug is fixed (or at least give a heads up before making it public, if it's not gonna be fixed quickly).
Bug hunters are also users. Delight them with excellent customer service. Money is just a way to say thank you, it isn't an excuse to have them do work.
  • Appreciation, Recognition and Transparency (ART) are the pillars of a well ran security response program.
  • If you find yourself buried in low quality reports, invest some time to help them. Ignoring the problem won't make it go away.
  • Invest as much time as possible knowing about your community. Meet them face to face, invite them for dinner! Bug hunters are and will always be your most valuable users (including those that just started learning).


1 comment:

  1. I totally agree! It's not about money - it's about helping a company put out better products for all of us. I enjoy a puzzle, a challenge - anything that gets my brain working. I also enjoy a little recognition for my efforts from time to time - hunting for bugs is just something I'm good at.

    Joey @ Amerika Link

    ReplyDelete