The Cost of Security Theater

I am an expert on security questionnaires. I have spent 1000s of hours on them and reviewed 10,000s of question/ answer pairs. Anyone who invests that kind of time can call themselves an expert, so I am calling myself an expert. LOL

And I hate them. Every time I have to help a customer review one, I find the process so tedious and boring. I once had a prospect tell me they wanted to hire an intern who would spend all day responding to security questionnaires. I bid them good luck. They never found anyone.

Anyway, I have also written “how to” articles about the best way to streamline response to security questionnaires. I have a lot of tips and tricks up my sleeve for security questionnaires, with or without software like ClearOPS, because I work on them so often. But then a customer asked me to help him in a way I had not thought about before. He is trying to justify a budget for security. So I thought, sure, I know something about that considering I have to justify the costs of ClearOPS all the time. So how much harder can it be to recommend a solid security budget that gives my customers something to use as an authoritative argument to management for budgeting purposes?

This blog post focuses on the cost of security questionnaires aka security theater. By the way, if you want to read the definitive guide on why they are security theater, please see Daniel Miessler's excellent write up.

Security Questionnaires = $180,000 per year [averaging 10 per month at $1500 each]: A few months ago, I found two job listings at different companies for someone who would spend their time exclusively responding to security questionnaires.  If you factor that the cost of one security questionnaire is between $250-$4500, then eventually a full-time hire makes economic sense. However, with that big of a delta, we need to do the math for you to estimate how much security questionnaires are costing your business. So let’s do the math.

While there are several standardized security questionnaires, most are customized meaning each  customer has their own form (which is annoying, and one of the reasons we built ClearOPS). It takes, on average, 4 minutes to respond per question to a custom questionnaire. I know this because I actually timed myself. In this timed experiment, I had the answers ready to find, copy and paste from one document to the next. So, my process was as efficient as humanly possible. It took me an average of 2 minutes per question. So, if someone has to draft answers for the first time or look up the information to verify, it doubles the average time per question, at the minimum. Even knowing some of the information off the top of your head doesn’t affect this time because we are taking a mathematical average. In my experience, spending 10–15 minutes on one question can and does happen, frequently.

Most people would nod their head at this. A 4 minute average sounds reasonable, right? Okay, so if the average questionnaire is 350 questions (which is the average these days), then that is 23 hours worth of work.

Do you think that’s insane? Well it is. Usually, the response I receive to this math is that there is “no way I spent that kind of time on a security questionnaire.” But you are. The reason most people don’t think they do spend this kind of time on them is because nobody can answer a security questionnaire in one sitting, so no one is actually tracking it. I have had people tell me their turnaround time is 24 hours. That’s B.S. Let’s ignore the fact that spending such little time on answering could result in incomplete or inaccurate answers or, worse, liability; it isn’t physically or emotionally possible to turn it around so quickly. Completing security questionnaires is an incredibly painful process (and now you have another reason why we built ClearOPS). Even if it is 50 questions long, you will most likely take a break and whenever you take a break, it takes time to get back into it. So a tedious task that no one wants to do means that, on average, it takes three months to complete an enterprise security questionnaire.

That delayed sale is costing you, my dear reader. If you are selling a $52k per year product, then every week of delay costs $1000 which means that month long delay is costing you $4000.

So, let’s assume the employee who fills out the questionnaire is making a salary of $45,000 per year, so roughly $21.50 an hour. That means every questionnaire is costing your business $494.50 + $4000= $4494.50.

There are flaws in my calculation. The main flaw is that there are other costs that are more deeply hidden. The first deeply hidden cost is the cost of other stakeholders who should be involved in responding, such as lawyers. The second deeply hidden cost is the cost of liability. Vendors can be sued for inaccurate answers, as seen in a current lawsuit between Delta Airlines and 24/7. For a vendor to defend a lawsuit, they have to spend a lot of money on lawyers and lawyers are expensive. These hidden costs are hard to calculate, so I am leaving them out of the math, but they do demonstrate how conservative my calculations are.

When I said flaws, you may have thought I was going the other way to admit that my estimation of the cost of a security questionnaire was too high, but I didn’t. In fact, there are even more hidden costs that no one can calculate, which I will describe next.

Imagine Inna the Security Engineer. After a grueling day at work putting out fires and attending meetings, she finally has a chunk of time to sit down and go through the questionnaire. Imagine how dismayed Inna is when she sees that there are 350 questions. Inna is tired and now frustrated that this task is on her desk. To get through it, she answers in paraphrases and acronyms and annoyance. She sometimes just links to a url as her answer. She writes N/A wherever she can. After all, a pillow is waiting for her and she knows that no one is checking her answers anyway.

We all have mind numbing tasks but many consider security questionnaires to be security theater and with good reason. 80% of security questionnaires go unreviewed by the buyer. So anyone whose job is to respond to security questionnaires is going to be unhappy because the work feels meaningless. An unhappy employee, especially a hard to find security engineer, is an incredibly high cost for any organization because they will leave and the vendor will have to replace them. This replacement cost is the one we focus on at ClearOPS. Don’t lose all that knowledge with the disgruntled employee. Instead, keep the employee happy and keep the knowledge safe in one place.

Bottom line: If you receive regular security questionnaires, then they are costing your business a lot of time and money. If you receive more than 10 per month, then you will actually save money if you hire someone to dedicate their time to responding. If you don’t see that type of volume yet, then consider a vCISO. At least they have the expertise and incentive to respond to them.

One final note, security questionnaires are a sales cost, not an IT cost, and they should be added to your customer acquisition cost. If it weren’t for your customer demanding them (which they have to now because of new regulations), no IT department would voluntarily be engaged in this type of security theater. Security questionnaires don’t help IT do their jobs (at least, not unless you are using ClearOPS). They do help close sales and if it is something that helps sales, then management will budget for it.

‍