0wnage and the null hypothesis

H0: Our systems are not hacked.

That is what management wishes to hear all the time and expects to hear it with absolute certainty.

– But …

There are no buts in such matters for management, right? Oh but there are…

H0 True H0 False
Reject H0 Type I error Correct
Do not Reject H0 Correct Type II error

In reality there is no way to know whether the systems we maintain are hacked or not. We can only know with absolute certainty that they are owned and this only when the fact is detected. To help management understand this, use a “simpler” example:

H0: This message is not spam

Work with the not-spam example and the table above. It seems fairly straight forward that if your anti-spam measures are relaxed you receive a lot of undetected spam (Type II error) and if you tighten the controls you risk having legitimate messages characterized as spam (Type I error).

In a similar fashion you can detect that your systems are hacked and therefore you can reject H0. You can have your Intrusion detection systems, monitoring systems, processes or other controls “cry wolf” (a Type I error) or they may stay silent while in fact infiltration has happened (a Type II error). A Type II error means that an opportunity to detect a breach was lost.

So you see management, we cannot under absolute certainty assure you that we are and will remain unbreakable till the end of time. After all, if you really think about it hard, time is on the side of the blackhats. We can only provide you with data that we are doing our best with the tools you are providing.

One thought on “0wnage and the null hypothesis

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s