Your best risk people are chasing spreadsheets

    By QuartzX Team••8 min read
    RCSAOperational Risk
    A QuartzIQ risk self-assessment summary for credential-compromise risks, with probability, severity, inherent, mitigation and residual ratings compared across two periods.

    Internal control, audit and operational risk are staffed by people who are expensive, hard to hire and harder to replace. They are hired for judgement: to look at what came back from the business, notice the answer that does not fit, and go and ask about it.

    Then look at what a risk and control self-assessment cycle actually consists of.

    What the cycle is made of Coordination Issuing and re-issuing the formsChasing the departments that have not repliedReconciling scales that were filled in differentlyMerging and de-duplicating the returnsRebuilding the pack when three arrive late Judgement Analysing what came back
    Five activities that any competent administrator could own, and one that is the reason you hired a risk professional.

    The imbalance is the problem, and it is not a motivation problem. A skilled risk analyst spends the cycle doing logistics because the logistics genuinely have to happen and nobody else is doing them. The analysis gets whatever is left, which in a bad quarter is a week before the committee.

    And here is the part that makes it worse. The ratings that expensive process delivers were largely made from memory — we ask people to recall how well a control they do not personally operate has been working across a period they have not been tracking. So the scarce judgement time, when it finally arrives, is spent interpreting numbers that were guesses.

    Two halves, and fixing only one is worse than useless. Automate the collection and you get to the same unreliable heat map faster.

    The assessor is not short of information. They are short of it on screen.

    Here is the strange part. By the time an assessment lands, the organisation already knows most of what the assessor is being asked to estimate. The indicators have been breaching or not breaching. The control tests have passed or failed. The operational losses have been booked. The department has reorganised, or replaced a system, or lost the person who knew how the process worked.

    None of it is usually in front of them when they choose a rating.

    The same question, asked two ways Indicator breaches this periodControl test failuresOperational losses bookedOrganisational and process changesExternal events in the sectorWhat you rated it last time The assessmentHow likely, how severe,how well mitigated? A rating youcan defendin the room
    Nothing here is new information. It is information the organisation already had, put in front of the person being asked to judge.

    The last row is the one people underestimate. Showing an assessor what they said last time changes the question from "rate this risk" to "has this got better or worse, and why do you think so". That is a far easier question to answer honestly, and a far harder one to answer with a shrug.

    Scenarios, rather than a single ranked list

    Once assessments are grounded, the top-risk list stops being a single output and becomes something you can interrogate. What does the ranking look like if we weight severity over likelihood? If we look only at risks where mitigation was rated "needs improvement"? If we take the entities separately rather than consolidated?

    Those are different lists, and the disagreements between them are the useful part — a risk that is top-five under three different weightings is a different proposition from one that is top-five under exactly one.

    The activity nobody assesses: the one you delegated

    Every organisation has processes a business unit owns but does not run — technology delivered by a central IT function, screening delivered by a shared operations team, a process outsourced to a provider. Ask who assesses that risk and you get one of two unsatisfying answers: the business unit, who cannot see how the work is done, or the central function, who cannot see what it matters to.

    Two views of the same delegated process The business unitowns the risk,cannot see the work The central functionruns the process,cannot see the impact The delegated activity assessed fromboth directions
    Assessed from one side only, a delegated process is rated by someone who can see half of it.

    Assessing from both ends is not duplication. The two assessments answer different questions — the delegate rates how well the process is operated, the delegating unit rates what it costs them when it is not — and where they disagree is precisely where a conversation was overdue.

    What this does not fix

    • Evidence does not settle a judgement. Two reasonable people looking at the same indicator breaches will still rate severity differently. The gain is that they are now disagreeing about the same facts, which is a productive argument rather than an unfalsifiable one.
    • Garbage indicators produce confident garbage. Putting a KRI in front of an assessor lends it authority. If the threshold was set arbitrarily and has never been revisited, you have made a weak number more persuasive, not more correct.
    • It does not shorten the first cycle. Connecting losses, tests, indicators and changes to the right risks is real work, and most of it happens before the first grounded assessment goes out.

    The point of an RCSA is not the heat map. It is the conversation the heat map is supposed to start — and that conversation only goes anywhere if the people in it believe the numbers.

    Security and compliance

    • SOC 2 Type IIAudited by Sensiba LLP.
    • ISO/IEC 27001Certified by Sensiba LLP, an ANAB-accredited certification body.
    • GDPRCompliant with the EU General Data Protection Regulation.
    • PIPEDAReady for Canada's Personal Information Protection and Electronic Documents Act.

    Reports available upon request