What changes are we making?
We are updating how compliance scores are calculated to make them more accurate, fairer for suppliers, and more meaningful for clients. These changes will take place on Monday, 16th February.
These changes ensure suppliers are only scored on controls that are both relevant to them and explicitly required by a client's policies.
Below is a breakdown of what is changing, why it matters and how it affects scores.
1. Only controls with a policy rule will be scored
Controls will now only be scored if a client has added a policy requirement to that control.
If a control does not have a policy rule attached, it will be marked as Not scored, regardless of how the supplier answers it.
Example:
A supplier answers No to A.5: Does your organisation have a formal policy on the use of mobile devices?
The client has not added a policy rule to A.5
Previously:
This would have been marked as Compliant
Now:
This will be marked as Not scored
What was the rule before?
Every control was scored, whether or not a client had set a policy rule for it.
What does this change fix?
Compliance scores now reflect what a client has explicitly said they care about
Scores are no longer inflated or deflated by irrelevant controls
Suppliers are only given a score when policy requirements exist
2. Out-of-scope controls will no longer be scored
If a supplier answers a scoping question in a way that sets a domain as out of scope, none of the controls in that domain will be scored, even if policy rules exist.
Example
A cleaning supplier answers No to E.0: Does your organisation develop any software or apps?
The client has policy requirements on E.2 and E.5
Previously:
The supplier would receive a mix of compliant and non-compliant scores for this domain.
Now:
All controls in the domain will be marked as Not scored
What was the rule before?
All controls were scored even if they were out of scope.
Controls with policy requirements were marked Non-compliant
Controls without policy requirements were marked Compliant
What does this change fix?
Suppliers are only scored on activities they actually perform
Organisations are not marked up or down for controls that do not apply to them
A supplier that does not develop software will never be scored on software-related controls
3. Some parent and child controls will become mini scoping questions
Certain parent controls that are phrased like scoping questions will now behave like domain scoping questions.
If the parent control is answered in a way that makes the topic out of scope, none of the child controls will be scored, even if policies exist.
Example
A cleaning supplier answers No to K.2: Does your organisation use Machine Learning or Generative AI for internal use-cases?
The client has policy requirements on K.2 through K.12
Previously:
The supplier would be marked as 1 Compliant and 12 Non-compliant
Now:
All controls will be marked as Not scored
Which parent and child controls are affected?
Only parent and child relationships that begin with a scoping-style question.
What was the rule before?
All parent and child controls were scored, even if the parent answer effectively made the children not applicable.
What does this change fix?
Suppliers are not scored on controls that are irrelevant to their organisation
Incorrect scoring scenarios where a valid parent answer led to unfair child penalties
Reduced need for manual overrides on known problem areas
What are the benefits of these changes?
Scores that reflect what clients actually care about
Only controls with policy requirements are scored. This means overall compliance scores represent a client's real priorities, not every possible control in the framework.
More accurate scores with fewer overrides
By removing scoring from irrelevant or out-of-scope controls, suppliers are less likely to be incorrectly marked. This reduces the need for manual overrides and increases confidence in the scores.
A stronger foundation for future improvements
A clearer and more accurate rules system allows clients to confidently automate more of their review processes over time.
FAQs
Can I opt in or out of the new scoring methodology?
No. Once enabled, this will be the standard way compliance is scored for all assessments. We believe this change is essential to providing accurate and meaningful compliance scores.
Will I be able to see when scores change?
Yes. There will be clear messaging in the platform, including an activity log entry on each supplier showing when the scoring change occurred.
What happens to overrides on “Not scored” controls?
Clients can override Not scored controls to either Compliant or Non-compliant if required.
How can I avoid large score fluctuations?
We recommend:
Using comprehensive policies with exhaustive rules, similar to our template policies. Policies with only a small number of rules can cause large score swings.
Ensuring all suppliers have at least one policy attached. Suppliers with no policies and only overrides may now show extreme scores such as 0 percent or 100 percent.
Do incomplete answers get scored if a policy rule exists?
Yes. If a policy requirement is attached, incomplete answers will still be scored.
How does “Not scored” affect the overall compliance score?
Only controls with a positive or negative score are included in the overall compliance calculation. Not scored controls are excluded entirely.
Does this affect the policy score breakdown?
Yes. If a policy contains only one requirement, it will now be scored only against that requirement.
Is control weighting included in this update?
No. Control weighting is not part of this release, but it is something we are considering for the future.
💡 If there is anything we haven't covered, please feel free to contact us at support@riskledger.com or alternatively, select the Chat icon in the bottom right corner.