A test needs a rule to decide whether the test was successful or a failure. For example we expect the first test to be run as a reference SSO flow with unmodified SAML messages. This test must result in a successful sign on. Other tests – for example with a wrong signature – should result in a login failure. In that case a successful login to the service provider would actually mean a failed test.
A rule is returned from the SSOCheck Execution API for every test case. The rule defines how the Service Provider should behave when the test is run.
One Example: If the test is a signature exclusion attack, a SAML Assertion is sent to the Service Provider which should not lead to a successful single sign on. In that case the test will be passed if the sign on process fails. The API will return a rule value of 1 (= should fail).
In the table below the rule values are listed:
|Rule value||Test outcome on SSO failure||Description|
|0||FAIL||Login should succeed|
|1||OK||Login should fail|
|2||INVALID||Only partial step (e.g. first request of a replay test)|
|10||WARN||Login should succeed – but failure is only a warning|
|11||WARN||Login should fail – but success is only a warning|
Please note: Rules are derived from specification or based on the attack type and may be subject to interpretation.