How do you test for non-deterministic?
A test is non-deterministic when it passes sometimes and fails sometimes, without any noticeable change in the code, tests, or environment. Such tests fail, then you re-run them and they pass. Test failures for such tests are seemingly random.
Which algorithms are non-deterministic?
One example of a non-deterministic algorithm is the execution of concurrent algorithms with race conditions, which can exhibit different outputs on different runs.
What is non-deterministic algorithm example?
One example of a non-deterministic algorithm is the execution of concurrent algorithms with race conditions, which can exhibit different outputs on different runs. Every outcome the non-deterministic algorithm produces is valid, regardless of the choices made by the algorithm during execution.
How to test randomness in a unit test?
A unit test should call a method and test the returned value (or object state) against an expected value. The problem with testing randomness is that there isn’t an expected value for most of the things you’d like to test. You can test with a given seed, but that only tests repeatability.
When do you need to unit test an algorithm?
In your example I think you are saying you want to unit test an algorithm that verifies a proposed solution. You’d want to cover the following cases: Sad path tests to ensure that the algorithm correctly handles non-candidates (e.g. a “solution” with 7 queens instead of 8, etc.)
Is there a way to test the randomness of a distribution?
The problem with testing randomness is that there isn’t an expected value for most of the things you’d like to test. You can test with a given seed, but that only tests repeatability. It doesn’t give you any way to measure how random the distribution is, or if it’s even random at all.
How to test the randomness of a rng?
Since the output of the randomization is known based on the seed (its inputs), then you can unit test as normal based on inputs vs expected outputs. If your RNG is not deterministic, then mock it with one that is deterministic (or simply not random). Test the randomization in isolation from the code that consumes it.