org.nuxeo.runtime.test.runner

Class RandomBug

java.lang.Object

org.nuxeo.runtime.test.runner.RandomBug

public class RandomBug
extends Object

Define execution rules for an annotated random bug.

Principle is to increase consistency on tests which have a random behavior. Such test is a headache because:

• some developers may ask to ignore a random test since it's not reliable and produces useless noise most of the time,
• however, the test may still be useful in continuous integration for checking the non-random part of code it covers,
• and, after all, there's a random bug which should be fixed!

Compared to the @Ignore JUnit annotation, the advantage is to provide different behaviors for different use cases. The wanted behavior depending on whereas:

• we are working on something else and don't want being bothered by an unreliable test,
• we are working on the covered code and want to be warned in case of regression,
• we are working on the random bug and want to reproduce it.

That means that a random bug cannot be ignored. But must attempt to reproduce or hide its random aspect, depending on its execution context. For instance:

 
 import org.nuxeo.runtime.test.runner.FeaturesRunner;
 import org.nuxeo.runtime.test.RandomBugRule;

 @RunWith(FeaturesRunner.class)
 public class TestSample {
     public static final String NXP99999 = "Some comment or description";

     @Test
     @RandomBugRule.Repeat(issue = NXP99999, onFailure=5, onSuccess=50)
     public void testWhichFailsSometimes() throws Exception {
         assertTrue(java.lang.Math.random() > 0.2);
     }
 }
 

In the above example, the test fails sometimes. With the RandomBug.Repeat annotation, it will be repeated in case of failure up to 5 times until success. This is the default RandomBug.Mode.RELAX mode. In order to reproduce the bug, use the RandomBug.Mode.STRICT mode. It will be repeated in case of success up to 50 times until failure. In RandomBug.Mode.BYPASS mode, the test is ignored.

You may also repeat a whole suite in the same way by annotating the class itself. You may want also want to skip some tests, then you should then annotate the tests method with the NoRepeat annotation.

Since:

5.9.5

See Also:

RandomBug.Mode

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RandomBug.Feature
static class	RandomBug.Mode BYPASS: the test is ignored.
static interface	RandomBug.Repeat Repeat condition based on
class	RandomBug.RepeatRule

Field Summary

Fields

Modifier and Type	Field and Description
RandomBug.Mode	DEFAULT The default mode if MODE_PROPERTY is not set.
static String	MODE_PROPERTY

Constructor Summary

Constructors

Constructor and Description
RandomBug()

Method Summary

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MODE_PROPERTY

public static final String MODE_PROPERTY

See Also:

Constant Field Values

DEFAULT

public final RandomBug.Mode DEFAULT

The default mode if MODE_PROPERTY is not set.

Constructor Detail

RandomBug

public RandomBug()