org.nuxeo.runtime.test.runner

Class RandomBug

java.lang.Object

org.nuxeo.runtime.test.runner.RandomBug

All Implemented Interfaces:

org.junit.rules.MethodRule

public class RandomBug
extends Object
implements org.junit.rules.MethodRule

Define execution rules for an annotated random bug.

Principle is to increase consistency on tests which have a random behavior. Such test is a headache because:

• some developers may ask to ignore a random test since it's not reliable and produces useless noise most of the time,
• however, the test may still be useful in continuous integration for checking the non-random part of code it covers,
• and, after all, there's a random bug which should be fixed!

Compared to the @Ignore JUnit annotation, the advantage is to provide different behaviors for different use cases. The wanted behavior depending on whereas:

• we are working on something else and don't want being bothered by an unreliable test,
• we are working on the covered code and want to be warned in case of regression,
• we are working on the random bug and want to reproduce it.

That means that a random bug cannot be removed, but must be ignorable or executable with attempts to reproduce or hide its random aspect, depending on its execution context.

For instance:

 
 import org.nuxeo.runtime.test.runner.FeaturesRunner;
 import org.nuxeo.runtime.test.RandomBugRule;

 @RunWith(FeaturesRunner.class)
 public class TestSample {
     public static final String NXP99999 = "Some comment or description";

     @Test
     @RandomBugRule.Repeat(issue = NXP99999, onFailure=5, onSuccess=50)
     public void testWhichFailsSometimes() throws Exception {
         assertTrue(java.lang.Math.random() > 0.2);
     }
 }
 
 

In the above example, the test fails sometimes.
With the @RandomBugRule.Repeat annotation, it will be repeated in case of failure up to 5 times until success. This is the default RandomBug.MODE.RELAX mode.
In order to reproduce the bug, use the RandomBug.MODE.STRICT mode. It will be repeated in case of success up to 50 times until failure.
In RandomBug.MODE.BYPASS mode, the test is ignored.

Since:

5.9.5

See Also:

RandomBug.MODE, MODE_PROPERTY

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RandomBug.MODE BYPASS: the test is ignored.
static interface	RandomBug.Repeat Repeat condition based on

Field Summary

Fields

Modifier and Type	Field and Description
static RandomBug.MODE	DEFAULT The default mode if MODE_PROPERTY is not set.
static String	MODE_PROPERTY

Constructor Summary

Constructors

Constructor and Description
RandomBug()

Method Summary

Methods

Modifier and Type	Method and Description
org.junit.runners.model.Statement	apply(org.junit.runners.model.Statement statement, org.junit.runners.model.FrameworkMethod method, Object target)
static RandomBug.MODE	getMode()

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MODE_PROPERTY

public static final String MODE_PROPERTY

See Also:

Constant Field Values

DEFAULT

public static final RandomBug.MODE DEFAULT

The default mode if MODE_PROPERTY is not set.

Constructor Detail

RandomBug

public RandomBug()

Method Detail

apply

public org.junit.runners.model.Statement apply(org.junit.runners.model.Statement statement,
                                      org.junit.runners.model.FrameworkMethod method,
                                      Object target)

Specified by:

apply in interface org.junit.rules.MethodRule

getMode

public static RandomBug.MODE getMode()