org.nuxeo.runtime.test.runner

Class RandomBug

java.lang.Object

org.nuxeo.runtime.test.runner.RandomBug

public class RandomBug
extends Object

Define execution rules for an annotated random bug.

Principle is to increase consistency on tests which have a random behavior. Such test is a headache because:

• some developers may ask to ignore a random test since it's not reliable and produces useless noise most of the time,
• however, the test may still be useful in continuous integration for checking the non-random part of code it covers,
• and, after all, there's a random bug which should be fixed!

Compared to the @Ignore JUnit annotation, the advantage is to provide different behaviors for different use cases. The wanted behavior depending on whereas:

• we are working on something else and don't want being bothered by an unreliable test,
• we are working on the covered code and want to be warned in case of regression,
• we are working on the random bug and want to reproduce it.

That means that a random bug cannot be ignored. But must attempt to reproduce or hide its random aspect, depending on its execution context. For instance:

 
 import org.nuxeo.runtime.test.runner.FeaturesRunner;
 import org.nuxeo.runtime.test.RandomBugRule;

  @RunWith(FeaturesRunner.class)
 public class TestSample {
     public static final String NXP99999 = "Some comment or description";

      @Test
      @RandomBug.Repeat(issue = NXP99999, onFailure=5, onSuccess=50)
     public void testWhichFailsSometimes() throws Exception {
         assertTrue(java.lang.Math.random() > 0.2);
     }
 }
 

In the above example, the test fails sometimes. With the RandomBug.Repeat annotation, it will be repeated in case of failure up to 5 times until success. This is the default RandomBug.Mode.RELAX mode. In order to reproduce the bug, use the RandomBug.Mode.STRICT mode. It will be repeated in case of success up to 50 times until failure. In RandomBug.Mode.BYPASS mode, the test is ignored.

You may also repeat a whole suite in the same way by annotating the class itself. You may want also want to skip some tests, then you can annotate them and set RandomBug.Repeat.bypass() to true.

Since:

5.9.5

See Also:

RandomBug.Mode

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	RandomBug.Bypass
static class	RandomBug.Feature
static class	RandomBug.Mode BYPASS: the test is ignored.
static interface	RandomBug.Repeat Repeat condition based on
protected class	RandomBug.RepeatOnFailure
protected class	RandomBug.RepeatOnSuccess
class	RandomBug.RepeatRule
protected class	RandomBug.RepeatStatement

Field Summary

Fields

Modifier and Type	Field and Description
RandomBug.Mode	DEFAULT The default mode if MODE_PROPERTY is not set.
static String	MODE_PROPERTY
protected static RandomBug	self

Constructor Summary

Constructors

Constructor and Description
RandomBug()

Method Summary

Modifier and Type	Method and Description
protected RandomBug.Mode	fetchMode()
protected RandomBug.RepeatRule	onMethod()
protected RandomBug.RepeatStatement	onRepeat(RandomBug.Repeat someParams, org.junit.runner.notification.RunNotifier aNotifier, org.junit.runners.model.Statement aStatement, org.junit.runner.Description description)
protected RandomBug.RepeatRule	onTest()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

self

protected static final RandomBug self

MODE_PROPERTY

public static final String MODE_PROPERTY

See Also:

Constant Field Values

DEFAULT

public final RandomBug.Mode DEFAULT

The default mode if MODE_PROPERTY is not set.

Constructor Detail

RandomBug

public RandomBug()

Method Detail

onTest

protected RandomBug.RepeatRule onTest()

onMethod

protected RandomBug.RepeatRule onMethod()

fetchMode

protected RandomBug.Mode fetchMode()

onRepeat

protected RandomBug.RepeatStatement onRepeat(RandomBug.Repeat someParams,
                                             org.junit.runner.notification.RunNotifier aNotifier,
                                             org.junit.runners.model.Statement aStatement,
                                             org.junit.runner.Description description)