JCheck.jl: Randomized Property Testing Made Easy

JCheck is a native Julia implementation of a randomized property testing (RPT) framework. It aims at integrating as seamlessly as possible to the Test.jl package in order to enable developers to easily use RPT along with more "traditional" approaches. Although a fair number of generators are included, designing novel ones for custom data types is a straightforward process. Additional features such as shrinkage and specification of "special" non-random input are available.

Slides JuliaHub

Since Julia's main purpose is technical computing, we believe its users could benefit from an easy-to-use framework for property testing. A lot of Julia code is in fact implementation of various kinds of abstract objects for which at least some theoretical properties are known beforehand. While randomized property testing alone is not usually sufficient for serious software development, it might definitely be a great addition to a battery of tests. For that reason, we designed JCheck.jl so it integrates seamlessly with Test.jl.

To make JCheck agreeable to use, a lot of care has been put into the efficiency of the input generation process. Random inputs are reused to reduce the number of generated data to a minimum. "Built-in" generators which can be used as a building block for more complex ones have been designed to be as efficient as possible.

JCheck can be extended to support custom types in 2 ways. Type unions of types for which generators are implemented are supported automatically. More intricate types are supported through method dispatch. Note that it is trivial to define a generator for a type for which we can already generate random instances.

JCheck support so-called "special cases", i.e. non-random cases that are always checked.

In order to make the analysis of failing cases easier, JCheck support shrinking. When such a case is detected, it will try to make it as simple as possible. Whether shrunk or not, failing cases can be serialized to a file to make further investigation easier.