Digging Deeper into Property-based Testing

Intro

Today, I attempted to dig deeper into Property-based Testing. Specifically, I wanted to understand which technique for filtering values of a property-based test was most advantageous. Specifically, I wanted to understand the difference between a precondition expression and a custom generator in regards to their value.

The Difference between Them

Mark Seemann provided an answer here. In summary, he stated that when using a precondition expression for a property-based test, the tester risks reducing the actual number of tests (i.e. coverage) being executed. For example, having a precondition that eliminates negative numbers will only generate 50 test scenarios instead of 100 (the default). Instead, Mark stated that leveraging a custom generator will preserve the number of scenarios to be tested.

I then attempted to integrate a custom generator into a property-based test for a FizzBuzz implementation that I saw. Unfortunately, the test that I wrote always passed and I didn’t understand why.

The code looked like this:

[<Fact>]
let ``FizzBuzz.transform returns FizzBuzz`` () =
    let fiveAndThrees = Arb.generate<int> |> Gen.map ((*) (3 * 5))
                                          |> Arb.fromGen

    Prop.forAll fiveAndThrees <| fun number ->

        let actual = transform number

        let expected = "jbdhjsdhjdsjhsdglsdjlljh"
        expected = actual</pre>

I then received some help that resolved my issue. It turns out that I didn’t assign the prop.forAll expression to a property value. Thus I could then do a check that the property held up based on the range of values.

The working test looks like this:

[<Fact>]
let ``FizzBuzz.transform returns FizzBuzz`` () =
    let fiveAndThrees = Arb.generate<int> |> Gen.map ((*) (3 * 5))
                                          |> Arb.fromGen

    let property = Prop.forAll fiveAndThrees <| fun number ->

        let actual = transform number

        let expected = "FizzBuzz"
        expected = actual

    Check.QuickThrowOnFailure property

Conclusion

In conclusion, I attempted to dig deeper into Property-based Testing. Specifically, I wanted to understand which technique for filtering values of a property-based test was most advantageous. Specifically, I wanted to understand the difference between a precondition expression and a custom generator in regards to their value.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: