SPA Conference session: Property-Based Test-Driven Development (As If You Meant It) | |||
| One-line description: | How does property-based testing change the TDD process compared to example-based testing? | ||
| Session format: | Workshop [read about the different session types] | ||
| Abstract: | When TDD was introduced in the late 1990s, it was described in terms of example-based testing tools, such as the xUnit family of frameworks. Since then, property-based testing tools have become widely available. Originating with the QuickCheck library for Haskell, property-based testing tools have now been ported to all major languages (and many minor languages). The TDD community has not embraced property-based testing. TDD practitioners "triangulate" a property of the system by testing it against multiple examples, selected manually. Because such tests do not explicitly define the property being tested, there is little scope for automatically selecting test data. It is easy for the programmer to write tests that do not have good condition/decision coverage. A reader of the test must infer the property implied by the test from the example data the programmer selected and the name of the test. Conversely, property-based testing literature has focused on testing for quality assurance. The literature does not describe property-based testing tools being used to drive development - focus attention, measure progress, and obtain design feedback. (Saff's article on JUnit theories is a notable exception). How does a property-based testing tool affect the TDD process & the designs that the process drives? Let’s find out! Questions we'd like to explore include: * How do we get from examples to properties? Do we always need to start with examples and generalise to properties, or can we start with properties in some cases? And if so, when, and when not? * How do we change properties as the system grows? Can we start with simple properties and generalise or specialise them? How does that compare to adding additional examples one at a time? * Can property-based testing drive design, and if so how? If we focus on properties, do we get different designs than if we focus on examples? Are some design styles easier than others to describe in terms of properties? * Top-down/outside-in vs. bottom-up/inside-out: does property based testing work better | ||
| Audience background: | - Developers - Proficient in any language (as long as they can find a property-based testing library for it and somebody to pair with) - Experienced in test-driven development - Ideally will have already attended Keith Braithwaite's "TDD As If You Meant It" workshop so we can compare & contrast doing the same exercise with different kinds of testing | ||
| Benefits of participating: | To learn how property-based testing tools can (or maybe cannot) fit into the TDD process. | ||
| Materials provided: | A QuickCheck implementation for Python. http://pypi.python.org/pypi/factcheck/1.1.0.0 A list of QuickCheck implementations for various languages. https://gist.github.com/4147916 Further reading: [CH2000] Koen Claessen and John Hughes (2000). “QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs.” Proc. Of International Conference on Functional Programming (ICFP), ACM SIGPLAN. http://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf [SB2006] David Saff, Marat Boshernitsan, The Practice of Theories: Adding “For-all” Statements to “There-Exists” Tests, Submitted to IEEE Software, Special Issue on Test-Driven Development. http://shareandenjoy.saff.net/2006/12/new-paper-practice-of-theories.html | ||
| Process: | Hands-on coding, in pairs, with occasional breaks for discussion. We will split the room. Half will work in an "outside-in" TDD style. The other half working in "inside-out". A wrap up at the end to compare how things went for the outside-in and inside-out teams and collect lessons learned in poster form. | ||
| Detailed timetable: | 0:00 – 0:20 Presentation. Introduce the rules of TDD as if you meant it, property-based testing tools & property-based TDD with QuickCheck, JUnit 4 Theories, or similar tools. A demo of writing and running a property-based test. 0:20 - 0:25 - Organise into pairs based on experience and language choice. Split the room into the inside-out and outside-in teams. 0:25 – 1:00 – First coding period. (Presenters go around encouraging participants to follow the rules of TDD As If You Meant It). 1:00 – 1:15 – Discussion. What’s hard? How does it compare to TDD with examples? Anyone got *anywhere*? Share what we’ve learned. 1:15 – 2:30 Continue coding, maybe trying a new approach based on what everyone has shared. 2:30 – 3:00 Show and Tell & discussion. * How do property-based testing tools change TDD? * Do some coding styles, languages or problems make it easier to use property-based testing in a TDD style? * Does property-based testing push your design in any particular direction? * How can we change the exercise to better explore property-based TDD? | ||
| Outputs: | Posters and/or a blog post collecting what the participants discovered about how to fit property-based testing into the TDD process and how their designs were affected by the different toolset. | ||
| History: | This session has been run at XP Day London 2012. | ||
| Presenters | |||
| 1. Nat Pryce Technemetis Ltd. |
2. Keith Braithwaite Zuhlke Ltd |
3. | |