![\"Hexagonal](\"../assets/img/hexagonalArchitecture.png\")
We at Agilogy love our craft. And we like having technical discussions, trainings and workshops that allow us to continuously improve the way we develop software. That’s the reason we devote part of our Friday’s mornings to what we call the Agilogy School. These sessions are the weekly occasion for everyone at Agilogy to share and learn about common interests like testing, programming languages, functional programming, software architecture, design, etc.
\n\nAs the main facilitator, I have decided to make 2024 the year of software testing at the Agilogy School.
\n\nTo do so, we needed a common starting point, for which we used the Hexagonal Architecture, which we reviewed so that all of us share an ubiquitous language when talking about its components You know, driven ports and adapters, driving ports and adapters… or where they driver ports and adapters?.
\n\n
From there on, we started, as we should, by the basic principles, patterns and strategies that are common to software testing in any technology or programming language.
\n\nIn the following weeks or months I’ll be blogging about these testing strategies we use at Agilogy. We may even write about Hexagonal Architecture before that, so we make it clear the way we talk about it.
\n\nIn the meantime, I’m sharing the slides of the workshop we are working on:
\n\n\n\nYou can download the PDF of the slides here.
\n\nFurthermore, I want to take the opportunity to review things I already shared in this blog, now that I have some names for the patterns and strategies I was talking about back then.
\n\nIn What is an automated test, again? I talked about how to test methods that have state:
\n\n// 1. Set initial state:\nval ma = MemoryAdder()\nma.add(23)\n// 2 and 3. Execute the method and assert\nassertEquals(25, ma.add(2))\n// 4. Collect the final state and assert\nassertEquals(2, ma.lastInput)\nI have identified this way of testing methods with state as a pattern I call Arrange, Act, Assert with State, or AAA with State for short.
\n\nIn Testing and persistent state I showed how we can apply the AAA with State pattern to test components whose state is persisted:
\n\n// 1. Set initial state\nval userId = 23L\nval userName = \"John\"\nconn.execute(\"delete from users\")\nconn.execute(\n \"insert into users(id, name, last_visit), ?, ?, null\", \n userId, \n userName\n)\n// 2 and 3. Execute the method and assert\nval lastVisit = Instant.now()\nval wasUpdated = updateUserLastVisit(userId, lastVisit)\nassertTrue(wasUpdated)\n// 4. Collect the final state and assert\nval endState =\n conn.execute(\"select id,name,last_visit from users order by id\"){ \n (i, n, lv) -> User(id, n, lv)\n }\nassertEquals(listOf(User(userId, userName, lastVisit)), endState)\nIf we apply this to an Hexagonal Architecture, we’ll see that the Driven Adapters are the perfect example of such a case. We say these are Integration Tests because we are testing (part of) our system in integration with an actual external system (an actual database, in the example).
\n\n![\"Integration](\"../assets/img/integration_test_of_driven_adapters.png\")
In Testing other side effects we started discussing how to test side effects other than state. That led us to discuss Dependency Injection and software design in general.
\n", "url": "https://blog.agilogy.com/2024-02-02-testing-the-agilogy-way.html", "date_published": "2024-02-02T00:00:00+01:00", "date_modified": "2024-02-02T00:00:00+01:00", "author": { "name": "Jordi Pradel" } }, { "id": "https://blog.agilogy.com/2024-01-12-writing-a-pbt-library-4-housekeeping.html", "title": "Writing a Property Based Testing library in Kotlin, a Journey. Part 4: Housekeeping", "content_text": "I’ve had some busy months since I last wrote anything for the blog. Let’s start 2024 by retaking the blog. Today I rescue an entry that I had in progress. We’ll do some housekeeping of the Property Based Library I was writing back in 2023.\n\nIn the previous articles in this series (1, 2 and 3) I was developing a property based testing library from scratch and blogging about the process. It turns out that such library actually exists as a companion repository you can follow while reading. It’s here. So far all articles in the series were about property based testing themselves. But, at the same time, I’m blogging about the actual design of a library.\n\n\nLet’s start!\n\nAs I was focusing on writing about PBT the library has accumulated some problems I’d like fixed. And you bet I’ll be blogging about this too.\n\nReorganizing tests\n\nAs I’m using the library to write this blog series, I’m actually programming (and running the tests) for the code I use in the blog. But some of those code fragments are actually usage examples (like Int tests, DayOfWeek and Geometry). Let’s put all of those in a simple package com.agilogy.wapbtl.examples so that they are not mixed with actual tests of the library, that will be in com.agilogy.wapbtl.test. Examples should probably be in a different project in the multiproject build but I’m frankly not in the mood for a fight with Gradle to get that done. Not now.\n\nAs a result of such cleaner separation, our examples of property based testing of Int where somehow poor. I thought I needed some other property checked, so I checked Int sum is associative:\n\n@Test\nfun testAssociativity(){\n forAny(Arb.int, Arb.int, Arb.int){ a, b, c -> (a + b) + c == a + (b + c)}\n}\n\n\nI also refactored the class names so all tests (those testing the library and those working as usage examples) all follow the same pattern: XXXTest.\n\nEncountering (and fixing) a bug\n\nWhen programming the additional Int test above I got a green test the first time I run it. I don’t know you, but if I don’t see a test fail I can’t trust it. So I hacked the property so that it failed and I got this:\n\n@Test\nfun testAssociativity(){\n forAny(Arb.int, Arb.int, Arb.int){ a, b, c -> (a + b) + c == a + (b + c + 1)}\n}\n\n\nProperty failed at attempt 1 with sample false.\nSeed: 4769791297438385931\n\n\nIt failed ok. I have a seed. Yeah!! But, hey, what the heck means “with sample false”??? I wanted it to give me… what, the 3 samples the test used? This is obviously a bug. A design bug, I’d say, as I’m not even sure what PropertyFailedException should be like.\n\n📝 A quick retrospective: I said I like to write tests of everything I develop I didn’t live up to these expectations. This property based testing library is indeed poorly tested. Let’s take note of this technical debt… and, as you do with debts, let’s pay it promptly.\n\nBut as much as I care about paying this debt, I have a production bug to fix first. Let’s fix that. And let’s start by designing the solution.\n\nOur current PropertyFailedException implementation isThis Any? type is something I already hate. And, oh! surprise! it is failing to us now. What I would like is for PropertyFailedException to be parameterized like class PropertyFailedException<A>(val attempt: Int, val seed: Long, val sample: A) but Kotlin complains with “Subclass of ‘Throwable’ may not have type parameters”. So let’s attempt to continue to live with that. For now.:\n\nclass PropertyFailedException(\n val attempt: Int,\n val seed: Long,\n val sample: Any?\n): Exception(...)\n\n\nWhen forAny takes more than one Arg, what would the sample be? We could use tuples and have a sample of type Pair<A, B> if we were given 2 args, and a Triple<A, B, C> if we were given 3. What if we want a forAny of 4 or more values? Coming from Scala I’d say bigger tuples (Tuple4, Tuple5…) would totally make sense here… even though they are not idiomatic in Kotlin. I could use a List<Any?> and put all samples in that list, but I hate to have something that poorly typed. So, lacking a better solution, I will go with the two tuples we already have. Any ideas are welcome here, if you want to tweet or toot (with a mention) or DM them. I’m @jordipradel@fosstodon.org (Mastodon, preferred) and @agile_jordi (Twitter).\n\nSo now that we decided we expect a tuple, we can fix the bug. But, hey, I just did a quick retrospective, right? Don’t let the gap grow bigger and fix the bug the way se should: by first reproducing it:\n\n@Test\nfun forAny2ArbsFailure() {\n val property: (Int, Int) -> Boolean = { a, b -> (a + b) > a }\n var attempts = 0\n val failure = assertFailsWith<PropertyFailedException> {\n forAny(Arb.int, Arb.int) { a, b ->\n attempts += 1\n property(a, b) \n }\n }\n @Suppress(\"UNCHECKED_CAST\")\n val failedSample = failure.sample as Pair<Int, Int>\n assertFalse { property(failedSample.first, failedSample.second) }\n assertEquals(attempts, failure.attempt)\n}\n\n\nSo, letting aside the ugly casts, we are testing that whenever a forAny that takes two Arb<Int> fails, it should return a Pair<Int,Int> as the failed sample, and that checking the property on those 2 values should fail. And, it fails. Yay! Yes, yay! Because when you are writing a test you expect it to be useful. And a test that you write but never fails is giving you exactly 0 value. We are also testing what we already tested for the one arbitrary version of forAny, to make sure we didn’t break the attempt result.\n\nclass java.lang.Boolean cannot be cast to class kotlin.Pair\njava.lang.ClassCastException: (...)\n\tat com.agilogy.wapbtl.test.ForAnyTest.(...)\n\t...\n\n\nOk, now fix this. The problem is the fancy trick I did to avoid creating a tuple:\n\nfun <A, B, C> forAny(\n a: Arb<A>, b: Arb<B>, c: Arb<C>, seed: Long? = null, \n property: (A, B, C) -> Boolean\n): Unit =\n forAny(Arb.product3(a, b, c, property), seed) { it }\n\n\nRemember? We were building an Arb<Boolean> that directly told us whether the property passed or not and passing that to the forAny that only takes one Arb. Therefore, we can’t recover the sample values once the forAny fails. But the tuple we wanted to avoid creating is now created at least when the test fails. So I’ll undo the trick and simplify things. In the end, this was a premature optimization, probably. Or the Scala developer in me being confused by the lack of tuples in Kotlin. So, let’s simplify:\n\nfun <A, B> forAny(\n a: Arb<A>, b: Arb<B>, seed: Long? = null, \n property: (A, B) -> Boolean\n): Unit = \n forAny(Arb.pair(a, b), seed) { (a, b) -> property(a, b) }\n\nfun <A, B, C> forAny(\n a: Arb<A>, b: Arb<B>, c: Arb<C>, seed: Long? = null,\n property: (A, B, C) -> Boolean\n): Unit =\n forAny(Arb.triple(a, b, c), seed) { (a, b, c) -> property(a, b, c) }\n\n\nAnd now our test passes!\n\nTest exceptions\n\nThat’s a basic one I didn’t introduce in the first version just for the sake of a simpler explanation of what property based testing means. Imagine you are trying to check a property of Integer division:\n\n@Test\nfun testIntegerDivision() {\n forAny(Arb.int, Arb.int) { a, b ->\n val d = a / b\n val m = a % b\n a == b * d + m\n }\n}\n\n\nAs you already know this is an actual property of integer division… except when b is 0. Of course we can fix the test with a simple if, but let’s imagine you are not aware what the problem is. The result you currently get is:\n\n/ by zero\njava.lang.ArithmeticException: / by zero\n\tat com.agilogy.wapbtl.examples....\n\t...\n\n\nNo PropertyFailedException, no seed, no samples. You are on your own to understand the problem. And it may be a flaky test, where you were fortunate enough to catch an example that fails… but that doesn’t come along that frequently in your test.\n\nAs is customary in testing libraries, we would like to treat an exception as a test failure. Therefore, we may have 3 possible outcomes of the evaluation of your property for a given example. Either the property successfully returns true, or it successfully returns false or it fails with an exception. We’ll consider both returning false or throwing an exception a way of telling us the property does not abide.\n\nFor that we need to… first write a test!\n\n@Test\nfun testForAnyException() {\n val property: (Int) -> Boolean = { i ->\n if (i < 0) throw IllegalArgumentException(\"$i\")\n true\n }\n var attempts = 0\n val failure = assertFailsWith<PropertyFailedException> {\n forAny(Arb.int) {\n attempts += 1\n property(it)\n }\n }\n val failedSample = failure.sample as Int\n assertTrue(failure.cause is IllegalArgumentException)\n assertEquals(\"$failedSample\", failure.cause?.message)\n assertEquals(attempts, failure.attempt)\n}\n\n\nNow we can fix our forAny implementation and make the test pass:\n\nfun <A> forAny(r: Arb<A>, seed: Long? = null, property: (A) -> Boolean) {\n fun test(currentSeed: Long, attemptNumber: Int) {\n val sample = r.generate(Random(currentSeed))\n runCatching { property(sample) }.fold(\n onSuccess = { result ->\n if (!result) throw PropertyFailedException(attemptNumber, currentSeed, sample)\n },\n onFailure = { exception ->\n throw PropertyFailedException(attemptNumber, currentSeed, sample, exception)\n }\n )\n }\n if (seed == null) {\n (1..100).forEach { attemptNumber ->\n test(Random.Default.nextLong(), attemptNumber)\n }\n } else {\n test(seed, 1)\n }\n}\n\n\nPropertyFailedException is now:\n\nclass PropertyFailedException(\n val attempt: Int,\n val seed: Long,\n val sample: Any?,\n override val cause: Throwable? = null\n) : Exception(\n \"\"\"Property failed at attempt $attempt\n |Sample $sample\n |Seed: $seed\n |Cause: ${cause?.message}\"\"\".trimMargin()\n)\n\n\nAnd our int division test now throws:\n\ncom.agilogy.wapbtl.PropertyFailedException: Property failed at attempt 1\nSample (-1838537511, 0)\nSeed: -2405689340444717625\nCause: / by zero\n\tat app//com.agilogy.wapbtl.ForAnyKt.forAny$test(forAny.kt:25)\n\t...\nCaused by: java.lang.ArithmeticException: / by zero\n\tat com.agilogy.wapbtl.examples.ForAnyIntTest$testIntegerDivision$1...\n\n\nYour favourite assertion library\n\nAs a side effect of handling exceptions as test failures, now you could use your favourite assertions library to check properties if you prefer:\n\nforAny(Arb.int, Arb.int) { a, b ->\n val d = a / b\n val m = a % b\n assertEquals(a, b * d + m)\n}\n\n\nThat has one important advantage: In case of failure now you get the descriptive behaviour your assertion library carefully provides. And one disadvantatge: it does not compile, because forAny expects a function that returns a Boolean. Of course you could return true, but that would be horrible. So let’s do something about it.\n\nRed:\n\n@Test\nfun testTestForAny() {\n val property: (Int) -> Unit = { i ->\n if (i < 0) throw AssertionError(\"$i\")\n }\n var attempts = 0\n val failure = assertFailsWith<PropertyFailedException> {\n testForAny(Arb.int) {\n attempts += 1\n property(it)\n }\n }\n val failedSample = failure.sample as Int\n assertTrue(failure.cause is AssertionError)\n assertEquals(\"$failedSample\", failure.cause?.message)\n assertEquals(attempts, failure.attempt)\n}\n\n\nGreen:\n\nfun <A> testForAny(aa: Arb<A>, seed: Long? = null, property: (A) -> Unit) =\n forAny(aa, seed) { a->\n property(a)\n true\n }\n\n\nRefactor: I could reuse some lines between the two functions that test how the property based tests fail, but the resulting code is too much difficult to grasp.\n\nHaving fun with fun interfaces\n\nOne more thing… So far, every time we define a new instance of Arb we go like this:\n\nval Arb.Companion.float: Arb<Float>\n get() = object : Arb<Float> {\n override fun generate(random: Random): Float = \n random.nextFloat()\n }\n\n\nBut you may know Kotlin has this nice feature called functional interfaces. We only need to add the fun keyword to our interface:\n\nfun interface Arb<A> {\n fun generate(random: Random): A\n\n companion object\n}\n\n\nAnd we can now simplify all those object : Arb<...> expressions like in the example below:\n\nval Arb.Companion.float: Arb<Float>\n get() = Arb { random -> random.nextFloat() }\n\n\nUpgrading Gradle and libraries\n\nFinally, since it’s been quite long since I last worked on this library, some of its dependencies are a bit outdated. Let us fix that.\n\nI’ll start with Gradle. Acording to ./gradlew --version, the project is currently using Gradle 7.3.3. What version should we upgrade to? Let’s do some research:\n\n\n The latest Gradle version at the time of writing is 8.5\n The Kotlin Gradle Plugin latest supported Gradle version is 8.1.1 acording to the documentation.\n But Ktor is using 8.4\n\n\nSo we’ll use 8.4.\n\nWe could change files by hand, but there is already a Gradle task to upgrade the gradle wrapper:\n\n$ gradle wrapper --gradle-version 8.4\n...\n$ git status\n...\nChanges not staged for commit:\n modified: gradle/wrapper/gradle-wrapper.jar\n modified: gradle/wrapper/gradle-wrapper.properties\n modified: gradlew\n modified: gradlew.bat\n...\n$ git commit -am \"Upgrade Gradle to latest supported version\"\n\n\nNow we can upgrade the Kotlin Gradle Plugin to the latest version. For that, we need to change the corresponding line in build.gradle.kts:\n\nkotlin(\"multiplatform\") version \"1.9.22\"\n\n\nBut then, we’ll find our build fails and, when fixed, emits some warnings. Here are some changes we need to do to build.gradle.kts:\n\n\n cssSupport.enabled = true is not supported anymore. We need to replace it with cssSupport{enabled = true}\n The Kotlin/JS Legacy compiler backend is deprecated. We need to replace js(LEGACY) with js.\n Variable nativeTarget is never used. I wanted a Kotlin multiplatform build so that the library can be used in any platform. But I don’t want to spend time configuring native builds now, so I’ll simply remove the build for those. We can add them later on if we want.\n\n\nFurthermore, Kotlin 1.9.20 brought us templates for multiplatform projects. The neat result is that we can simply remove all the manual sourceSets configuration we had and replace it with just the configuration of the dependencies we need:\n\ndependencies {\n commonTestImplementation(kotlin(\"test\"))\n}\n\n\nRecap\n\nWe made some maintenance tasks on our beloved Property Based Testing library. We reorganized packages to better distinguish between tests and examples. As a result of that, we improved some example tests and found a design failure that we fixed using TDD. We enhanced the usability of the library by adding explicit support to exceptions during tests. Finally, we upgraded both Gradle and Kotlin to the latest current versions.\n\nAll articles in the series\n\n\n Writing a property based testing library, part 1\n Writing a property based testing library, part 2\n Writing a property based testing library, part 3\n Writing a property based testing library, part 4\n\n", "content_html": "I’ve had some busy months since I last wrote anything for the blog. Let’s start 2024 by retaking the blog. Today I rescue an entry that I had in progress. We’ll do some housekeeping of the Property Based Library I was writing back in 2023.
\n\nIn the previous articles in this series (1, 2 and 3) I was developing a property based testing library from scratch and blogging about the process. It turns out that such library actually exists as a companion repository you can follow while reading. It’s here. So far all articles in the series were about property based testing themselves. But, at the same time, I’m blogging about the actual design of a library.
\n\n![\"Pulp](\"../assets/img/winstonWolf.gif\")
\nLet’s start!
As I was focusing on writing about PBT the library has accumulated some problems I’d like fixed. And you bet I’ll be blogging about this too.
\n\nAs I’m using the library to write this blog series, I’m actually programming (and running the tests) for the code I use in the blog. But some of those code fragments are actually usage examples (like Int tests, DayOfWeek and Geometry). Let’s put all of those in a simple package com.agilogy.wapbtl.examples so that they are not mixed with actual tests of the library, that will be in com.agilogy.wapbtl.test. Examples should probably be in a different project in the multiproject build but I’m frankly not in the mood for a fight with Gradle to get that done. Not now.
As a result of such cleaner separation, our examples of property based testing of Int where somehow poor. I thought I needed some other property checked, so I checked Int sum is associative:
@Test\nfun testAssociativity(){\n forAny(Arb.int, Arb.int, Arb.int){ a, b, c -> (a + b) + c == a + (b + c)}\n}\nI also refactored the class names so all tests (those testing the library and those working as usage examples) all follow the same pattern: XXXTest.
When programming the additional Int test above I got a green test the first time I run it. I don’t know you, but if I don’t see a test fail I can’t trust it. So I hacked the property so that it failed and I got this:
@Test\nfun testAssociativity(){\n forAny(Arb.int, Arb.int, Arb.int){ a, b, c -> (a + b) + c == a + (b + c + 1)}\n}\nProperty failed at attempt 1 with sample false.\nSeed: 4769791297438385931\nIt failed ok. I have a seed. Yeah!! But, hey, what the heck means “with sample false”??? I wanted it to give me… what, the 3 samples the test used? This is obviously a bug. A design bug, I’d say, as I’m not even sure what PropertyFailedException should be like.
📝 A quick retrospective: I said I like to write tests of everything I develop I didn’t live up to these expectations. This property based testing library is indeed poorly tested. Let’s take note of this technical debt… and, as you do with debts, let’s pay it promptly.
\n\nBut as much as I care about paying this debt, I have a production bug to fix first. Let’s fix that. And let’s start by designing the solution.
\n\nOur current PropertyFailedException implementation isThis Any? type is something I already hate. And, oh! surprise! it is failing to us now. What I would like is for PropertyFailedException to be parameterized like class PropertyFailedException<A>(val attempt: Int, val seed: Long, val sample: A) but Kotlin complains with “Subclass of ‘Throwable’ may not have type parameters”. So let’s attempt to continue to live with that. For now.:
class PropertyFailedException(\n val attempt: Int,\n val seed: Long,\n val sample: Any?\n): Exception(...)\nWhen forAny takes more than one Arg, what would the sample be? We could use tuples and have a sample of type Pair<A, B> if we were given 2 args, and a Triple<A, B, C> if we were given 3. What if we want a forAny of 4 or more values? Coming from Scala I’d say bigger tuples (Tuple4, Tuple5…) would totally make sense here… even though they are not idiomatic in Kotlin. I could use a List<Any?> and put all samples in that list, but I hate to have something that poorly typed. So, lacking a better solution, I will go with the two tuples we already have. Any ideas are welcome here, if you want to tweet or toot (with a mention) or DM them. I’m @jordipradel@fosstodon.org (Mastodon, preferred) and @agile_jordi (Twitter).
So now that we decided we expect a tuple, we can fix the bug. But, hey, I just did a quick retrospective, right? Don’t let the gap grow bigger and fix the bug the way se should: by first reproducing it:
\n\n@Test\nfun forAny2ArbsFailure() {\n val property: (Int, Int) -> Boolean = { a, b -> (a + b) > a }\n var attempts = 0\n val failure = assertFailsWith<PropertyFailedException> {\n forAny(Arb.int, Arb.int) { a, b ->\n attempts += 1\n property(a, b) \n }\n }\n @Suppress(\"UNCHECKED_CAST\")\n val failedSample = failure.sample as Pair<Int, Int>\n assertFalse { property(failedSample.first, failedSample.second) }\n assertEquals(attempts, failure.attempt)\n}\nSo, letting aside the ugly casts, we are testing that whenever a forAny that takes two Arb<Int> fails, it should return a Pair<Int,Int> as the failed sample, and that checking the property on those 2 values should fail. And, it fails. Yay! Yes, yay! Because when you are writing a test you expect it to be useful. And a test that you write but never fails is giving you exactly 0 value. We are also testing what we already tested for the one arbitrary version of forAny, to make sure we didn’t break the attempt result.
class java.lang.Boolean cannot be cast to class kotlin.Pair\njava.lang.ClassCastException: (...)\n\tat com.agilogy.wapbtl.test.ForAnyTest.(...)\n\t...\nOk, now fix this. The problem is the fancy trick I did to avoid creating a tuple:
\n\nfun <A, B, C> forAny(\n a: Arb<A>, b: Arb<B>, c: Arb<C>, seed: Long? = null, \n property: (A, B, C) -> Boolean\n): Unit =\n forAny(Arb.product3(a, b, c, property), seed) { it }\nRemember? We were building an Arb<Boolean> that directly told us whether the property passed or not and passing that to the forAny that only takes one Arb. Therefore, we can’t recover the sample values once the forAny fails. But the tuple we wanted to avoid creating is now created at least when the test fails. So I’ll undo the trick and simplify things. In the end, this was a premature optimization, probably. Or the Scala developer in me being confused by the lack of tuples in Kotlin. So, let’s simplify:
fun <A, B> forAny(\n a: Arb<A>, b: Arb<B>, seed: Long? = null, \n property: (A, B) -> Boolean\n): Unit = \n forAny(Arb.pair(a, b), seed) { (a, b) -> property(a, b) }\n\nfun <A, B, C> forAny(\n a: Arb<A>, b: Arb<B>, c: Arb<C>, seed: Long? = null,\n property: (A, B, C) -> Boolean\n): Unit =\n forAny(Arb.triple(a, b, c), seed) { (a, b, c) -> property(a, b, c) }\nAnd now our test passes!
\n\nThat’s a basic one I didn’t introduce in the first version just for the sake of a simpler explanation of what property based testing means. Imagine you are trying to check a property of Integer division:
\n\n@Test\nfun testIntegerDivision() {\n forAny(Arb.int, Arb.int) { a, b ->\n val d = a / b\n val m = a % b\n a == b * d + m\n }\n}\nAs you already know this is an actual property of integer division… except when b is 0. Of course we can fix the test with a simple if, but let’s imagine you are not aware what the problem is. The result you currently get is:
/ by zero\njava.lang.ArithmeticException: / by zero\n\tat com.agilogy.wapbtl.examples....\n\t...\nNo PropertyFailedException, no seed, no samples. You are on your own to understand the problem. And it may be a flaky test, where you were fortunate enough to catch an example that fails… but that doesn’t come along that frequently in your test.
As is customary in testing libraries, we would like to treat an exception as a test failure. Therefore, we may have 3 possible outcomes of the evaluation of your property for a given example. Either the property successfully returns true, or it successfully returns false or it fails with an exception. We’ll consider both returning false or throwing an exception a way of telling us the property does not abide.
For that we need to… first write a test!
\n\n@Test\nfun testForAnyException() {\n val property: (Int) -> Boolean = { i ->\n if (i < 0) throw IllegalArgumentException(\"$i\")\n true\n }\n var attempts = 0\n val failure = assertFailsWith<PropertyFailedException> {\n forAny(Arb.int) {\n attempts += 1\n property(it)\n }\n }\n val failedSample = failure.sample as Int\n assertTrue(failure.cause is IllegalArgumentException)\n assertEquals(\"$failedSample\", failure.cause?.message)\n assertEquals(attempts, failure.attempt)\n}\nNow we can fix our forAny implementation and make the test pass:
fun <A> forAny(r: Arb<A>, seed: Long? = null, property: (A) -> Boolean) {\n fun test(currentSeed: Long, attemptNumber: Int) {\n val sample = r.generate(Random(currentSeed))\n runCatching { property(sample) }.fold(\n onSuccess = { result ->\n if (!result) throw PropertyFailedException(attemptNumber, currentSeed, sample)\n },\n onFailure = { exception ->\n throw PropertyFailedException(attemptNumber, currentSeed, sample, exception)\n }\n )\n }\n if (seed == null) {\n (1..100).forEach { attemptNumber ->\n test(Random.Default.nextLong(), attemptNumber)\n }\n } else {\n test(seed, 1)\n }\n}\nPropertyFailedException is now:
class PropertyFailedException(\n val attempt: Int,\n val seed: Long,\n val sample: Any?,\n override val cause: Throwable? = null\n) : Exception(\n \"\"\"Property failed at attempt $attempt\n |Sample $sample\n |Seed: $seed\n |Cause: ${cause?.message}\"\"\".trimMargin()\n)\nAnd our int division test now throws:
\n\ncom.agilogy.wapbtl.PropertyFailedException: Property failed at attempt 1\nSample (-1838537511, 0)\nSeed: -2405689340444717625\nCause: / by zero\n\tat app//com.agilogy.wapbtl.ForAnyKt.forAny$test(forAny.kt:25)\n\t...\nCaused by: java.lang.ArithmeticException: / by zero\n\tat com.agilogy.wapbtl.examples.ForAnyIntTest$testIntegerDivision$1...\nAs a side effect of handling exceptions as test failures, now you could use your favourite assertions library to check properties if you prefer:
\n\nforAny(Arb.int, Arb.int) { a, b ->\n val d = a / b\n val m = a % b\n assertEquals(a, b * d + m)\n}\nThat has one important advantage: In case of failure now you get the descriptive behaviour your assertion library carefully provides. And one disadvantatge: it does not compile, because forAny expects a function that returns a Boolean. Of course you could return true, but that would be horrible. So let’s do something about it.
Red:
\n\n@Test\nfun testTestForAny() {\n val property: (Int) -> Unit = { i ->\n if (i < 0) throw AssertionError(\"$i\")\n }\n var attempts = 0\n val failure = assertFailsWith<PropertyFailedException> {\n testForAny(Arb.int) {\n attempts += 1\n property(it)\n }\n }\n val failedSample = failure.sample as Int\n assertTrue(failure.cause is AssertionError)\n assertEquals(\"$failedSample\", failure.cause?.message)\n assertEquals(attempts, failure.attempt)\n}\nGreen:
\n\nfun <A> testForAny(aa: Arb<A>, seed: Long? = null, property: (A) -> Unit) =\n forAny(aa, seed) { a->\n property(a)\n true\n }\nRefactor: I could reuse some lines between the two functions that test how the property based tests fail, but the resulting code is too much difficult to grasp.
\n\nOne more thing… So far, every time we define a new instance of Arb we go like this:
val Arb.Companion.float: Arb<Float>\n get() = object : Arb<Float> {\n override fun generate(random: Random): Float = \n random.nextFloat()\n }\nBut you may know Kotlin has this nice feature called functional interfaces. We only need to add the fun keyword to our interface:
fun interface Arb<A> {\n fun generate(random: Random): A\n\n companion object\n}\nAnd we can now simplify all those object : Arb<...> expressions like in the example below:
val Arb.Companion.float: Arb<Float>\n get() = Arb { random -> random.nextFloat() }\nFinally, since it’s been quite long since I last worked on this library, some of its dependencies are a bit outdated. Let us fix that.
\n\nI’ll start with Gradle. Acording to ./gradlew --version, the project is currently using Gradle 7.3.3. What version should we upgrade to? Let’s do some research:
So we’ll use 8.4.
\n\nWe could change files by hand, but there is already a Gradle task to upgrade the gradle wrapper:
\n\n$ gradle wrapper --gradle-version 8.4\n...\n$ git status\n...\nChanges not staged for commit:\n modified: gradle/wrapper/gradle-wrapper.jar\n modified: gradle/wrapper/gradle-wrapper.properties\n modified: gradlew\n modified: gradlew.bat\n...\n$ git commit -am \"Upgrade Gradle to latest supported version\"\nNow we can upgrade the Kotlin Gradle Plugin to the latest version. For that, we need to change the corresponding line in build.gradle.kts:
kotlin(\"multiplatform\") version \"1.9.22\"\nBut then, we’ll find our build fails and, when fixed, emits some warnings. Here are some changes we need to do to build.gradle.kts:
cssSupport.enabled = true is not supported anymore. We need to replace it with cssSupport{enabled = true}js(LEGACY) with js.nativeTarget is never used. I wanted a Kotlin multiplatform build so that the library can be used in any platform. But I don’t want to spend time configuring native builds now, so I’ll simply remove the build for those. We can add them later on if we want.Furthermore, Kotlin 1.9.20 brought us templates for multiplatform projects. The neat result is that we can simply remove all the manual sourceSets configuration we had and replace it with just the configuration of the dependencies we need:
dependencies {\n commonTestImplementation(kotlin(\"test\"))\n}\nWe made some maintenance tasks on our beloved Property Based Testing library. We reorganized packages to better distinguish between tests and examples. As a result of that, we improved some example tests and found a design failure that we fixed using TDD. We enhanced the usability of the library by adding explicit support to exceptions during tests. Finally, we upgraded both Gradle and Kotlin to the latest current versions.
\n\nThis time I’m in the mood for playing a bit with my good old friend Java. While preparing a kata for my students I wanted some simple database access library they could use without previous knowledge or experience. I could have looked at many alternatives, but I really wanted something simple.
\n\nSimple like I’m assuming Java 11 or later here. As a reminder, 11 is the LTS that was released on 09/2018 and is planned to be supported through 2026. I will mostly need lambdas (which are around since Java 8, released on 3/2014 and supported through at least 2030). You’ll see vars as well, which would require Java 11 (counting only LTSs), but they are easy to replace with their corresponding types if you are stuck with Java 8.:
List<User> users = tx.select(\n \"select u.id, u.name, u.surname from users u \" +\n \"where u.active = true amd u.role = ?\",\n (rs) -> new User(rs.getString(1), rs.getString(2), rs.getString(3)),\n p(\"student\") // The parameter value for the ? in the query\n);\nSummarizing the idea:
\n\nResultSet.next(), so there is no way she can fail to do so correctly.p(xxx) for any admissible value xxx where admissible means it must be a value of a type supported by the library.I’ve used this very same approach with Scala (I admit some implicits where used there) and Kotlin (look, ma! no implicits!). So let’s try my hand at Java.
\n\nUnder the hood I’m using Jdbc. Yes, it blocks threads, but that may be solved sooner than anyone expected. And it throws ugly checked exceptions we will need to deal with. In fact, it very poorly handles errors in a non standard way. But other than that, it is fine and very well known.
\n\nThe functions I will offer (like select, selectOne, update, etc.) will need a Connection. As per Jdbc itself, we need to be able to use the same Connection to invoke several database operations in a given transaction. So let’s use dependency injection! I could simply add a parameter to every funciton needing a Connection, but I prefer to create a class to represent the scope of a connection, a class where those fancy operations will live.
public class ConnectionScope {\n private final Connection connection;\n\n public ConnectionScope(Connection connection) {\n this.connection = connection;\n }\n\n public <A> List<A> select(String sql, RowReader<A> reader, Param... params) {...}\n public int update(String sql, Param... params) {...}\n ...\n}\nSo now our beloved user needs a way to get her ConnectionScope. The standard way to get a Connection is a DataSource that will usually have a connection pool like HikariDb behind it. But the main problem with Connection and every other resource we use is that we need to remember to close it. Fortunately enough, we got Autocloseable long ago and its a bit easier. But you can still forget to use try-with-resources. So (inspired by the idea of Resource in Cats or Arrow) let’s try to build a simple, specific API to use connections without having to remember to close them. And let’s inject a DataSource to that function by injecing it to the class where the method is:
public final class Jdbc {\n\n private final DataSource dataSource;\n\n public Jdbc(DataSource dataSource) {\n this.dataSource = dataSource;\n }\n\n public <A> A withConnection(Function1<Connection, A> f) {\n try (Connection conn = dataSource.getConnection()) {\n return f.apply(new ConnectionScope(conn));\n }\n }\n}\nThat way we can add other operations to Jdbc that use the very same DataSource, likeI could have used withConnection instead of duplicating the try-with-resources here.:
public <A> A inTransaction(Function<ConnectionScope, A> f) {\n try (Connection conn = dataSource.getConnection()) {\n conn.setAutoCommit(false);\n var res = f.apply(new ConnectionScope(conn));\n conn.commit();\n return res;\n } \n}\nNice, huh?
\n\nWell, I’m not being totally honest with you…
\n\n![\"image-20230303173817429\"](\"../assets/psGetConnectionThrows.png\")
No, you can’t call dataSource.getConnection() in your withConnection method because it throws the checked (and dreaded) exception SQLException. You need to either declare to also throw it or deal with it.
But the truth is I hate SQLException to be checked. Unless you know how to deal with them, exceptions should be unchecked. And:
This being just a litle game, I will simply embed that ugly exception in a RuntimeException and call it a day. If I ever want to recover from a duplicate key or something else I’ll go and figure out a solution. Future problem, YAGNI, KISS, etc I could use any other error handling mechanism (like an error monad), but I need this to be familiar to most Java programmers. So I’ll keep it simple familiar..
So:
\n\npublic <A> A withConnection(Function<Connection, A> f) {\n try (Connection conn = dataSource.getConnection()) {\n return f.apply(new ConnectionScope(conn));\n } catch (SQLException e) {\n throw new RuntimeException(e);\n }\n}\nThe very same happens when you call not only getConnection but also setAutocommit in inTransaction. So…
public <A> A inTransaction(Function<ConnectionScope, A> f) {\n try (Connection conn = dataSource.getConnection()) {\n conn.setAutoCommit(false);\n var res = f.apply(new ConnectionScope(conn));\n conn.commit();\n return res;\n } catch (SQLException e) {\n throw new RuntimeException(e);\n }\n }\nThat may be starting to get unpleasant. DRY apart, I don’t like the verbose nature of try..catch. But let’s live with it. After all, this try...catch boilerplate is one Alt+Enter away in my IntelliJ. And we don’t have that many of those.
Now our users can…
\n\nvar jdbc = new Jdbc(dataSource);\nvar users = jdbc.inTransaction((tx) -> ...use the ConnectionScope...));\nLet’s get that one done. It’s something like…:
\n\npublic class ConnectionScope {\n private final Connection connection;\n\n public <A> List<A> select(String sql, RowReader<A> reader, Object... params) {\n try (var stmt = connection.prepareStatement(sql)) {\n for (int i = 0; i < params.length; i++) stmt.setObject(i + 1, params[i]);\n try (var rs = stmt.executeQuery()) {\n var res = new LinkedList<A>();\n \tvar rsv = new ResultSetView(rs):\n while (rs.next()) {\n res.add(reader.apply(rsv));\n }\n return Collections.unmodifiableList(res);\n }\n } catch (SQLException ex) {\n throw new RuntimeException(ex);\n }\n }\n ...\n}\nDon’t worry about that RowReader<A> thing for a moment. I’m covering it later.
Some notes here, just in case you are new to Jdbc or too old to remember that:
\n\nConnection needs to be closed. And PreparedStatement, and ResultSet need to be closed too. At least they are Autocloseable and they caused some of my first professional nightmares back in the 1990’s. So, I’ll close each of them.PreparedStatement.setXXX methods like I do here, right? Tell me you are not injecting values into the SQL query by concatenating Strings!unmodifiableList because this is as close as I can get to an immutable colleciton in the standard library. Not that it matters a lot here, since I’m returning a freshly created instance every time. But, you know, the older I get, the less I want to reason about mutability.Did you see that ugly Object type? A signal of a disaster coming after me here… There are a thousand (painful) reasons I’m averse to things that could fail at compile time but fail at runtime. Did you say select(\"...\", ..., user1) and user1 is an instance of User, a class with 4 fields in it? No way. stmt.setObject won’t probably like that.
So let’s typesafe that. How? Ok, here is the (hopefully) cool idea I came with some time ago and I keep using everywhere I want some casual Jdbc:
\n\nPreparedStatement if its of a type X such that the API of PreparedStatement has a method void setX(int parameterIndex, X x). Boolean? Check! URL? Wut? Yes. User? Nope.PreparedStatement would be a valid argument value. Why? Because the existence of such a method in PreparedStatement proves… well, that we will be able to set the parameter value in the statement.void setX(int parameterIndex, X x) in PreparedStatementcan be seen as a java.util.function.Consumer but with 3 parameters: TriConsumer<PreparedStatement,Integer,X> (a function from these 3 types to void)… Except Java has no TriConsumerclass. If it had one we could model, for example, Integer parameters as: If you are frowning at setInt used with Integer… I know. I didn’t remember. Keep reading and I’ll fix it later.public class IntParam{\n private final Consumer3<PreparedStatement, Integer, Integer> \n preparedStatementSetter;\n private final Integer value;\n\n public IntParam(Integer value){\n this.preparedStatementSetter = PreparedStatement::setInt;\n this.value = value;\n }\n}\nTriConsumer<PreparedStatement,Integer,X> and a value of type X can be converted by currying the function. That is, we create a BiConsumer<PreparedStatement, Integer> where the third argument (of type X) is whatever value we wanted to set as a parameter:public class IntParam{\n private final BiConsumer<PreparedStatement, Integer> preparedStatementSetter;\n\n public IntParam(Integer value){\n this.preparedStatementSetter = (ps, index) -> ps.setInt(index, value);\n }\n}\nInteger value in its constructor and can be abstracted. If we squint, we can model that as an abstract class with one single method that, in fact, does not depend on Integer at all:public abstract class Param {\n // Define aan abstract method instead of a Consumer\n public abstract void set(PreparedStatement ps, int column);\n}\nParam. We could create a subclass for each type we support, but simply creating anonymous instances is easier:public Param integerParam(Integer value) {\n return new Param() {\n public void set(PreparedStatement ps, int column){\n ps.setInt(column, value);\n }\n };\n}\nFunctionalInterface::@FunctionalInterface\npublic interface Param {\n void set(PreparedStatement ps, int i)\n}\nNow I simply need more functions that, given a value, return an instance of Param. And they will use a lambda to create the resulting instance.
static Param p(Integer value) {\n return (ps, i) -> ps.setInt(i, value);\n}\n\nstatic Param p(String value) {\n return (ps, i) -> ps.setString(i, value);\n}\nI named all such funtions p so that the compiler, and not our user, gets to choose which one applies to the given parameter type. And all of a sudden my beloved user doesn’t need to remember what function to call, but just try p(x) for any x she wants passed as argument and only care about it if the compiler (rightfully) complains to not have any function p that takes values of such type as a parameter.
Now our select function can use the Param instance to set the parameter in a typesafe way:
public <A> List<A> select(String sql, RowReader<A> reader, Param... params) {\n try (var stmt = connection.prepareStatement(sql)) {\n for (int i = 0; i < params.length; i++) params[i].set(stmt, i + 1);\n ...\n }\n }\nEverything is nice and fine. Except, of course, I’m lying. Again.
\n\n![\"image-20230303181943679\"](\"../assets/psSetIntThrows.png\")
Naturally, I could surrond every sinlge invocation of setXxxx methods with try...catch. But you know how many setXxx methods are there in PreparedStatement? 50. I counted them. I’ve been told DRY is abused often. But I’m not writing 6 extra lines of try...catch (including its braces) multiplied by 49 more times. Enough is enough.
But this time the solution is actually simple. Although BiConsumer can’t represent anything that throws checked exceptions, let’s simply admit our Param functional interface can throw SQLExceptions and, all of a sudden, you can use lambdas like there were no exceptions to be seen.
Jedi stuff. These aren’t the exceptions you are looking for.\n![\"Obi](\"../assets/obi-wan-kenobi-these-are-not-the-droids.gif\")
@FunctionalInterface\npublic interface Param {\n void set(PreparedStatement ps, int i) throws SQLException;\n\n static Param p(Integer value) {\n return (ps, i) -> ps.setInt(i, value);\n }\n}\nIt may be no rocket science if you are really used to (modern-ish) Java. But in my opinion it shows the smarts of the designers of Java. setInt throws SQLException but it can be written in a Lambda because the FunctionalInterface you expect to return (Param) already declares the function to throw SQLException.
I’m just writing about the design and implementation of this toy library. But I’m fond of testing and, yes, I’m also testing it. So I’ve found a bug caused by my years away from Java.
\n\nDid you spot it already? It is here:
\n\nstatic Param p(Integer value) {\n return (ps, i) -> ps.setInt(i, value);\n}\nDid you guess it? Yes, ps.setInt takes an int value. When you pass an Integer value, it auto-unboxes. And if the value was null… Kaboom! NullPointerException. I could rewrite the previous section with String as the example and pretend I never did the mistake. But let’s be honest here. I did it. And now I’m fixing it:
static Param p(final Integer value) {\n return (ps, i) -> ps.setObject(i, value, Types.INTEGER);\n}\n\nstatic Param p(final int value) {\n return (ps, i) -> ps.setInt(i, value);\n}\nLast… and, for once, possibly least. We want our user to read the rows in a ResultSet in such a way that:
ResultSetResultSet after the Connection was closed (probably returned to the pool)next()wasNull() (assuming we can live with wrapping every primitive value in nullable columns)For that, we are asking our user to provide a lambda that we will execute to read each row. As we don’t want her to mess with next() we will handle her a trimmed-down ResultSet where she can only get results but not move the cursor. That class can also facilitate things when she deals with nulls and wrapper types:
public class ResultSetView {\n\n private final ResultSet resultSet;\n public ResultSetView(ResultSet resultSet) {\n this.resultSet = resultSet;\n }\n \n public String getString(int column) throws SQLException {\n return resultSet.getString(column);\n }\n \n public Boolean getBoolean(int column) throws SQLException{\n var res = resultSet.getBoolean(column);\n return resultSet.wasNull() ? null : res;\n }\n \n public boolean getBool(int column) throws SQLException{\n var res = resultSet.getBoolean(column);\n if(resultSet.wasNull()) throw new NullPointerException(\"Column \" + column + \" is null\");\n return res;\n }\n \n ...\n}\nFinally, we can close the loop:
\n\nclass ConnectionScope {\n ...\n public <A> List<A> select(String sql, RowReader<A> reader, Param... params) {\n try (var stmt = connection.prepareStatement(sql)) {\n for (int i = 0; i < params.length; i++) params[i].set(stmt, i + 1);\n try (var rs = stmt.executeQuery()) {\n var rsView = new ResultSetView(rs);\n var res = new LinkedList<A>();\n while (rs.next()) {\n res.add(reader.apply(rsView));\n }\n return Collections.unmodifiableList(res);\n }\n } catch (SQLException ex) {\n throw new RuntimeException(ex);\n }\n }\n}\n\n@FunctionalInterface\npublic interface RowReader<A> {\n A apply(ResultSetView rs) throws SQLException;\n}\nSo, same Jedi trick here: RowReader is a functional interface that declares to throw SQLException. So users of select can simply pass a lambda whose contents throw SQLException happily. Which is what our getXxx methods in ResultSetView do:
List<User> users = tx.select(\n \"select u.id, u.name, u.surname from users u \" +\n \"where active = true amd u.role = ?\",\n (rs) -> new User(rs.getString(1), rs.getString(2), rs.getString(3)),\n p(role)\n);\nOnce your users are using Param instead of PreparedStatement.setXxxx and ResultSetView instead of ResultSet, you now can extends the capabilities of Jdbc with new types. In particular, I’ve always found Jdbc’s lack of support for Instant and java.net.URI disturbing, just to mention a couple examples:
@FunctionalInterface\npublic interface Param {\n ...\n static Param p(URI value) {\n return (ps, i) -> ps.setString(i, value.toString());\n }\n static Param p(Instant value) {\n return (ps, i) -> ps.setTimestamp(i, java.sql.Timestamp.from(value));\n }\n}\n\npublic class ResultSetView {\n\t...\n\tpublic URI getURI(int column) throws SQLException {\n var res = resultSet.getString(column);\n return res == null ? null : URI.create(res);\n }\n public Instant getInstant(int column) throws SQLException {\n var res = resultSet.getTimestamp(column);\n return res == null ? null : res.toInstant();\n }\n}\nFurther than that, you can hide (by omiting them from your implementation), those types you don’t want your users to use directly. As an example, let’s say you have your issues with URL. You encourage your users to not use it by not implementing the p and getUrl methods.
Of course, I just implemented a couple simple methods that use Jdbc in some particular ways. But you can implement any other common usage of the api in order to simplify the scenarios you and your team use the most.
\n\nLet’s see an example. Let’s say your database generates a surrogate key whenever you insert a new row in a given table. Do you know how to perform a Jdbc insert and get the generated key? It was something like:
\n\npublic <K> K insertAndGetKey(String sql, RowReader<K> keyReader, Param... params) {\n try (var stmt = connection.prepareStatement(sql, RETURN_GENERATED_KEYS)) {\n for (int i = 0; i < params.length; i++) params[i].set(stmt, i + 1);\n stmt.executeUpdate();\n try(var keys = stmt.getGeneratedKeys()){\n keys.next();\n return keyReader.apply(new ResultSetView(keys));\n }\n } catch (SQLException ex) {\n throw new RuntimeException(ex);\n }\n}\nSo, one nice thing of our abstraction is that you can create a mehod for each typical use case. Once your users will use by simply providing lambdas for the parts they are interested in. It reminds me of the template pattern, if you want. But it is simpler with lambdas.
\n\nWe developed a small, lightway Jdbc helper library that:
\n\nnull while reading / writing columnsConnection, PreparedStatement or ResultSetIn our previous post we created a small Scala parser capable only of parsing empty Json arrays with possible whitespace. While doing so we designed a parser to be a function that takes an input and an index within that input and returns some result of a given type A or fails with a ParseError:
type Parser[A] = (String, Int) => Either[ParseError, A]\nWe then saw that we can create values of type Parser[A] and functions that, given some argument, return parsers:
val whitespace: Parser[Int] = ...\nval array: Parser[JsonArray] = ...\ndef string(token: String): Parser[Int] = ...\nFinally we created a couple of combinators: sequence (aka **) takes 2 parsers of type Parser[Int] (where the Int represents the position after parsing something) and it returns a parser that applies both parsers in sequence and returns the last result, the position after parsing both; and map takes a Parser[A] and a function A => B and returns a Parser[B].
That allowed us to create a parser like this:
\n\nval array: Parser[JsonArray] =\n (string(\"[\") ** whitespace ** string(\"]\")).map(_ => JsonArray(List.empty))\nIn this post we will explore the parsing of other Json elements that will give us new parser combinators.
\n\nA boolean in Json is simply either true or false. We already know how to build a Parser that parses such strings:
val jsonTrue: Parser[JsonBoolean] = \n string(\"true\").map(_ => JsonBoolean(true))\nval jsonFalse: Parser[JsonBoolean] = \n string(\"false\").map(_ => JsonBoolean(false))\nBut we want a parser of JsonBoolean that is able to parser any of these 2 values:
\n\n test(\"Parse boolean\") {\n assert(boolean(\"true\", 0) == Right(JsonBoolean(true)))\n assert(boolean(\"false\", 0) == Right(JsonBoolean(false)))\n }\n🔴
\n\nWe can implement a parser for Json booleans trying to parse true first. If that fails it will simply try to parse false at the same position. This is doable with the orElse member of Either that returns the this if its a Right or returns its argument, which is another (lasyly evaluated) Either:
val boolean: Parser[JsonBoolean] = (s, position) =>\n jsonTrue(s, i) orElse jsonFalse(s, position)\n✅ We try to parse a true and if we succeed, we call it a day. If we don’t succeed, orElse will return the result of trying to parse a false.
♻️ (refactor) In the solution above, like we did in the previous post, we are programming what the parser does by hand, dealing with the s and position parameters of parsers. But we would like to abstract those into a combinator of parsers:
extension [A](self: Parser[A])\n def |(other: Parser[A]): Parser[A] = (s, position) =>\n self(s, position) orElse other(s,position) \n\nval boolean: Parser[JsonBoolean] = jsonTrue | jsonFalse\n✅ Great!
\n\nBut what about failure? We would like to get an error that tells we were expecting either true or false:
test(\"Parse boolean failure\") {\n val input = \"notABoolean\"\n assert(boolean(input, 0) == Left(ParseError(input, 0, List(\"true\", \"false\"))))\n}\n🔴 Compilation error. Our current ParseError class can only hold one expected String and we now want a List of them. Let’s change that class and fix all the compilation issues and we are still in red:
Left(ParseError(\"notABoolean\", 0, List(\"false\"))) did not equal Left(ParseError(\"notABoolean\", 0, List(\"true\", \"false\")))\nScalaTestFailureLocation: com.agilogy.wapl.JsonParserTest at (JsonParserTest.scala:45)\nWhen it fails to parse, our solution so far only keeps the last alternative it evaluated and complains about expecting only that last option. Let’s try to fix that:
\n\nextension [A](self: Parser[A])\n def |(other: Parser[A]): Parser[A] = (s, position) =>\n self(s, position) match\n case Right(a) => Right(a)\n case Left(e1) => other(s, position) match\n case Right(a) => Right(a)\n case Left(e2) => Left(e2.copy(expected = e1.expected ++ e2.expected))\n✅
\n\nOk, I’m feeling lucky! Let’s try to parse something more difficult. What about [true, false] or the other combinations of an array of two boolean values?
test(\"Parse array of true values\") {\n assert(booleanArray(\"[true,false]\", 0) == \n Right(JsonArray(List(JsonBoolean(true), JsonBoolean(false))))\n )\n}\n🔴, of course.
\n\nLet’s implement booleanArray like we only cared about arrays of 2 booleans, for now:
val booleanArray: Parser[JsonArray] =\n string(\"[\") ** jsonBoolean ** string(\",\") ** jsonBoolean ** string(\"]\")\nThe first issue is we don’t have a sequencing combinator (**) for parsers other than Parser[Int]. Let’s try to fix that. We want to generalize ` sequence (**`) to any kind of parsers.
extension [A](self: Parser[A])\n\tinfix def sequence[B](other: Parser[B]): Parser[???] = ...\nFirst question arised. What should we return when we parse an A and then a B? It seems only natural to return the tuple of both values:
extension [A](self: Parser[A])\n infix def sequence[B](other: Parser[B]): Parser[(A,B)] = (s, position) =>\n for\n a <- self(s, position)\n b <- other(s, ???)\n yield (a,b)\nBut now we have another issue. We implemented sequence for parsers that returned the position after parsing. But we now have parsers that already return a meaningful value (a JsonBoolean) and we need to know the position after parsing so that we can combine them with other parsers in sequence.
One solution is to redefine our Parser type. We git stash save wip our changes and:
type Parser[A] = (String, Int) => Either[ParseError, (A, Int)]\n🔴 Lots of things are broken, because the new Parser type is not comptaible with the old one.
Let’s just fix them. Some examples:
\n\ndef string(token: String): Parser[Unit] = (s, position) =>\n if (s.startsWith(token, position)) Right(() -> (position + token.length))\n else Left(ParseError(s, position, List(token)))\n\nextension [A](self: Parser[A])\n def map[B](f: A => B): Parser[B] = (s, position) =>\n self(s, position).map((a, finalPosition) => f(a) -> finalPosition)\nIn particular, let’s see how sequence ends up implementedIt seems the bug that makes destructuring not possible in for comprehensions, which is an old friend of mine, is still around in Scala 3. That was not expected. Oh, it seems to be resolved… in the nightly build of upcoming Scala 3.3.:
extension[A](self: Parser[A])\n infix def sequence[B](other: Parser[B]): Parser[(A, B)] = (s, position) =>\n for\n aI0 <- self(s, position)\n (a, i0) = aI0\n bI1 <- other(s, i0)\n (b, i1) = bI1\n yield (a -> b) -> i1\nAnd our tests for successful parsings need to be rewritten to add the position after parsing:
\n\ntest(\"Parse empty array\") {\n assert(array(\"[]\", 0) == Right(JsonArray(List.empty),2))\n}\n✅ Green! ♻️ But those tests are starting to be unconvenient. All of our tests try to parse at position 0, like the end users of our parsers will do. And they tend to not care about what the final position of the parsing is. Let’s add another apply for that:
extension[A](self: Parser[A])\n def apply(s: String): Either[ParseError, A] = self(s, 0).map(_._1)\nSo our tests are now simpler:
\n\ntest(\"Parse empty array\") {\n assert(array(\"[]\") == Right(JsonArray(List.empty)))\n}\n✅ But with one concern. We expect the end user parser to fail if we encounter unexpected content after parsing a json value:
\n\ntest(\"Parse empty array failure, unexpected content\") {\n val input = \"[]wut?\"\n assert(array(input) == Left(ParseError(input, 2, List(\"end of input\"))))\n}\n🔴 Right(JsonArray(List())) did not equal Left(ParseError(\"[]wut?\", 2, List(\"end of input\")))
Let’s fix it:
\n\ndef apply(s: String): Either[ParseError, A] =\n self(s, 0) match\n case Right((_, endPosition)) if endPosition < s.length =>\n Left(ParseError(s, endPosition, List(\"end of input\")))\n case r => r.map(_._1)\n✅ Good!
\n\n♻️ But now, when a parser error says it was expecting end of input does it mean the end of the input or the string \"end of input\"? Let’s put all other string between quotes to distinguish that:
test(\"Parse empty array failure, missing ]\") {\n val input = \"[\"\n assert(array(input) == Left(ParseError(input, 1, List(\"\\\"]\\\"\"))))\n}\n\ntest(\"Parse empty array failure, unexpected content\") {\n val input = \"[]wut?\"\n assert(array(input) == Left(ParseError(input, 2, List(\"end of input\"))))\n}\nNow we can git stash pop our parser of arrays of two booleans and finish it:
val booleanArray: Parser[JsonArray] =\n (string(\"[\") ** boolean ** string(\",\") ** boolean ** string(\"]\"))\n .map { case ((((_, b1), _), b2), _) => JsonArray(List(b1, b2)) }\n✅! ♻️ But that destructured argument to map was ugly. Let’s try to fix it with some more sugar. My idea is that whenever we sequence a parser of Unit and any other parser of any type A, we don’t care about the Unit value and we just want the A, not a Pair<Unit,A>. I’ll create an extension for Parser[Unit] where sequence ignores the Unit and I’ll add a sequence version taking a Parser[Unit] that ignores that other Unit. As I’ll need the original sequence in multiple places, I define it at the root level:
def _sequence[A, B](a: Parser[A], b: Parser[B]): Parser[(A, B)] = ... \n\nextension (self: Parser[Unit])\n\n infix def sequence[B](other: Parser[B]): Parser[B] = _sequence(self, other).map(_._2)\n def **[B](other: Parser[B]): Parser[B] = _sequence(self, other).map(_._2)\n\nend extension\n\nextension [A](self: Parser[A])\n\n infix def sequence[B](other: Parser[B]): Parser[(A, B)] = _sequence(self, other)\n @targetName(\"sequenceUnit\")\n infix def sequence(other: Parser[Unit]): Parser[A] = _sequence(self, other).map(_._1)\n\n def **[B](other: Parser[B]): Parser[(A, B)] = sequence(other)\n @targetName(\"starStarUnit\")\n def **(other: Parser[Unit]): Parser[A] = sequence(other)\n\nend extension\n\nNow our array parser is:
\n\nval booleanArray: Parser[JsonArray] =\n (string(\"[\") ** boolean ** string(\",\") ** boolean ** string(\"]\"))\n .map { case (b1, b2) => JsonArray(List(b1, b2)) }\nLet’s say we want to parse arrays (of booleans) of any length, now:
\n\ntest(\"Parse array of boolean values\") {\n val length = random.between(3, 5)\n val booleans = (0 until length).map(_ => random.nextBoolean()).toList\n assert(booleanArray(s\"[${booleans.map(_.toString).mkString(\",\")}]\") ==\n Right(JsonArray(booleans.map(JsonBoolean)))\n )\n}\n🔴 After the second boolean it expects ] but we give it more booleans.
This time, we will start by solving the problem with some combinators and invent new ones whenever we need them:
\n\nstring(\"[\") **\n ((boolean ** (string(\",\") ** boolean).repeated) | empty) ** \nstring(\"]\")\nThe idea is to have [, then the array contents and then ]. The array contents may be empty or they may be a boolean followed by 0 or more times the combination of a comma and a boolean. But now we want our types to fit in this idea. Let’s see that in detail:
We already know that (string(\",\") ** boolean) is of type JsonBoolean
p.repeated is intended to represent a parser that parses whatever p parses any number of times, including 0. If p is of type A, then it will parse any number of chunks as instances of type A, giving us a list of values of type A:
extension [A](self: Parser[A])\n def repeated: Parser[List[A]] = ???\nTherefore, (string(\",\") ** boolean).repeated gives us a List[JsonBoolean]. And (boolean ** (string(\",\") ** boolean).repeated, is of type (JsonBoolean,List[JsonBoolean]). If we want all in a JsonArray, we must map that parser. The empty parser returns whatever value we want; and we want an empty JsonArray.
val booleanArray: Parser[Json] =\n string(\"[\") ** (\n (boolean ** (string(\",\") ** boolean).repeated).map { case (b, l) => JsonArray(b :: l)} | \n empty(JsonArray(List.empty))\n ) ** string(\"]\")\n![\"repetition\"](\"../assets/recursiveRepetition.gif\")
Repetition… by self-recursion! How cool is that?
\n\nNow we need to implement repeated. Here is one fancy way with just the combinators we already have and a recursive definition:
We can define a parser repeated that, when needed, tries to parse some content with repeated itself. Like a function calling itself but with parsers.
Let’s see how that turns out:
\n\ndef repeated: Parser[List[A]] = (self ** repeated).map(_ :: _) | empty(List.empty)\nIt compiles! Let’s run our tests:
\n\nAn exception or error caused a run to abort. \njava.lang.StackOverflowError\n\tat com.agilogy.wapl.Parser$package$ParserOps.repeated(Parser.scala:54)\n\tat com.agilogy.wapl.Parser$package$ParserOps.repeated(Parser.scala:54)\n🔴 Ops! When defining repeated it calls repeated. StackOverflowError.
But we may use some lazyness here. After all, sequence (or **) won’t need the second parser unless the first one succeeds. It seems only natural for it (and its variants) to evaluate that second parser lazyly:
def _sequence[A, B](a: Parser[A], b: => Parser[B]): Parser[(A, B)] = ...\n\nextension (self: Parser[Unit])\n\n infix def sequence[B](other: => Parser[B]): Parser[B] = _sequence(self, other).map(_._2)\n def **[B](other: => Parser[B]): Parser[B] = _sequence(self, other).map(_._2)\n\nend extension\n\nextension [A](self: Parser[A])\n\n infix def sequence[B](other: => Parser[B]): Parser[(A, B)] = _sequence(self, other)\n @targetName(\"sequenceUnit\")\n infix def sequence(other: => Parser[Unit]): Parser[A] = _sequence(self, other).map(_._1)\n def **[B](other: => Parser[B]): Parser[(A, B)] = _sequence(other)\n @targetName(\"starStarUnit\")\n def **(other: Parser[Unit]): Parser[A] = _sequence(other)\n\nend extension\n✅ ! But… What about long arrays. Let’s change our test to generate arrays of length random.between(10000, 10005):
An exception or error caused a run to abort. \njava.lang.StackOverflowError\n\tat scala.runtime.BoxesRunTime.boxToInteger(BoxesRunTime.java:63)\n\tat com.agilogy.wapl.Parser$package$ParserOps.map$$anonfun$1(Parser.scala:37)\n🔴 Ouch! This time the cause is not an infinite recursion, but simply one too deep. So let’s be a bit less fancy and implement repeated by hand. That is, we run a tailrec loop where we try to parse the content with the parser we want repeatedly applied. If it doesn’t parse, we stop repeating and return the accumulated list of parsed values so far (which may be empty). If it does parse, we repeat at the new position.
def repeated: Parser[List[A]] = (s, position) =>\n @tailrec\n def loop(acc: List[A], pos: Int): (List[A],Int) =\n self(s, pos) match\n case Left(_) => (acc.reverse, pos)\n case Right(a, newPos) => loop(a::acc, newPos)\n Right(loop(List.empty, position))\n✅ !
\n\n♻️ Let’s refactor a bit… We now have 2 array parsers: an array parser that only accepts whitespace and a booleanArray parser that only accepts booleans. Let’s remove the former and make the later accept whitespace by using whitespaceI should, in fact, allow whitespace in many other places, like between the \",\" and the next boolean. But let me add that to the backlog and keep going. instead of empty:
val array: Parser[Json] =\n string(\"[\") **\n ((boolean ** (string(\",\") ** boolean).repeated).map {\n case (b, l) => JsonArray(b :: l)\n } | whitespace.map(_ => JsonArray(List.empty)))\n ** string(\"]\")\n✅
\n\nLet me continue with the parsing of other primitive values. Let’s start with Json numbers. The grammar for Json numbers occupies some space of your screen but it is not complex:
\n\n![\"img\"](\"../assets/json-number.png\")
We chose to represent JsonNumber by its internal string representation instead of trying to represent it with a numeric Scala or Java value. That allows us to just focus on Json and let our users treat that number however they wantJson numbers are decimal numbers without any bound. Int, Long and any other integer primitive type won’t be able to hold values if they are out of the allowed range of values. And Float and Double types, being floating point representations, won’t be able to correctly represent some decimal values. Still, in many occasions, a Long or a Double may suffice if you know your Json use case is not amongst those that cause issues. We could use something like a BigDecimal, but I prefer to simply give our user the String and let them decide how to use them.. Now we won’t try to normalize parsed numbers, just parse them and return their string representation.
test(\"Parse json numbers\") {\n assert(number(\"1\") == Right(JsonNumber(\"1\")))\n assert(number(\"0.1\") == Right(JsonNumber(\"0.1\")))\n assert(number(\"-0.1\") == Right(JsonNumber(\"-0.1\")))\n assert(number(\"-0.1\") == Right(JsonNumber(\"-0.1\")))\n assert(number(\"-0.1e2\") == Right(JsonNumber(\"-0.1e2\")))\n assert(number(\"-0.1e+2\") == Right(JsonNumber(\"-0.1e+2\")))\n assert(number(\"-0.1e-2\") == Right(JsonNumber(\"-0.1e-2\")))\n}\n🔴 because it doesn’t compile and 🔴 once we write number to return ???.
We could parse numbers by hand with our current combinators, probably. But them being tokens, it seems easier to just use a Regex:
\n\nval number: Parser[JsonNumber] =\n regex(\"number\", \"-?([1-9][0-9]*|0)(\\\\.[0-9]+)?([eE][\\\\-+]?[0-9]+)?\".r).map(JsonNumber.apply)\nWhere:
\n\ndef regex(label: String, regex: Regex): Parser[String] = (s, position) =>\n regex.findPrefixOf(s.substring(position))\n .map(m => Right(m, position + m.length))\n .getOrElse(Left(ParseError(s, position, List(label))))\n✅ All tests green.
\n\nWhat about Json strings? The full grammar is:
string\n '\"' characters '\"'\ncharacters\n \"\"\n character characters\ncharacter\n '0020' .. '10FFFF' - '\"' - '\\'\n '\\' escape\nLet me implement just a simplified version where we don’t have escape characters. We can implement them fully later. To test valid characters I’ll test the two edge values. The char 10FFFF can be encoded in UTF-16 as 0xDBFF 0xDFFF. So:
test(\"Parse json strings\") {\n assert(string(\"\\\"\\\"\") == Right(JsonString(\"\")))\n assert(string(\"\\\"Json\\\"\") == Right(JsonString(\"Json\")))\n assert(string(\"\\\"No només ASCII\\\"\") == Right(JsonString(\"No només ASCII\")))\n assert(string(\"\\\"\\u0020\\udbff\\udfff\\\"\") == Right(JsonString(\"\\u0020\\udbff\\udfff\")))\n}\n🔴 In fact, I find I used the name string for generic parsers and now I want to use it for JsonParser. I don’t like the mess I get where I need to import the correct one, so I rename the generic parsers string to token and add a string parser to JsonParser. Now it compiles, let’s implement it so it passes the tests.
As I’m just implementing a simplified version without escape chars, I’ll just use another regex to accept any char that is not a \":
val string: Parser[JsonString] = regex(\"string\", \"\\\"[^\\\"]*\\\"\".r)\n .map(s => JsonString(s.substring(1, s.length - 1)))\n✅ Done!
\n\nNow I can close the circle and try to implement arrays of arbitrary Json values… except objects which I don’t have implemented yet.
\n\n test(\"Parse array of values and arrays\") {\n assert(array(\"[1,false,\\\"hello\\\",[true,3]]\") ==\n Right(JsonArray(List(\n JsonNumber(\"1\"),\n JsonBoolean(false),\n JsonString(\"hello\"),\n JsonArray(List(JsonBoolean(true), JsonNumber(\"3\")))\n ))))\n }\n🔴 This time it compiles, but the test fails with:
\n\nExpected :Right(...)\nActual :Left(ParseError(\"[1,false,\"hello\",[true,3]]\", 1, List(\"\"]\"\")))\nI don’t like that error message very much. That’s my fault, as I have paid no attention to error messages for many commits now. But I’m in the midle of a Red-Green-Refactor cycle, so this is not the moment. Let’s add that to a backlog. And now let’s focus on the task at hand. My approach is to finally parse any Json value we support so far:
val json: Parser[Json] = boolean | string | number | array\nAnd now array can contain json values instead of just boolean values:
val array: Parser[Json] =\n token(\"[\") ** (\n (json ** (token(\",\") ** json).repeated).map { case (b, l) => JsonArray(b :: l) } | \n whitespace(JsonArray(List.empty))\n ) ** token(\"]\")\nMy IntelliJ warns me that referring json, which I have defined after array, is a suspicious forward reference. Furthermore, the reference is circular, because array uses json and json uses array. But let’s just run the tests and see what happens…
✅ Wow! How that even worked? Remember we defined **’s second parameter to be lazyly evaluated and, therefore, json wont’t get immediately evaluated when defining array. It is no longer using a forward reference before that reference takes a value. And there is no more a problematic circular reference: when defining array it won’t use json right away, then json will get defined and, finally, by when we use the array parser, json will have been already evaluated.
♻️ Refactor? There may be many options. I’ll peek some:
\n\nMake |’s second parameter lazyly evaluated like we do in **. After all, the second argument to | will only be needed if the first argument fails to parse.
Sort the tests so all Json primitive tests go first and all array tests come later.
\nMark parsers not representing json types private: whitespace, jsonTrue, jsonFalse
Add a helper function for the usual case where we map to some value ignoring the current value ( e.g. p.map(_ => JsonBoolean(true))): p.as(JsonBoolean(true)).
And now, objects! Let’s go directly to the TDD cycle:
\n\ntest(\"Parse object\") {\n val input = \"{\\\"a\\\":23,\\\"b\\\":[{\\\"c\\\":null}]}\"\n val expected = JsonObject(Map(\n \"a\" -> JsonNumber(\"23\"),\n \"b\" -> JsonArray(List(JsonObject(Map(\"c\" -> JsonNull))))\n ))\n assert(json(input) == Right(expected))\n}\n🔴 Red, of course.
\n\nLet’s just copy the array solution but parse object members (a key, \":\" and a value) instead of just values:
val member: Parser[(String, Json)] = string.map(_.value) ** token(\":\") ** json\n\nval obj: Parser[Json] =\n token(\"{\") ** (\n (member ** (token(\",\") ** member).repeated).map { case (b, l) => JsonObject((b :: l).toMap) } | \n whitespace.as(JsonObject(Map.empty))\n ) ** token(\"}\")\n\nval json: Parser[Json] = boolean | string | number | array | obj\n🔴 What!?
\n\nExpected :Right(JsonObject(Map(\"a\" -> JsonNumber(\"23\"), \"b\" -> JsonArray(List(JsonObject(Map(\"c\" -> JsonNull)))))))\nActual :Left(ParseError(\"{\"a\":23,\"b\":[{\"c\":null}]}\", 7, List(\"\"true\"\", \"\"false\"\", \"string\", \"number\", \"\"[\"\", \"\"}\"\")))\nWhy on earth is it complaining about position 7? What is to be found there is clearly a member \"b\":[{\"c\":null}]…
After actually spending a lot of minutes trying to figure out here it is: json does not yet support nulls. That is of course my fault for using null in my test when I don’t yet support it. But, why such an ugly error message?? Let me add that to the backlog too and fix the issue now that we know what it was.
val jsonNull: Parser[JsonNull.type] = \"null\".as(JsonNull)\nval json: Parser[Json] = boolean | string | number | jsonNull | array | obj\n✅ Wa yeah!
\n\nThe current version of our parser library so far has these functions:
\n\ntype Parser[A] = (String, Int) => Either[ParseError, (A, Int)]\n\ndef token(token: String): Parser[Unit] = (s, position) => ...\ndef empty[A](value: A): Parser[A] = (_, position) => ...\ndef sequence[A, B](a: Parser[A], b: => Parser[B]): Parser[(A, B)] = \n (s, position) => ...\ndef regex(label: String, regex: Regex): Parser[String] = (s, position) => ...\n\nextension (self: Parser[Unit])\n\n infix def sequence[B](other: => Parser[B]): Parser[B] = ...\n def **[B](other: => Parser[B]): Parser[B] = ...\n\nend extension\n\nextension [A](self: Parser[A])\n\n def apply(s: String): Either[ParseError, A] = ...\n def map[B](f: A => B): Parser[B] = (s, position) => ...\n def as[B](b: B): Parser[B] = ...\n def |(other: => Parser[A]): Parser[A] = (s, position) => ...\n infix def sequence[B](other: => Parser[B]): Parser[(A, B)] = ...\n infix def sequence(other: => Parser[Unit]): Parser[A] = ...\n def **[B](other: => Parser[B]): Parser[(A, B)] = ...\n def **(other: Parser[Unit]): Parser[A] = ...\n def repeated: Parser[List[A]] = (s, position) => ...\n\nend extension\nIf we examine the functions returning Parser carefully we’ll see that most of them are defined by the implementation of the actual parser, where we receive the string and the position and return the parse result. We’ll call those, primitives. But some others, like as, are derived from the primitives; we don’t implement them by saying how they handle the string and the position to return a parse result but we derive them from some other(s) parser(s) and applying functions on them.
We can, in fact, further refactor our current solution to discover more useful derived functions.
\n\nOne of them may be useful to represent the repetition of values with some separator:
\n\ndef separatedBy(separator: Parser[Unit]): Parser[List[A]] =\n (self ** (separator ** self).repeated).map { case (h, t) => h :: t } | empty(List.empty) \nAnd now we can simplify a couple of definitions in JsonParser:
val array: Parser[Json] =\n token(\"[\") ** whitespace ** \n json.separatedBy(token(\",\")).map(JsonArray.apply) \n ** token(\"]\")\nval obj: Parser[Json] =\n token(\"{\") **\n member.separatedBy(token(\",\")).map(members => JsonObject(members.toMap))\n ** token(\"}\")\nThis being Scala (3) we could add some more sugar. Those token(\"foo\") could be replaced with their string value directly:
given Conversion[String, Parser[Unit]] with\n def apply(str: String): Parser[Unit] = token(str)\nSo now we can write parsers like these:
\n\nval array: Parser[Json] =\n \"[\" ** whitespace ** json.separatedBy(\",\").map(JsonArray.apply) ** token(\"]\")\nval obj: Parser[Json] =\n \"{\" ** member.separatedBy(\",\").map(members => JsonObject(members.toMap)) ** \"}\"\nWe have now a quite capable Json parser. Some known limitations we may want to overcome:
\n\nAfter the very limited but promising features developed in the first part of the series, we developed an almost complete Json parser using a TDD aproach. At the same time, at each “refactor” phase, we abstracted a bunch of useful parser combinator functions, therefore developing a parser combinator library, as we intended.
\n\nBeyond the (known) limitations of our Json parser, we now have a quite capable parser combinator library… although we only tested one example grammar we developed with it. And this is one of the drawbacks of this approach compared to the approach in the Red book. We now have a parser combinator library but we don’t have any tests of the library itself, but just tests of one example parser we implemented with it.
\n\nYou can see the complete source code of this post here: https://github.com/agile-jordi/writingAParserLibrary/tree/part2.
\n\nYou know I like to write libraries from scratch. I find this to be a longer but better kata, where you implement something you could actually use. And, who knows, after the zombie apocalypse, once we recover from everything else, you may need a parsers library and all the good old ones are gone forever. So, let’s survive the apocalypse by writing one ourselves.
\n\nTo do so I’ll use the perfect excuse. I want to write yet another Json parser. I want it to be reasonable fast but, above everything else, I want it to be crystal clear when it comes to report errors. At least as good as those of most Json parsers I use, or better. Because if we are going to write a library from scracth let’s make it professional grade at least in one aspect.
\n\nThis time I’ll use Scala. I’ve been blogging about Kotlin recently but I’ve been a Scala developer for years and I use both now. And I’ll be writing Scala 3 for the first time (not counting some small divertimentos), so be patient and kind if something I write is not completely idiomatic in Scala 3 or if the library we end up with does not take advantage of every single Scala 3 wonderful new feature.
\n\nOk, I’ll admit it up front. I will end up writing a parser combinator library. And it so happens that the language (Scala) and example (Json) I have chosen are the same the famous Red book uses. Why write about it, then, if you can read Paul Chiusano and Runar Bjarnason? First of all, no matter what, you must read the red book if you haven’t yetYou probably can read it again if you already have and you will probably still learn a ton of things.. But I’ll try my best at a different approach to the parser combinator library. Instead of applying algebraic design, I’ll use a classical TDD approach.
\n\n![\"scalaAndTdd\"](\"../assets/scalaAndTdd.png\")
There are a couple of points I’d like to make with this:
\n\nLet’s parse Json. For that, let’s write a Json type:
\n\nenum Json:\n case JsonString(value: String)\n case JsonNumber(value: String)\n case JsonBoolean(value: Boolean)\n case JsonNull\n case JsonArray(value: List[Json])\n case JsonObject(value: Map[String, Json])\nYou may think JsonNumber(value: String) is an odd choice. I’ll talk about this in the future. Trust me with that one. But in any case, for the matter of this series, it doesn’t matter that much if you prefer to model them as JsonNumber(value: Double) or something else.
Done. Now the parser…
\n\nLet me try to answer this question with some requirements…
\n\nLet’s start by just parsing something simple, like an empty array. Acording to json.org an array is:
\n\n![\"img\"](\"../assets/json-array.png\")
Here whitespace represents any number of whitespace characters. It may be the empty string.
So, setting aside the parsing of values inside the array, we want a function that given the String \"[ ]\" returns JsonArray(List.empty) (for any number and type of whitespace characters between the brackets).
Let’s do it TDD style:
\n\ntest(\"Parse empty array\") {\n assert(parseArray(\"[]\") == JsonArray(List.empty))\n}\n🔴 Red. Easy peasy:
\n\nobject JsonParser:\n def parseArray(s: String): JsonArray = JsonArray(List.empty)\n✅ Green!
\n\nLet’s delay the implementation of whitespace. What else do we need? Here is what: What do we want the parser to do if feeded only with \"[\"? We expect it to fail. The failure should tell us where it failed and what was expected. That is, we failed at the second character (s[1]) where we expected ] but we found the end of the string.
We can deal with errors with exceptions, with Try, with Either or with a custom type. Let’s try using Either:
test(\"Parse empty array\") {\n assert(parseArray(\"[]\") == Right(JsonArray(List.empty)))\n}\n\ntest(\"Parse empty array failure\") {\n val input = \"[\"\n assert(parseArray(input) == Left(ParseError(input, 1, expected = \"]\")))\n}\n🔴 It does not even compile. Let’s create the ParseError class and change the parseArray signature:
case class ParseError(input: String, position: Int, expected: String)\n\nobject JsonParser:\n def parseArray(s: String): Either[ParseError, JsonArray] = Right(JsonArray(List.empty))\n🔴 It now compiles but…
\n\nRight(JsonArray(List())) did not equal Left(ParseError(\"[\", 1, \"]\"))\nScalaTestFailureLocation: com.agilogy.wapl.JsonParserTest at (JsonParserTest.scala:17)\nExpected :Left(ParseError(\"[\", 1, \"]\"))\nActual :Right(JsonArray(List()))\nNow what? We could DTSTTCPW and just implement parseArray to return this particular error when the input string is \"[\". But this is not a bad TDD training Just in case: I’m not saying TDD is bad at all, of course. Neither I’m saying training people in TDD with really small steps is bad. I’m just making a joke about confusing baby steps with implementing each test case with an if/else branch. Even though I’m making a bit of a pun to code without designing, which should be the opposite of TDD.. So let’s think a bit and design a solution.
Looking at the json.org diagram above, we see [ and ] are different nodes in the grammar chart. Maybe our mission of parsing just the empty array was more than we can chew now, afer all. Let’s try something simpler. Let’s try to parse the [ token. So let’s git stash save wip or comment out our failing test We are doing TDD right, no? And we are in the midle of a red. So let’s go back to green, make an increment, and we will retake what we where doing when in green again.and start anew. Parsing a token, when it succeeds, can only do so in one way. When succeeding, therefore, we only need the Unit value as a return:
test(\"Parse start array token\") {\n assert(parseStartArrayToken(\"[\") == Right(()))\n}\n\ntest(\"Parse start array token failure\") {\n val input = \"notTheStartArrayToken\"\n assert(parseStartArrayToken(input) == Left(ParseError(input, 0, expected = \"[\")))\n}\n🔴 Does not compile. Let’s go directly to green:
\n\ndef parseStartArrayToken(s: String): Either[ParseError, Unit] =\n\tif (s == \"[\") Right(()) else Left(ParseError(s, 0, \"[\"))\n✅ Now refactor ♻️. You see, I’m going to try to parse the ] token. So it seems we are going to parse tokens in general. Let’s refactor that:
def parseToken(s: String, token: String): Either[ParseError, Unit] =\n if (s == token) Right(()) else Left(ParseError(s, 0, token))\nMmmm… 🤔 I prefer the token to be the first argument… and the string to be a second argument list:
\n\ndef parseToken(token: String)(s: String): Either[ParseError, Unit] = ...\nAnd our tests are now:
\n\ntest(\"Parse token\") {\n assert(parseToken(\"[\")(\"[\") == Right(()))\n}\n\ntest(\"Parse token failure\") {\n val input = \"notTheStartArrayToken\"\n assert(parseToken(\"[\")(input) == Left(ParseError(input, 0, expected = \"[\")))\n}\nGreat!
\n\nNow we want to parse the token [ and then the token ]. That’s the question. And here is a simple idea: we want our token parser to be able to parse any string that starts with that token:
def parseToken(token: String)(s: String): Either[ParseError, Unit] =\n if (s.startsWith(token)) Right(()) else Left(ParseError(s, 0, token))\nAfter parsing [ in \"[]\" we now know that the string starts with [ and that we can continue to parse ]… after the [. I could play with our current design like this:
def parseArray(s: String): Either[ParseError, JsonArray] =\n parseToken(\"[\")(s) match\n case Right(()) =>\n parseToken(\"]\")(s.substring(1)) match {\n case Right(()) => Right(JsonArray(List.empty))\n case Left(e) => Left(e)\n }\n case Left(e) => Left(e)\nOr, monadically idiomatically:
def parseArray(s: String): Either[ParseError, JsonArray] =\n for\n _ <- parseToken(\"[\")(s)\n _ <- parseToken(\"]\")(s.substring(1))\n yield JsonArray(List.empty)\n✅! But if you now git stash pop or uncomment our \"Parse empty array failure\" test:
test(\"Parse empty array failure\") {\n val input = \"[\"\n assert(parseArray(input) == Left(ParseError(input, 1, expected = \"]\")))\n}\n🔴 The test fails:
\n\nLeft(ParseError(\"\", 0, \"]\")) did not equal Left(ParseError(\"[\", 1, \"]\"))\nScalaTestFailureLocation: com.agilogy.wapl.JsonParserTest at (JsonParserTest.scala:17)\nExpected :Left(ParseError(\"[\", 1, \"]\"))\nActual :Left(ParseError(\"\", 0, \"]\"))\nSo, the parser correctly fails to parse, but it is telling us that it expected \"]\" at position 0 (where we have \"[\") instead of telling us it expected it at position 1. And it tells us the string that failed to compile is \"\" when it should complain about the actual string it got: \"[\". Why? As we passed the substring(1) to the second parseToken, the input and position values of the resulting error are wrong. We want the error to contain the entire string and the position to be a position of that string, so we will need them in our parseArray function. And once we have the original input and the position we want to parse at, we no longer need the substring. We can simply check whether the string starts with the token at that index:Yes, there is a version of startsWith that takes an index. No, I didn’t know either.
def parseToken(token: String)(s: String, position: Int): Either[ParseError, Unit] =\n if (s.startsWith(token, position)) Right(()) else Left(ParseError(s, position, token))\nSo now:
\ndef parseArray(s: String): Either[ParseError, JsonArray] =\n for\n _ <- parseToken(\"[\")(s, 0)\n _ <- parseToken(\"]\")(s, 1)\n yield JsonArray(List.empty)\nAnd after fixing the compilation errors of adding a new parameter… ✅ Green!
\n\n🤔 Do you remember what this section was about? The question was: “What is a parser”? Looking at our parseToken we have one possible answerMind the gap. We will refine this answer later in the series. now: A parser is a function that given a String input and a position in that string either returns a parsed value (like the empty array above) or a parse failure:
type Parser[A] = (String, Int) => Either[ParseError, A]\n♻️ We ended our previous TDD cycle with green. A refactor is due. But what are we going to refactor? 🤔
\n\nWith that definition you don’t even need to squint too hard to see that parseToken can be seen as a function that given a String (the token) returns a parser:
def parseToken(token: String): Parser[Unit] = (s, position) =>\n if (s.startsWith(token, position)) Right(()) else Left(ParseError(s, position, token))\nAs parseToken it is no longer parsing any token but returning a parser of strings, maybe we better call it string:
def string(token: String): Parser[Unit] = (s, position) =>\n if (s.startsWith(token, position)) Right(()) else Left(ParseError(s, position, token))\nNow, why not make our parseArray a parser too?
def parseArray(): Parser[JsonArray] = (s, position) =>\n for\n _ <- string(\"[\")(s, position)\n _ <- string(\"]\")(s, position + 1)\n yield JsonArray(List.empty)\nAnd as it is not taking arguments, why not make it a value?:
\n\nobject JsonParser:\n val array: Parser[JsonArray] = (s, position) =>\n for\n _ <- string(\"[\")(s, position)\n _ <- string(\"]\")(s, position + 1)\n yield JsonArray(List.empty)\n\nAnd our tests:
\n\ntest(\"Parse empty array\") {\n assert(array(\"[]\", 0) == Right(JsonArray(List.empty)))\n}\n\ntest(\"Parse empty array failure\") {\n val input = \"[\"\n assert(array(input, 0) == Left(ParseError(input, 1, expected = \"]\")))\n}\nLet’s now imagine you want to accept the next kind of empty arrays. Those with some whitespace within.
\n\nprivate val random = new Random()\n\ntest(\"Parse empty array with whitespace\") {\n val ws = \" \" * random.between(1, 5)\n assert(array(s\"[$ws]\", 0) == Right(JsonArray(List.empty)))\n}\n🔴:
\n\nLeft(ParseError(\"[ ]\", 1, \"]\")) did not equal Right(JsonArray(List()))\nScalaTestFailureLocation: com.agilogy.wapl.JsonParserTest at (JsonParserTest.scala:21)\nExpected :Right(JsonArray(List()))\nActual :Left(ParseError(\"[ ]\", 1, \"]\"))\nWe will need a parser for whitespace.
![\"json-whitespace\"](\"../assets/json-whitespace.png\")
Acording to the Json grammar this should parse 0 or more whitespace characters. It can’t fail. If our string starts with whitespace in the given index, it succeeds. If it doesn’t it also succeeds. So we don’t even need tests for that one. Which is already suspicious. We may be missing something….
\n\nval whitespace: Parser[Unit] = (_, _) => Right(())\nLet’s try now to rewrite our array parser:
\n\nval array: Parser[JsonArray] = (s, position) =>\n for\n _ <- string(\"[\")(s, position)\n _ <- whitespace(s, position + 1)\n _ <- string(\"]\")(s, ???)\n yield JsonArray(List.empty)\nAnd here it is. The bit we were missing. After parsing some whitespace we need to know how many whitespace characters there were. We could solve that one ad hoc by stating that whitespace is a Parser[Int], which returns the number of whitespace characters it found:
private val whiteSpaceChars = Set(' ', '\\n', '\\r', '\\t')\n\nval whitespace: Parser[Int] = (s, position) =>\n Right(s.substring(position).takeWhile(c => whiteSpaceChars.contains(c)).length)\n\nval array: Parser[JsonArray] = (s, position) =>\n for\n _ <- string(\"[\")(s, position)\n whiteSpaceCount <- whitespace(s, position + 1)\n _ <- string(\"]\")(s, position + 1 + whiteSpaceCount)\n yield JsonArray(List.empty)\n✅!
\n\n♻️ I start to be fed up with the position arithmetics in our array parser. In the second step of our for comprehension we pass position + 1 because what the first step parsed occupies 1 character. Then in the third step we pass position + 1 + whitesSpaceCount because the previous 2 steps parsed 1 + whitesSpaceCount characters. Every step needs to add the number of parsed characters by the previous steps. It is not nice. What about our steps all returned the position at which they stopped parsing?
val whitespace: Parser[Int] = (s, position) =>\n Right(position + s.substring(position).takeWhile(c => whiteSpaceChars.contains(c)).length)\n\nval array: Parser[JsonArray] = (s, position) =>\n for\n i0 <- string(\"[\")(s, position)\n i1 <- whitespace(s, i0)\n _ <- string(\"]\")(s, i1)\n yield JsonArray(List.empty)\n\ndef string(token: String): Parser[Int] = (s, position) =>\n if (s.startsWith(token, position)) Right(position + token.length) \n\telse Left(ParseError(s, position, token))\nMuch better! Now I need to remember which one of the i{n} variables to pass, but I don’t need to be calculating the next position to pass by hand. And after some adjusting of the expected result of parsing in the tests we are still green.
And now, try to honor this section name It seems to be a recurrent theme. I name a section “X” and then I spend lines and more lines without any mention to “X”, until I remember what I was talking about. Sorry not sorry. 😙. Our array parser is combining the parseToken(\"[\"), whitespace and parseToken(\"]\") parsers by hand. That is, it is implementing a parser that, given the string and the position, uses those parsers. But we could just build the new parser by combining the existing parsers before we actually run the parser. Let’s first forget about the whitespace for a moment:
val array: Parser[JsonArray] = (s, position) =>\n for\n i0 <- string(\"[\")(s, position)\n _ <- string(\"]\")(s, i0)\n yield JsonArray(List.empty)\nI’m proposing a function that given 2 parsers (namely string(\"[\") and string(\"]\") ) returns a parser that internally does what the code above does:
def sequence(a: Parser[Int], b: Parser[Int]): Parser[Int] = ???\nAs all our parsers here return the position after parsing what they parse, I’m assuming we want to sequence this kind of parsers. And I’m assuming we want to return the position after parsing the combination:
\n\ndef sequence(a: Parser[Int], b: Parser[Int]): Parser[Int] = (s, position) =>\n for\n ia <- a(s, position)\n ib <- b(s, ia)\n yield ib\nAnd now, our simplified array parser without whitespaces is:
val array: Parser[JsonArray] = \n sequence(string(\"[\"), string(\"]\"))\n🔴 Ops! That does not compile. We have a parser that returns Int but we need a JsonArray. So let’s fix that:
val array: Parser[JsonArray] = (s, position) =>\n for\n _ <- sequence(string(\"[\"), string(\"]\"))(s, position)\n yield JsonArray(List.empty)\nOr, if you prefer:
\n\nval array: Parser[JsonArray] = (s, position) =>\n sequence(string(\"[\"), string(\"]\"))(s, position)\n\t\t.map(_ => JsonArray(List.empty))\n✅ (except our whitespace test, of course)
\n\nBut before recovering our whitespace, let’s dig a bit deeper in this idea. We tried to combine the parsers to get a parser before actually running the resulting parser. Let’s say we apply the same idea to that .map. We may want to be able to transform the parser we got by combining parseToken(\"[\") and parseToken(\"]\"), which was a Parser[Int] by transforming it into a Parser[JsonArray]. We need a function… let’s call it map, of course:
def map[A,B](p: Parser[A])(f: A => B): Parser[B] = (s, position) =>\n p(s, position).map(f)\nAnd now:
\n\nval array: Parser[JsonArray] =\n map(sequence(string(\"[\"), string(\"]\")))(_ => JsonArray(List.empty))\n✅ (again, without the whitespace stuff)
\n\nThis is Scala. Let’s add some sugar.
\n\nimplicit class ParserOps[A](self: Parser[A]):\n def map[B](f: A => B): Parser[B] = (s, position) =>\n self(s, position).map(f) \n\nval array: Parser[JsonArray] =\n sequence(string(\"[\"), string(\"]\")).map(_ => JsonArray(List.empty))\nOps! That was the Scala 2 way of having extension methods. It all changed in Scala 3 and I didn’t remember In fact, if you were to look at the companion repository in github and followed the commits along, I realized about that while finishing the writing of the second part of this series. I prefer to correct it here, though, to avoid the confusion of mixing Scala 2 and Scala 3 features.. So:
\n\nextension [A](self: Parser[A])\n def map[B](f: A => B): Parser[B] = (s, position) =>\n self(s, position).map(f)\nMore sugar!
\n\nextension(self: Parser[Int])\n infix def sequence(other: Parser[Int]): Parser[Int] = (s, position) =>\n for\n ia <- self(s, position)\n ib <- other(s, ia)\n yield ib\n\nval array: Parser[JsonArray] =\n (string(\"[\") sequence string(\"]\")).map(_ => JsonArray(List.empty))\nOr, if you prefer operators:
\n\nextension(self: Parser[Int])\n\n infix def sequence(other: Parser[Int]): Parser[Int] = ...\n def **(other: Parser[Int]): Parser[Int] = sequence(other)\n\nend extension\n\nval array: Parser[JsonArray] =\n (string(\"[\") ** string(\"]\")).map(_ => JsonArray(List.empty))\nAnd now, let’s recover our whitespace parsing:
\n\nval array: Parser[JsonArray] =\n (string(\"[\") ** whitespace ** string(\"]\"))\n .map(_ => JsonArray(List.empty))\n✅✅ Everything green again. Including our JsonArray with whitespace test.
We applied TDD to implement a parser capable of parsing… just empty Json arrays with or without whitespace in between. Even though their capability is very limited, this exercise served the purpose of defining our design for a parser type Parser[A] = (String, Int) => Either[ParseError, A]. Then we discovered that instead of having functions that happen to parse what we want, we can haver functions that return parsers and values of type parser. Finally we found a way of combining 2 or more of these parsers to build a new parser without actually programming what the parser does when it receives the String and Int arguments, but designing how it is a sequence of 2 other parsers.
You can see the companion repository to this series at https://github.com/agile-jordi/writingAParserLibrary/tree/part1.
\n\nIn our previous posts we developed a minimal property based testing (PBT) library capable of checking properties on primitive types, nullable types, tuples and types mapped from other types, giving us the possibility of testing properties of our programs and functions for many kinds of input values.
\n\nOne of the (many) shortcoming of this small thing we created is in the reproducibility of failed tests. If you are checking a property for simple values like forAny(Arb.int) { i -> i + i >= i } the outcome is quite informative:
Property failed at attempt 3 with sample -836667656\ncom.agilogy.wapbtl.PropertyFailedException: Property failed at attempt 3 with sample -836667656\n\tat app//com.agilogy.wapbtl.ForAnyKt.forAny$test(forAny.kt:14)\n\tat app//com.agilogy.wapbtl.ForAnyKt.forAny(forAny.kt:18)\n\tat app//com.agilogy.wapbtl.ForAnyKt.forAny$default(forAny.kt:9)\nWe have the sample that made our property fail. If we want to reproduce the failed test, we just need to write an example based test with that example. But… What if our sample is a more or less complex datatype whose toString() is not suitable for a copy & paste into Kotlin code? What if the test is not a one-liner and “simply” writing an example based test involves copying an important amount of lines?
What I would like is to be able to change a single line of code and, hop!, make our property based test just test the exact sample for which I got a failure.
\n\nGrab your dice instance of Random and let’s dig into how randomness works…
![\"Roll](\"https://media1.giphy.com/media/VGoZVlR9naOZCiRLSy/giphy.gif\")
As you probably already know, random values generation by computers is actually quite a complex topic. Long story short, Kotlin’s Random generates random values using an, in fact, predictable function that takes a seed and returns an endless sequence of aparently random numbers. Give it the same seed and poof! randomness is gone: you’ll get the same sequence of values. This would be dangerous if you were using pseudo-randomness for cryptogrhaphic purposes, but it is really useful for us.
So, what we need is to be able to seed our tests whenever we want to reproduce a failure.
\n\nLet’s do this TDD style. That is, let’s write the test first so that we design the feature by thinking how we would like to use it instead of thinking about how to implement it:
\n\n@Test\nfun forAnySeedParameterShouldReproduceTheFailingTest() {\n val property: (Int) -> Boolean = { i -> i + i >= i }\n val failure = assertFailsWith<PropertyFailedException> {\n forAny(Arb.int, property = property)\n }\n var actualExecutionsWithSeed = 0\n val seededFailure = assertFailsWith<PropertyFailedException> {\n forAny(Arb.int, failure.seed) {\n actualExecutionsWithSeed++\n property(it)\n }\n }\n assertEquals(failure.sample, seededFailure.sample)\n assertEquals(failure.seed, seededFailure.seed)\n assertEquals(1, seededFailure.attempt)\n assertEquals(1, actualExecutionsWithSeed) \n}\nSo, whenever we get a PropertyFailedException we want to get a seed. And, forAny now will take an optional seed as an argument and will give us some guarantees:
So now our forAny functions will have a signature like this one (here the one for one Arb argument):
fun <A> forAny(r: Arb<A>, seed: Long? = null, property: (A) -> Boolean): Unit\nAnd in case of failure it will throw:
\n\nclass PropertyFailedException(val attempt: Int, val sample: Any?, val seed: Long) : Exception(\n \"Property failed at attempt $attempt with seed $seed, sample $sample\"\n)\nOk, now let’s go for the green.
\n\nOur Arbitrary instances are currently using kotlin.Random like this:
val Arb.Companion.int: Arb<Int> get() = object : Arb<Int> {\n override fun generate(): Int = kotlin.random.Random.nextInt()\n}\nWhat does kotlin.random.Random.nextInt() do? If we look under the hood at the implementation of the standard library, we’ll find:
package kotlin.random\npublic abstract class Random {\n // ...\n companion object Default : Random(), Serializable {\n // ...\n private val defaultRandom: Random = defaultPlatformRandom()\n override fun nextInt(): Int = defaultRandom.nextInt()\n }\nThere is an important amount of things going on here:
\n\ndefaultRandom is createdRandom.nextInt() it uses defaultRandom to generate an IntI didn’t know, but companion objects can have a name. Don’t let that distract you. If the companion object of Random has a method nextInt() then you can use it as Random.nextInt() no matter what the name of the companion object is . See the kotlin docs.Int value is returned each time, some kind of state must have also changed in defaultInt each time we invoke nextInt().Initially, I naively assumed I could get the “current seed” out of such state and, therefore, implement our tests like this:
\n\nval seed = Random.getCurrentSeed() // <-- invented function, does not exist\nval sample = r.generate()\nval result = property(sample)\nif (!result)\n throw PropertyFailedException(attemptNumber, seed, sample)\n\nUnfortunately, I found no such function. Furthermore, like the name suggests, defaultPlantformRandom() is platform specific and you can’t reason about the implementation used unless you throw the current platform (JVM, Javascript or Native) into the discussion.
But we have a function in the standard library to get a Random instance with a given seed:
public fun Random(seed: Long): Random = XorWowRandom(seed.toInt(), seed.shr(32).toInt())\nThis time, for some reason, the returned type is not dependant on the platform, but we get an XorWowRandom instance no matter the plantform. Again, looking inside it we can actually see its mutable state:
internal class XorWowRandom internal constructor(\n private var x: Int,\n private var y: Int,\n private var z: Int,\n private var w: Int,\n private var v: Int,\n private var addend: Int\n) : Random() // ...\nSo, for each tests execution I need to create an instance of Random from a known seed, make our generators use that instance of Random and, in case of failure, inform about the seed I used. How do we generate such a seed? We could use the current time in milliseconds or we can simply use… Random (the default instance):
val seed = Random.nextLong()\nval random = Random(seed)\nval sample = r.generate(random)\nval result = property(sample)\nif (!result)\n throw PropertyFailedException(attemptNumber, seed, sample)\nSo our Arbitrary interface is now:
interface Arb<A> {\n fun generate(random: Random): A\n companion object\n}\nAnd for primitives we simply use the Random argument instead of the platform default Random:
val Arb.Companion.int: Arb<Int> get() = object : Arb<Int> {\n override fun generate(random: Random): Int = random.nextInt()\n}\nBut what about combinators? Let’s look at how product2 ens up:
fun <A, B, Z> Arb.Companion.product2(\n a: Arb<A>, b: Arb<B>, f: (A, B) -> Z\n): Arb<Z> =\n object : Arb<Z> {\n override fun generate(random: Random): Z = f(a.generate(random), b.generate(random))\n }\nAlthough aparently simple, there are some interesting implications in this design. As random is mutable (like we already saw), this combinator uses it twice, mutating its state twice. And that is exactly what we want: given a seed, the combinator will generate always the same pair of values. And this is valid for any of our other combinators. Let’s see some of them:
fun <A, B, C, Z> Arb.Companion.product3(\n a: Arb<A>, b: Arb<B>, c: Arb<C>, f: (A, B, C) -> Z\n): Arb<Z> =\n object : Arb<Z> {\n override fun generate(random: Random): Z = f(a.generate(random), b.generate(random), c.generate(random))\n }\n\nfun <A, B> Arb<A>.map(f: (A) -> B): Arb<B> = object: Arb<B>{\n override fun generate(random: Random): B = f(this@map.generate(random))\n}\nFinally, we need our forAny function to accept an optional seed argument. If such a seed is provided it should just run one test with the samples generated by such seed:
fun <A> forAny(r: Arb<A>, seed: Long? = null, property: (A) -> Boolean) {\n if (seed == null) {\n (1..100).forEach { attemptNumber ->\n val currentSeed = Random.nextLong()\n val sample = r.generate(Random(currentSeed))\n val result = property(sample)\n if (!result)\n throw PropertyFailedException(attemptNumber, currentSeed, sample)\n }\n } else {\n val sample = r.generate(Random(seed))\n val result = property(sample)\n if (!result)\n throw PropertyFailedException(1, seed, sample)\n }\n\n}\nI can improve many things but, at least, let’s get rid of the code repetition in forAny I used a function definition inside the forAny function, which may be not that familiar to some readers. I like functions insde functions because they don’t bloat the namespace of your class and they don’t require visibility modifiers. I know I don’t need to look at them when reasoning about any other function that the one they are defined in. But if you don’t like them, simply define a new private function in your class and call it a day.:
fun <A> forAny(r: Arb<A>, seed: Long? = null, property: (A) -> Boolean,) {\n fun test(currentSeed: Long, attemptNumber: Int) {\n val sample = r.generate(Random(currentSeed))\n val result = property(sample)\n if (!result)\n throw PropertyFailedException(attemptNumber, currentSeed, sample)\n }\n if (seed == null) {\n (1..100).forEach { attemptNumber ->\n test(Random.nextLong(), attemptNumber)\n }\n } else {\n test(seed, 1)\n }\n}\nSo, that’s it! Now, whenever you get a test failure, if you want to run it again after some bug fixing attempt or you need to debug it, you simply look for the seed returned by the error message (let’s say its 123456L) and replace forAny(arb, property) with:
forAny(arb, 123456L, property)\nYou can see the current version of our library at https://github.com/agile-jordi/wapbtl/tree/part3.
\n\nIn our previous post we developed a minimal Property Based Testing library in Kotlin that was capable of testing properties of Int values. Amongst its limitations, I find its inhability to test properties on other types disturbing. So let’s add some more Arb values to our beloved library.
We could start by adding Arb instances for some types for which kotlin.random.Random already knows how to generate random valuesWhat about Char and String, aren’t they simple? They aren’t. Not if you work outside the ASCII limits. Which I do. Because I speak catalan, and we have things like è or ç or even l·l… and I work in projects that use asiatic languages which for us, poor latin alphabet users, are a complex fantasy.:
val Arb.Companion.long: Arb<Long> get() = object : Arb<Long> {\n override fun generate(): Long = kotlin.random.Random.nextLong()\n}\n\nval Arb.Companion.float: Arb<Float> get() = object : Arb<Float>{\n override fun generate(): Float = kotlin.random.Random.nextFloat()\n}\n\nval Arb.Companion.double: Arb<Double> get() = object : Arb<Double>{\n override fun generate(): Double = kotlin.random.Random.nextDouble()\n}\n\nval Arb.Companion.boolean: Arb<Boolean> get() = object : Arb<Boolean>{\n override fun generate(): Boolean = kotlin.random.Random.nextBoolean()\n}\nStudying kotlin.random.Random we can see it is also easy to generate random values within a range. So, let’s add that:
fun Arb.Companion.int(range: IntRange): Arb<Int> = object : Arb<Int> {\n override fun generate(): Int = kotlin.random.Random.nextInt(range)\n}\nAnd so on…
\n\nWhat about nullable types like Int? or Float?. We could build a nullable version of each Arb we have, but that would be tedious and repetitive. What we would like is to be able to convert an Arb<A> to an Arb<A?> for any given A .
Let’s try it! We can even adjust the probability we want to get a null value: We parameterize the type as <A: Any> so that you can’t use orNull on already nullable types. That will avoid confusion, as applying multiple times orNull would exagerate the probability of null values being generated.
fun <A : Any> Arb<A>.orNull(nullProbability: Double = 0.5): Arb<A?> =\n object : Arb<A?> {\n override fun generate(): A? =\n if (Random.nextDouble(0.0, 1.0) <= nullProbability) null\n else this@orNull.generate()\n}\nLet’s say I want to check a typical property, like the commutativity of the sum of integers:
\n\n\n\n\nFor any pair of ints a, b: a + b = b + a
\n
In this example I need arbitrary values of type Pair<Int, Int>. Again, instead of thinking about a specific solution every time we need one, let’s generalize that to arbitrary values that are tuples of other types:
fun <A, B> Arb.Companion.pair(a: Arb<A>, b: Arb<B>): Arb<Pair<A, B>> =\n object : Arb<Pair<A, B>> {\n override fun generate(): Pair<A, B> = a.generate() to b.generate()\n }\n\nfun <A, B, C> Arb.Companion.triple(\n a: Arb<A>, b: Arb<B>, c: Arb<C>\n): Arb<Triple<A, B, C>> =\n object : Arb<Triple<A, B, C>> {\n override fun generate(): Triple<A, B, C> =\n Triple(a.generate(), b.generate(), c.generate())\n }\nOf course, in the Kotlin tradition of not having tuples of arity greater than 3, we can add versions of this idea for whatever arity as long as we ask for a function to build the result:
\n\nfun <A, B, Z> Arb.Companion.product2(\n a: Arb<A>, b: Arb<B>, f: (A, B) -> Z\n): Arb<Z> =\n object : Arb<Z> {\n override fun generate(): Z = f(a.generate(), b.generate())\n }\n\nfun <A, B, C, Z> Arb.Companion.product3(\n a: Arb<A>, b: Arb<B>, c: Arb<C>, f: (A, B, C) -> Z\n): Arb<Z> =\n object : Arb<Z> {\n override fun generate(): Z = \n f(a.generate(), b.generate(), c.generate())\n }\n\nfun <A, B, C, D, Z> Arb.Companion.product4(\n a: Arb<A>, b: Arb<B>, c: Arb<C>, d: Arb<D>, f: (A, B, C, D) -> Z\n): Arb<Z> =\n object : Arb<Z> {\n override fun generate(): Z = \n f(a.generate(), b.generate(), c.generate(), d.generate())\n }\nAnd now we can see pair and triple as simple helpers over product2 and product3:
fun <A, B> Arb.Companion.pair(a: Arb<A>, b: Arb<B>): Arb<Pair<A, B>> =\n product2(a, b, ::Pair)\n\nfun <A, B, C> Arb.Companion.triple(\n a: Arb<A>, b: Arb<B>, c: Arb<C>\n): Arb<Triple<A, B, C>> =\n product3(a, b, c, ::Triple)\nNow we can test our commutativity:
\n\n@Test\nfun testSumCommutativity() {\n forAny(Arb.pair(Arb.int, Arb.int)){ (a, b) -> a + b == b + a }\n}\nLooking at this example, it seems interesting to make forAny accept more than one Arb so that our user doesn’t need to build a tuple to test properties over a number of values:
fun <A, B> forAny(a: Arb<A>, b: Arb<B>, property: (A, B) -> Boolean) =\n forAny(Arb.pair(a, b)){ (a, b) -> property(a,b) }\nUnfortunately this approach builds an unnecessary Pair and is limited to arities 2 and 3. Let’s try a different approach:
fun <A, B> forAny(a: Arb<A>, b: Arb<B>, property: (A, B) -> Boolean) =\n forAny(Arb.product2(a, b, property)){ it }\n\nfun <A, B, C> forAny(a: Arb<A>, b: Arb<B>, c: Arb<C>, property: (A, B, C) -> Boolean) =\n forAny(Arb.product3(a, b, c, property)){ it }\nWait! What did I just do here? I must admit it. I actually wrote these articles some weeks ago and now, while proofreading it before publishing, I couldn’t initially remember how this works. So let’s try to explain it a bit.
\n\nThe trick is using property as the function argument of productN. That way, the productN function directly builds the result of evaluating the property. Hence, we expect, forAny such evaluated property, to simply be true. It’s like:
fun <A, B> forAny(a: Arb<A>, b: Arb<B>, property: (A, B) -> Boolean) =\n forAny(Arb.product2(a, b, property)){ propertyEvaluationResult -> propertyEvaluationResult == true }\n\nThe first version I showed simply uses the implicit it parameter name and avoids the ugly it == true expression.
Now we can use any arity we have a productN for and we will not be building an intermediate tuple just to get its parts when we need to actually evaluate the property.
Finally, our property would be:
\n\n@Test\nfun testSumCommutativity() {\n forAny(Arb.int, Arb.int){ a, b -> a + b == b + a }\n}\nNow let’s say you have a nice Enum representing some important business type:
enum class DayOfWeek{\n Mon, Tue, Wed, Thu, Fri, Sat, Sun;\n fun next(days: Int): DayOfWeek = ...\n}\nAnd let’s imagine we want to check some properties involving days of week. We may want to check that days of the week repeat after 7 days. Or we may want to check that we know how to write the day of the week in a String and read it afterwards. Whatever the property, we need an Arb<DayOfWeek>.
We can build one by generating a random number between 0 and 6 and getting the enum value in that position:
\n\nfun Arb.Companion.dayOfWeek() = object : Arb<DayOfWeek> {\n override fun generate(): DayOfWeek {\n val i = kotlin.random.Random.nextInt(0..6)\n return DayOfWeek.values()[i]\n }\n}\nBut it is a shame we repeated the generation of random integers. We already implemented that. Let’s say we want to be able to apply DayOfWeek.values()[i] to whatever i is generated by Arb.int. Does that sound familiar? Not yet? Ok, let’s call it map:
![\"scary-monad\"](\"../assets/scary-monad.gif\")
}\nWait! What was that???
fun <A, B> Arb<A>.map(f: (A) -> B): Arb<B> = object: Arb<B>{\n override fun generate(): B = f(this@map.generate())\n}\nNow, we can rewrite our dayOfWeek Arb like this:
fun Arb.Companion.dayOfWeek() = Arb.int(0..6).map { DayOfWeek.values()[it] }\nIn fact, we can generalize further by providing a way to generate arbitrary values of any Enum. The trick is the function enumValues<E> that Kotlin provides. If you invoke it (with a type parameter which is an Enum) it will return all the values of the Enum. If you try, IntelliJ will complain that your function must be inline and your type param reifiedIf this sounds alien to you, you can learn more about inline functions and reified parameters in the excellent Kotlin documentation.. Just Alt+Enter it!
inline fun <reified E : Enum<E>> Arb.Companion.enum(): Arb<E> {\n val values = enumValues<E>()\n return Arb.int(values.indices).map { values[it] }\n}\nDid you see what I did? We are now equipped with:
\nArbitrary instances for primitive typesorNull() to go from Arbitrary<A> to Arbitrary<A?>map to go from Arbitrary<A> to Arbitrary<B> given a function (A) -> Bproduct2, product3, product4, etc. to build an instance of Arbitrary from several instances of (possibly) different types.Armed with all these tools, we can now generate many of the instances we need. In particular, we can now generate an Arbitrary for any data class as long as we have the needed primitive instances I’m aware we have unsigned integers in Kotlin, but I wanted an example of map being used and that was what I got.:
data class Natural(val value: Int)\ndata class Coordinates(val x: Int, val y: Int)\ndata class Circle(val center: Coordinates, val radius: Int)\nval distanceArb = Arb.int(0..10).map{ Natural(it) }\nval coordinateArb = Arb.int(-10..10)\nval coordinatesArb = Arb.product2(coordinateArb, coordinateArb) {\n x, y -> Coordinates(x, y) \n}\nval circleArb = Arb.product2(coordinatesArb, distanceArb) { \n coords, radius -> Circle(coords, radius) \n}\nHow cool is that!!?? 😎 Now, if you know about ADTs you know we have product types… but we are missing sum types. This one is long enough, so let’s add those to our backlog.
\n\nWe added values and functions to get Arb instances of several types. Beyond primitive Arb types we built by hand (like Boolean, Int, Double and so on), we started creating Arb combinators, functions that take Arb instances and maybe other parameters and return new Arb instances.
In particular, we first created a combinator orNull() that given an Arb<A> (where A is not nullable) returns an Arb<A?>. Then we saw how to create an Arb of pairs of types given a pair of Arbs of (possibly) different types and generalized the idea to any arity. Finally we saw how to map over Arb instances to get an Arb that returns the result of applying a function to the result of the original Arb.
You can see the current version of our library at https://github.com/agile-jordi/wapbtl/tree/part2.
\n\nI feel like doing that: writing a property based testing library. I’ve been blogging about property based testing in the past few weeks. In particular, about value shrinking. You can find the articles here and here. The idea was to talk about the general concept but still show it in real(ish) code. At the same time, I wanted to avoid talking specifically about Kotest, Scalatest or any other library. Not because they are’nt amazing (they indeed are) but because they may be daunting and full-featured, which could be distracting to the purposes of my intent.
\n\nBut now I’m in the mood for writing an actual property based testing library. Not one you will use in real projects, but one I can use to learn (and hopefully show, explain or teach) something about property based testing. Furthermore through my experience with those libraries, I’ve seen things I’d like to improve. Why not try my best at it?
\n\n![\"Let's](\"../assets/img/letsDoThis.gif\")
So… let’s do this! Let’s write a property based testing library from scracth, in Kotlin, and tell the world about what I learn in the way…
\n\nYou may already know about property based testing and maybe even used it. You may as well have read my previous blog posts about shrinking. Or property based testing may be a new concept to you. I hope whatever your situation is, this will be an interesting read, as we dive into such an amazing exercise.
\n\nI’m not goint to lie here. There are plenty of good introductory articles to property based testing, I’m in a hurry, I’m lazy and I don’t want to write a lengthy explanation. So let’s go direct to the point.
\n\nOur statement will be: We want to generate hundreds or thousands of random tests cases automatically with which to test our software instead of testing using hardcoded examples. Whenever our library finds a test case that fails, we want it to search for the simplest test case that still fails, and provide a clear, concise error message about what failed.
\n\nEasy peasy. So, if you are testing a nice sum(a:Int, b:Int): Int function, instead of writing a test like…
val a = 5\nval b = 9\nassertEquals(14, sum(a, b))\nYou’d write something like I know, I know… Bear with me and this ugly design. I just try to avoid spoiling the nice design we’ll use just a couple of paragraphs below…:
\n\ntestManyTimes {\n val a = someRandomInt()\n val b = someRandomInt()\n assertEquals(`???`, sum(a, b))\n}\nBut what value do we use for ??? Why not assertEquals(a + b, sum(a, b)? - you may say… Well, because that would mean, basically duplicating the implementation of the function under test in your test code. Which would have the classical DRY violations flaws and, worst of all, it would only test that 2 probably buggy implementations behave the same way.?
When you use random data for test inputs and initial states, you get results that you can’t foresee. Therefore, you can’t write a test that expects a particular result (or end state). The assertions you need to do depend on the test data randomly generated. We say you are checking properties.
\n\nHere you have an example of one such property:
\n\n\n\n\nFor any integer number i, sum(i, 0) = i
\n
How would we code a simple property like the one above? Clearly not with the testManyTimes and someRandomInt names I used above. We need:
randomInt.So let’s go with this naming:
\n\nforAny(randomInt){ i -> sum(i, 0) == i }\nWe could define one such function like this I’m assuming a certain degree of imperative programming here: The function forAny will throw if there is a test failure. We could of course have it return a result functionally, but I prefer to keep things more familiar to more people.:
\n\nfun forAny(r: RandomInt, property: (Int) -> Boolean): Unit\nI don’t know about you, but this is asking me to generalize to other types than Int:
fun <A> forAny(r: Random<A>, property: (A) -> Boolean): Unit\nUnfortunately, Random is the name of a class in kotlin.random that is used to… generate random numbers. I’d prefer to use a different name. The usual name for such a thing in property based testing is Arbitrary or Arb, for short. So, let’s do some renaming:
fun <A> forAny(a: Arb<A>, property: (A) -> Boolean): Unit\nLet’s try to implement this forAny function. We want to test, let’s say, 100 test cases generated by Arb:
fun <A> forAny(a: Arb<A>, property: (A) -> Boolean) {\n (1..100).forEach { attemptNumber ->\n val sample = a.generate()\n if (!property(sample)) \n throw PropertyFailedException(attemptNumber,sample)\n }\n}\nSo, we found out we need Arb<A> to implement generate() I added a companion object so that we can later add extension functions and values to it, see below.:
interface Arb<A> {\n fun generate(): A\n companion object\n}\nThere are, of course, several important limitationsSome examples of such limitations: We would like to be able to provide a seed that makes our random values reproducible, we’d like to have better error messages, we’d like to be able to configure how many iterations we want on each property we check, we will want the library to simplify test cases that fail to provide the simplest value that still fails, etc. in this implementation, but we want to start with something we can run as soon as possible and iterate from there.
\n\nSo, to check our initial property, forAny(arbInt){ i -> sum(i, 0) == i }, we need an Arb<Int>, a generator of arbitrary Int values. If you know a bit about the Kotlin (or Java) way of generating random values it is not complicated. To make it easier to find using our IDE/editor auto-complete, we will add this Arb value as an extension to Arb companion object:
val Arb.Companion.int get() = object: Arb<Int>{\n override fun generate(): Int = kotlin.random.Random.nextInt()\n}\nNow, everything is in place to actually testWe are using kotlin.test here, but any testing library/framework can work with our library, as it only needs to mark a test failed when a PropertyFailedException is thrown. some properties on Ints:
@Test\nfun testSum0() = forAny(Arb.int) { i -> sum(i, 0) == i }\n✅ Green!
\n\nWhat about finding a test case that fails? We’ll use a property that is not true for the sake of demonstration purposes, but that would be what you would get if you were testing a property of your system and found a bug:
\n\n@Test\nfun testDoubleIsGreater() = forAny(Arb.int) { i -> sum(i, i) >= i }\n🔴 Property failed at attempt 3 with sample 2018188761
\n\ncom.agilogy.wapbtl.PropertyFailedException: Property failed at attempt 3 with sample 2018188761\n\tat app//com.agilogy.wapbtl.ForAnyKt.forAny(forAny.kt:12)\n\tat app//com.agilogy.wapbtl.ForAnyIntTest.testDoubleIsGreater(ForAnyIntTest.kt:14)\n\tat java.base@11.0.13/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nSo far so good… But, hey, I just realized I write automated tests for everything I develop. A property based testing library will be no exeption. So, I’d like to write a test about how a failed property check throws an exception:
\n\n@Test\nfun testDoubleIsGreater() =\n assertFailsWith<PropertyFailedException> { \n forAny(Arb.int) { i -> sum(i, i) >= i } \n }\nBut this seems not enough… Not any PropertyFailedException is ok. At least, we would like to check that the sampleUnfortunately, Kotlin does not allow us to define a parameterized class that extends Exception and, therefore, the sample in our PropertyFailedException is of type Any?. That’s the reason for that ugly cast in the code below. we get returned effectively fails our property check. As I’ll be using the property { i -> i + i >= i } twice, I extracted a value for it:
@Test\nfun testDoubleIsGreater() {\n val property: (Int) -> Boolean = { i -> sum(i, i) >= i }\n val failure = assertFailsWith<PropertyFailedException> {\n forAny(Arb.int, property)\n }\n val failedSample = failure.sample as Int\n assertFalse { property(failedSample) }\n}\nSo we created a minimalist property based testing library that is capable of testing properties of Int values. We implemented it in under 30 lines of code (without relying to magic one-liners). I hope this dissipates some magic feeling arround property based testing.
On top of that, we started writing tests of our property based testing library. I have the intuition that testing a library that generates random values will soon get quite complicated. But, hey, if it were easy it wouldn’t be fun!
\n\nAt the same time, we uncovered several limitations of our property testing function. And we are only capable, so far, of testing properties on Int values.
\n\nYou can see the current version of our library at https://github.com/agile-jordi/wapbtl/tree/part1.
\n\nWill our intrepid developers overcome such limitations? Stay tuned!
\n\n![\"Batman](\"../assets/img/same-bat-channel.png\")
In our previous blog post on shrinking we saw that we want to find the simplest failing example we can in a shrinking tree. We had an initial example that failed our tests:
\n\nItemSearchCriteria(\n ItemFilter(23, 110, 2022-01-01T00:00:00.000, setOf(Tag.clothes)),\n AgetItemOrder\n)\nTo try to find simpler examples that may still be good examples of how our system fails, we decided to simplify the initial example, to shrink it. To do so, we started by removing filters one by one. With this strategy we got a list of simplifications we could try and further simplify if we wanted.
\n\nToday we’ll try to discuss how this intuititive idea of shrinking can be automated.
\n\nLet’s say our PBT library encodes generators of values of type A as an interface Arb<A>. In our example above, we would have a Arb<ItemSearchCriteria>. Such an interface would be something like:
interface Arb<A>{\n fun generate(random: Random): A\n}\nIntuitively we would like to also have Shrinkers, that we could define as a separate interface or as part of the Arb interface. In any case, they would provide a function that given a value of type A returns the list of possible first level shrinks:
fun shrinks(value: A): List<A>\nThe type of such a function would be:
\n\ntype ShrinkFunction<A> = (A) -> List<A>\nA shrinking function takes the value you want to shrink and returns a list of one-step shrinks from that value.
\n\nThe shrink function must return a list of values because there may be multiple different ways of shrinking the same value. In our example, when simplifying ItemFilter we could remove minWeightInKgs, maxWeightInKgs, olderThan or hasAllTags and each one gives us a different value..
On the other hand, it is important that the shrink function only returns different ways of slightly shrink a value, since the shrinking tree will be built by recursively applying the shrink function to a shrinks of the list when needed. In our example, when simplifying ItemFilter we don’t need to remove combinations of more than one filter (like removing both minWeightInKgs and maxWeightInKgs) as that example will be obtained by further simplifying an example where we first removed just one filter.
Given such a definition, the PBT library can define shrink functions for some ubiquitous simple types like Int or Instant Unrelated: Because you know and use Instant, right? You know DateTime does not correctly represent when something happened but else what did the clocks somewhere in the world display when that happened, right?.
An example naive implementation of Int shrinking may be:
fun intShrinks(value: Int): List<Int> =\n if (value == 0) emptyList()\n else if (value < 0) intShrinks(-value).map { -it }\n else listOfNotNull(\n if (value >= 20) sqrt(value.toDouble()).toInt() else null,\n if (value >= 10) value / 2 else null,\n if (value < 20) (value - 1) else null\n )\nThe value 110 would produce the following shrinking tree Note this is a different shrinking function than the one I used in the previous post. If you noticed, sorry about that… and I’m frankly impressed you did.:
graph LR\n 110 --> 10 --> 5\n 10 --> 9 --> 8 --> 7 --> 6 --> 5 --> 4 --> 3 --> 2 --> 1 --> 0\n 110 --> 55 --> 7\n 55 --> 27 --> 5\n 27 --> 13 --> 6\nWe can go as far as we want here. Depending on your use case, you could also simply rule that the only shrink of an Int value you want to try is 0, simplifying things. But most of the time you can rely on the shrinking function provided by your PBT library and call it a day.
Trying to generalize generic shrinks seems also easy for nullable types:
\n\nfun <A> shrinks(value: A?): List<A?> = listOf(null)\nBut, “Wait a minute!” - you say. - “Then, the shrinks for 55 of type Int are listOf(7,27) but the shrinks for 55 of type Int? are just listOf(null)?”. And you are right, we are loosing the nice shrink function we already have for a type and not applying it to the nullable version of it. So let’s try to generalize that to a type that already has a shrinking function:
fun <A> nullableShrinks(value: A?, notNullShrinks: (A) -> List<A>): List<A?> = \n if (value == null) emptyList() \n else listOf(null) + notNullShrinks(value)\n\nfun nullableIntShrinks(value: Int?) = nullableShrinks(value, ::intShrinks)\nThat is, you can shrink the nullable value to null or either try to shrink it as you would if it weren’t nullable.
\n\nNow that I already introduced a high order function (nullableShrinks is a function that takes a function as a parameter) allow me to dig deeper. I can’t help but do some functional programming here. You can skip this section if you are already tired of me selling this drug:
\n\nFirst let’s redefine nullableShrinks to not only take a function as a parameter but to also return one. The idea is that it only takes the not null shrinking function (without the value to shrink) and it returns the shrinking function:
fun <A> nullableShrinks(notNullShrinks: (A) -> List<A>): <A> -> List<A?> = { a ->\n if (a == null) emptyList() else listOf(null) + notNullShrinks(a)\n}\nval nullableIntShrinks: (Int?) -> List<Int?> = nullableShrinks(::intShrink)\nOh! I forgot our nice type alias:
\n\nfun <A> nullableShrinks(notNullShrinks: ShrinkFunction<A>):\n ShrinkFunction<A?> = { a ->\n if (a == null) emptyList() else listOf(null) + notNullShrinks(a)\n}\nval nullableIntShrinks: ShrinkFunction<Int?> = nullableShrinks(::intShrink)\nFurthermore, if you like Kotlin function receivers as much as I like them, you can define and use such shrink function as:
\n\nfun <A> ShrinkFunction<A>.nullable(): ShrinkFunction<A?> = { a ->\n if (a == null) emptyList() else listOf(null) + this(a)\n}\nval nullableIntShrinks = ::intShrink.nullable()\nval nullableInstantShrinks = ::instantShrink.nullable()\nAnd now, we can imagine ourselves, users of the PBT library, implementing shrink functions for our types:
\n\nval itemFilterShrinks: ShrinkFunction<ItemFilter> = { value ->\n listOf(\n value.copy(minWeightInKgs = null),\n value.copy(maxWeightInKgs = null),\n value.copy(olderThan = null),\n value.copy(hasAllTags = emptySet())\n )\n}\nBut why not use the nice nullableIntShrink we just defined, instead of just trying minWeightInKgs=null? In fact, for each parameter we could try all the shrink values its shrink function provides. Assuming we also have an instantShrinks, we could do something like:
val itemFilterShrinks: ShrinkFunction<ItemFilter> = { value ->\n // Either shrink minWeightInKg\n ::intShrinks.nullable(value.minWeightInKgs).map { i -> \n value.copy(minWeightInKgs = i) \n } + // or maxWeightInKg...\n ::intShrinks.nullable(value.maxWeightInKgs).map { i -> \n value.copy(maxWeightInKgs = i) \n } + // or olderThan...\n instantShrink.nullable(value.olderThan).map { i ->\n value.copy(olderThan = i) \n } + // or hasAllTags...\n listOf(value.copy(hasAllTags = emptySet()))\n}\nOne way to understand this shrinking function is:
\n\nminWeightInKg) there are as many possible shrinks as the field type shrinking function provides (e.g. nullableIntShrink(value.minWeightInKgs) for the minWeightInKgs field value).Ok, we got this far. Nice! We can build shrink functions for all the types we use as test inputs. Isn’t that a large amount of types? That probably is a large amount of types. Isn’t it tedious? It is probably tedious. Is it error prone? It is indeed very error prone!
\n\nLet’s look at these 2 problems in isolation:
\n\n![\"The](\"../assets/img/TheIncredibleShrinkingMan.jpg\")
Learn from the incredible shrinking man: A bug while playing with shrinking is much more dangerous!
\n\nThe user provided shrink functions are… functions, indeed. And they are code. And code has bugs. But these are an insidious type of bugs, as they only show when you are already trying to understand and fix a failing test that may be as well a bug. Yikes! So, in your happy path, you won’t be running your shrink functions (neither in production nor in tests). They will run only when you have some test failure.
\n\nWhen I’ve found a bug in one of our shrink functions, it usually goes like this:
\n\nSo, I know, I don’t like bugs and I’m complaining like that was your fault. But, “hey, Jordi!” - you say - “good programmers write tests”. And tests I write, indeed. But I want my shrinks to properly shrink whatever input I provide, and I’m not in the mood to be just writing examples and the shrinking I expect of them.
\n\nI’m leaving the mystery about how we test our shrinkers for a different post. Suffice to say that it is not easy. And, in case you want one spoiler, we don’t test in isolation but integrated in the PBT library, that applies them repeatedly in search for the simplest example.
\n\nLet’s think about our original example. In fact we wanted to simplify a value of type ItemSearchCriteria:
data class ItemSearchCriteria(val filter: ItemFilter, val order: ItemOrder)\nSo we need a shrink function for it too. But the strategy is exactly the same:
\n\nItemSearchCriteria, 4 for ItemFilter). We don’t care about combination of fields.filter for ItemSearchCriteria or minWeightInKg for ItemFilter), there are as many possible shrinks as the field type shrinking function provides.So, assuming we also defined itemOrderShrinks:
val itemSearchCriteriaShrinks: ShrinkFunction<ItemSearchCriteria> = { value ->\n // Either shrink filter\n itemFilterShrinks(value.filter).map { f -> \n value.copy(filter = f) \n } + // or order...\n itemOrderShrinks(value.order).map { o -> \n value.copy(order = o) \n }\n}\nWhat a repetition, huh? If only we could generalize that and automate it…
\n\nOur shrinking functions heaven would be to automatically derive the shrinking functions our types need. For that we need a generic algorithm that is capable of returning a shrinking function for a given type (spoiler, it will take other shrinking functions as inputs). On top of that, some black magic may prove useful; something that, given the type we want to shrink (e.g. ItemSearchCriteria) automatically invokes such function and returns the shrink function. That could be based on introspection, macros… But my doctor says I’m not allowed to use black magic while programming; I seem to be allergic. So I prefer to compose generators using monads and have the shrinking function derived for me, as I’ll show in the (hopefully near) future.
As you may already know, Property Based Testing (PBT) is an automated testing technique where you test properties of your programs or functions instead of testing particular examples like we do in classical unit testing. A PBT library allows you to program properties (e.g. “for all integers i such that i != 0, 0 / i = 0”) and generators of values that generate random valid examples (e.g. a generator of integers i such that i != 0). It then checks that your program abides by the property you defined for many How many are “many examples”? This is a matter of a different post. generated random examples, assuming that if no counter-example is found after a number of attempts bigger enough, the property is true. Such properties can be written as tests that, instead of hard-coded examples, use randomly generated inputs (and initial state); we will say a property holds for a given example if the test doesn’t fail.
In this article I will focus on what happens when the property does not hold and an example is found that fails the test. In traditional automated testing, we only need to know which of our test scenarios failed an assertion (and which one), as the test has hard-coded examples we can reason about. But in PBT, beyond the property that failed an assertion, we may need to know what particular counter-example was found. Unfortunately, that randomly generated example may be very complex and there may be simpler values that also fail the test. Shrinking is a feature of property based testing libraries that tries to find such a simpler failing value.
\n\nLet’s say you have items in some repository represented here as a the function searchItems For the sake of simplification we will assume this repository has a fixed, hardcoded state, a number of hardcoded items so it can be represented as a pure function and we don’t have to deal with state.:
data class Tag(val value: String)\ndata class Item(\n val id: String, \n val weightInKgs: Int, \n val createdAt: Instant, \n val tags: Set<Tag>\n)\n\nsuspend fun searchItems(criteria: ItemSearchCriteria): List<Item> = notShown() \ndata class ItemSearchCriteria(val filter: ItemFilter, val order: ItemOrder)\ndata class ItemFilter(\n val minWeightInKgs: Int?, \n val maxWeightInKgs: Int?, \n val olderThan: Instant?, \n val hasAllTags: Set<Tag> = emptySet()\n)\ninterface ItemOrder {\n companion object{\n object Indifferent: ItemOrder\n object Age: ItemOrder\n object Weight: ItemOrder\n }\n}\n\nAnd let’s say you have a property such that:
\n\nfun Item.matches(filter: ItemFilter): Boolean =\n (filter.minWeightInKgs?.let { this.weightInKgs >= it } ?: true) &&\n (filter.maxWeightInKgs?.let { this.weightInKgs <= it } ?: true) &&\n (filter.olderThan?.let { this.createdAt < it } ?: true) &&\n this.tags.containsAll(filter.hasAllTags)\n\ntestAll(itemSearchCriteriaArb) { searchCriteria ->\n val result = searchItems(searchCriteria)\n val failingResults = result.filter { it.matches(searchCriteria.filter) }\n assertEquals(emptySet(), failingResults.toSet())\n}\nAssume we get a failing test with the following sample, for which our function returns items with weight above 110.
ItemSearchCriteria(ItemFilter(23, 110, 2022-01-01T00:00:00.000, setOf(Tag.clothes, Tag.exclusive)), AgetItemOrder)\nNow you know that the searchItem function is not behaving correctly with this particular example. Imagine there is a bug in the filter by maxWeightInKg that, by mistake, filters by the reverse condition in your implementation, but, of course, the developers are unaware of it.
![](\"https://y.yarn.co/76c65c90-5e05-419f-9a84-0f8adc431d92_text.gif\")
In our example, we found a test that fails when the randomly search criteria are the ones shown above. Let the game begin!
\n\nOf course we may notice that the returned items all have a weight above 110 and we would be done. But that may be difficult to realize. Imagine your search returns hundreds or thousands of items and that each item, instead of a blog example is a complex entity. Now you are probably facing a daunting screen of thousands of lines printing the returned examples and it is difficult to see what of the filters they fail to satisfy.
\n\nThe problem is that the example for which we got a failure is way too complex. We have (potentially) a lot of distracting values that don’t matter for the bug at hand. Knowing what the bug is, you and me know that it doesn’t matter wether you filter by olderThan or not. Knowing what the bug is, if I asked you, a wonderful and smart human being, what the simplest counter-example is, you would probably give me something like ItemSearchCriteria(ItemFilter(null, 0, null, emptySet()), Indifferent).
You are indeed smart… but, out there, you don’t know what the bug is in advance. So let’s try to simplify the example bit by bit.
\n\nLooking at the problem at hand and assuming I don’t have any clue about what is failing, I’d try to remove criteria I think won’t affect filtering. So, let’s try telling our function we don’t care about sorting:
\n\nItemSearchCriteria(ItemFilter(23, 110, 2022-01-01T00:00:00.000, setOf(Tag.clothes, Tag.exclusive)), Indifferent)\n1️⃣ 🔥 A failure. And now we have a simpler example, one with less distracting values.
\n\nNow I’d try removing filters one by one. Let’s go left to right:
\n\nItemSearchCriteria(ItemFilter(null, 110, 2022-01-01T00:00:00.000, setOf(Tag.clothes, Tag.exclusive)), Indifferent)\n2️⃣ 🔥Ok, it still fails. So far so good, it seems that the sorting and min weight aren’t unrelated to the problem.
\n\nLet’s go for maxWeightInKgs:
ItemSearchCriteria(ItemFilter(null, null, 2022-01-01T00:00:00.000, setOf(Tag.clothes, Tag.exclusive)), Indifferent)\n![](\"https://c.tenor.com/TlfAvuz0tLMAAAAC/obi-wan-kenobi-these-are-not-the-droids.gif\")
3️⃣ ✅ Woa! That one passes the tests! This is already a very good hint! We may have a bug in maxWeightInKgs. But what about the other filters? Are they related to the failure? Our example is still not as nice as the one we know we want if we know where the bug is. At the same time, as this example didn’t fail, I wouldn’t probably keep shrinking it. In other terms, setting maxWeightInKgs to null opens a path (that would admit more simplifications) where our bug is not manifested. These are not the examples we are looking for.
But we can try to keep simplifying 2️⃣ to check wether olderThan and hasAllTags are unrelated. So, let’s try to remove olderThan:
ItemSearchCriteria(ItemFilter(null, 110, null, setOf(Tag.clothes, Tag.exclusive)), Indifferent)\n4️⃣ 🔥 Ok, it keeps failing. olderThan seems unrelated to our failure.
ItemSearchCriteria(ItemFilter(null, 110, null, emptySet()), Indifferent)\n5️⃣ 🔥 hasAllTags seems unrealted to the failure too.
What about going one step further?
\n\nItemSearchCriteria(ItemFilter(null, null, null, emptySet()), Indifferent)\n6️⃣ ✅ The test now succeeds. maxWeightInKgs definetively seems the cause of our trouble.
So the simplest example we’ve got so far is example 5️⃣. But we can play this game even further. Do we have a problem with the number 110? Or any number would fail? As humans, round decimal numbers and numbers closer to 0 seem simpler, so we could keep shrinking this example to test for maxWeightInKgs 100 7️⃣, 50 8️⃣, 0 9️⃣ and it would still fail. So we got:
ItemSearchCriteria(ItemFilter(null, 0, null, emptySet()), Indifferent)\nAssuming 0 is the simplest value for the maxWeightInKgsfilter (other than null), the example we have can’t be further simplified.
Now, that’s a nice example to use to try to understand our bug!
\n\nIf we think carefully at what we just did we can see we are doing a search in a tree of simplifications Here 🔥 means the example fails the test and ✅ means the example passes the test.:
\n\ngraph LR\n 0[initial 🔥] --> 1[1 🔥] --> 2[2 🔥] --> 3[3 ✅]\n 0 --> initialOthers[...]\n 1 --> 1others[...]\n 2 --> 4[4 🔥] --> 5[5 🔥]--> 6[6 ✅]\n 2 --> 2others[...]\n 4 --> 4others[...]\n 5 --> 7[7 🔥] --> 8[8 🔥] --> 9[9 🔥]\n 5 --> 5others[...]\n 7 --> 7others[...]\n 8 --> 8others[...]\nWe did this more or less by hand. But, hey, there is a reason they taught you how to traverse a graph! Computers like to do this kind of stuff and they excel at it!! Why don’t ask our fancy open source PBT library to shrink for us?
\n\nWe have seen how property based testing gives us complex examples of inputs for which our program doesn’t satisfy the property we expect it to. Once we have one such example it may be difficult to diagnose what the bug was. One strategy we may use is to simplify or shrink our example to get simpler examples that still fail. That can be useful to reason about what is failing but also to debug our program with less distracting inputs, for example.
\n\nIn the next post in the series we’ll discuss how a property based testing library can automate this shrinking and what are the difficulties of doing so.
\n\nPreviously on Agilogy blog…
\n\n\n\n\n\n\nWhat if your function is not deterministic because… it generates random values? Or just because it uses the current system time? And what about it doing some I/O, like reading from the file system, a database or a socket? Even more, what about procedures generating some kind of externally observable effect, like writing to the file system or a databse?
\n\n \n
Let’s take a very simple function:
\n\nfun age(birthDate: LocalDate): Int = \n Period.between(birthDate, LocalDate.now()).years\nSimple, right? One line, and that means simple, right? Of course not. But why? Let’s try to write an automated test for it.
\n\nNow, there are at least these 3 typical moments in the life of a developer where you can be in. You may be reading this puzzled, not knowing how the hell to test that without having to change the test yearly. Or you start thinking mocks. Or you start thinking how to redesign this one-linerI mean, beyond the ugly implicit usage of the system default time zone, which I, at least, would refactor out of my sight..
\n\nI’m going to try to take you from puzzled to a better design without going through mocks Are you already thinking about them? Please, consider this article an alternative to mocking..
\n\nThe issue with testing this function is, you may have guessed, Instant.now(), which would return a different value on each test execution.
You could, of course, try something like this:
\n\n@Test\nfun testAge() {\n assertEquals(7, age(LocalDate.of(2014, 7, 30)))\n}\nBut then, of course, after my son’s birthday party the test will fail and you would need to update it. Nasty.
\n\nThe problem, here is that we are getting the current date LocalDate.now() and that has the odd habit of chaning as time goes by. We are also calculating how many years are there between 2 local dates, which would be fine to test, but… Yes, these 2 things this function does are what Rich Hickey would call complected If you haven’t seen this famous talk, make yourself a favour and bookmark it: Simple Made Easy. Then, when you are ready, take a cup of your best coffee, tea, mate or whatever is your poison and enjoy an hour of pure talent.:
\n\n\nOkay. So there’s this really cool word called complect. I found it. I love it. It means to interleave or entwine or braid. Okay?\n…\nHaving state in your program is never simple because it has a fundamental complecting that goes on in its artifacts. It complects value and time. (…) Well, if every time you call that method with the same arguments, you can get a different result, guess what happened? That complexity just leaked right out of there (…). If the thing that’s wrapping it is stateful (…) by stateful I mean every time you ask it the same question you get a different answer, you have this complexity and it’s like poison.
\n\nRich Hickey, Simple Made Easy
\n
So, even though it is a one-liner, we want to simplify this function. A first step would be to rule off the function that we find difficult to test:
\n\nfun ageOn(today: LocalDate, birthDate: LocalDate): Int = \n Period.between(birthDate, today).years\nThat one is easy to test. We say it is testable Testability is, indeed, a software quality factor. But, curiously enough, aiming at testability enhances the quality of your software in other aspects as well.. But now the user of the function needs to do what was part of the responsibilities of age before: getting the current local date. We can recover our original age function defining a new function like this:
fun age(birthDate: LocalDate): Int = ageOn(LocalDate.now(), birthDate)\nNow we can test we correctly calculated the number of years between two dates and that that is what we actually mean by age:
\n\n@Test\nfun testAgeOn() {\n assertEquals(\n 7,\n ageOn(LocalDate.of(2022, 6, 1), LocalDate.of(2014, 6, 30))\n )\n}\nWhat about the original age function? How do we test that one? We still can’t. The problem with testing age was, to begin with, that we don’t know how to test the part that gets the current local date… So, here are some questions that will make you think:
LocalDate.now() returns the current date?LocalDate.now(), how do we test that age uses LocalDate.now() and not some other wrong / untrusted / untested method to get the current date?age correctly gets the current date somehow, how do we know it is (correctly) using ageOn?Or, put it in another way, even with our test of ageOn, each risk in the following list corresponds to one of the questions we were asking above:
LocalDate.now() could be producing the wrong date, not the current dateage could be getting the current date using some untested method that gets it wrongage could be correctly getting the current date but fail to calculate the number of years between the 2 dates, as we tested ageOn but we didn’t test wether age is invoking it and, if so, if it is handling it the correct arguments in the correct order.Let’s start with the last two concerns here…
\n\nLet’s imagine we solve the first question/concern above somehow More on that at the end of the article. But let’s consider it solved now.. What would we need to solve the other two? We want some new version of age that we can test, so that we solve the last question. And we want to make sure it uses what we want to get the current local date, so that the second question is also solved.
That is, we want a function that, given the way of getting the current date of our choice and a birth date, gets it and properly calculates the number of years since the birth date to that current date. Do you see what I did there? I just used the same expression for “the way of getting the current date” and “a birth date”. Yes, they both can be parameters:
\n\nfun age(getCurrentDate: () -> LocalDate, birthDate: LocalDate): Int =\n Period.between(birthDate, getCurrentDate()).years\nWe can now test almost everything we were concerned about:
\n\n@Test\nfun testAge() {\n val getDate: () -> LocalDate = { LocalDate.of(2022, 6, 1) }\n assertEquals(\n 7,\n age(getDate, LocalDate.of(2014, 6, 30))\n )\n}\nNote that getDate here is a function that always returns the same result. () -> LocalDate is the type of functions that have no parameters and return a LocalDate.
Now we can build tests where we check that age uses whatever function we pass it as the first argument (our second concern), and that age properly calculates the number of years between birthDate and the date returned by getCurrentDate.
We say that age depends on the function that returns the current date and that we injected that dependency into age.
This is just one particular style of dependency injection, one with which you may not be familiar. Some other styles come to mind:
\n() -> LocalDate we could have injected an instance of some named type like interface Clock that contained methods for what we wanted to acomplish:fun age(clock: Clock, birthDate: LocalDate): Int =\n Period.between(birthDate, clock.getCurrentDate()).years\nage, we could have put the age function in a class and inject something to the class:class AgeModule(val clock: Clock){\n fun age(birthDate: LocalDate): Int =\n Period.between(birthDate, clock.getCurrentDate()).years\n}\nMonad I may talk about that in a future article. defining age as a function that returns a function that given whatever it needs injected, returns the result we want:fun age(birthDate: LocalDate): (Clock) -> LocalDate = { clock ->\n Period.between(birthDate, clock.getCurrentDate()).years \n}\nThat one, the one I left unsolved so far is the easiest one to solve. I don’t have any clue about how to properly test that, but I don’t even care. “How do I test that, then?” - you say. My answer: Don’t. Don’t test it. Nada. Niente. Res de res.
\n\nThis may seem silly but it is, in fact, a very important part of our testing strategy at Agilogy. You should not be testing other’s code. And by “other’s” I mean code in another module. Once you think about it, in fact, it is quite common sense:
\n\nage function here and my point is that testing that a dependency of age properly does its work is out of the scope of the tests for age..So I cheated you. I told you I was going to talk about testing and I ended up talking about complection, software design and dependency injection…. or didn’t I?
\n\n![\"The](\"../assets/img/Matrix-Oracle.jpeg\")
The Oracle, not the Oracle DB
\n\nSo we have built a simpler, better age function. But, paraphrasing the Oracle, what’s really going to bake your noodle later on is, would we still have reached such simple, elegant solution if I hadn’t said anything about testing?
The recipe consisted of (1) setting the initial known state, (2) executing the functionality to test, (3) collecting and asserting over results and (4) collecting and asserting over final state
\n\nIn our previous post about software testing, we gave the perfect receipt to test something even in the presence of state. Armed with that, we were able to test pure functions and “functions” whose responses depend on some state. I left you with the promise of talking about getting the current time, reading or writing from files and some other nasty side effects. We will now focus on reading from and writing to some persistent storage. Here we go.
\n\nYou remember the silly example of our test of MemoryAdder, don’t you? Its linked up there. I wait for you just here. Done? Ok.
So, now imagine you have a database (of any kind) where you store your valuable data. A database basically keeps state for as long as you want and (probably) indexes it to allow you to query it in fancy ways. Now, let’s say you have a “function” that saves something to a database. I’m assuming something similar to an sql database here, and talking about rows, etc. It doesn’t matter, for the purpose of this article. We could be speaking of a document database and its documents instead.
\n\nNote: You wouldn’t put Connection as a parameter of this function? Nice, me neither. Stay with me on this one. If I manage to remember what I’m just saying, which I probably won’t 🐠, I plan to write about dependency injection using this very example as our starting point.
fun updateUserLastVisit(\n conn: Connection, \n userId: Long, \n lastVisit: Instant\n): Boolean {\n val updatedRows = \n conn.execute(\n \"update users set last_visit = ? where id = ?\",\n lastVisit,\n userId\n )\n return updatedRows > 0\n}\nTo test this method you simply:
\n\nSo, let’s tryIt won’t work this time, keep reading for a better solution:
\n\n// 1. Set initial state\nval userId = 23L\nval userName = \"John\"\nconn.execute(\"insert into users(id, name, last_visit), ?, ?, null\", userId, userName)\n// 2 and 3. Execute the method and assert\nval lastVisit = Instant.now()\nval wasUpdated = updateUserLastVisit(userId, lastVisit)\nassertTrue(wasUpdated)\n// 4. Collect the final state and assert\nval endState = conn.execute(\"select last_visit from users where id = ?\", userId).first()\nassertEquals(lastVisit, endState)\nNice, no? Nope. You may already know me. Sparing English not being my native tongue, when I say something I mean it. And I said, I quote, “Collect the final state”. That’s not what I did here. Red flag! 🚩Alert! 🚨 Why? Well, I just checked the value I assumed I was modifying, I didn’t collect the (whole) end state.
\n\nSide effects everywhere
\n\n![\"side](\"../assets/side%20effects.gif\")
What’s the problem with that, you say? After all, some projects out there forget to test the final state and many of the ones that don’t, check it like we did above. The problem, as you may already see, are… unintended side effects.
\n\nWhat if updateLastUserVisit also updates the last_updated column? Would that be intended (and missing from our test) or unintended (and passing our test wrongly)? What if it deletes the row in some particular case? Or if it affects some other row?
So, let’s fix that step 4Only step 4, here. See the complete solution later.:
\n\n// 4. Collect the final state and assert\ndata class User(userId: Long, userName: String, lastVisit: Instant) \nval endState = \n conn.execute(\"select id,name,last_visit from users order by id\"){ \n (i, n, lv) -> \n // oh dear, how I miss Scala's tuples here...\n User(id, n, lv)\n }\nassertEquals(listOf(User(userId, userName, lastVisit)), endState)\nOne intesting side benefit of checking the final state this way is that the auxiliary method we need to retrieve the final state is always the same, no matter the particular test at hand. That saves some nice hours of coding tests.
\n\nBut why would we only check the users table? What about other tables? Sure, you could check absolutely all the end state if you want. I’ve done that sometimes. Just balance the effort against the probability a function supposed to use one table manipulating the data in another one.
\n\nAnd what about the performance of querying all the rows in a table? This database is one you never share, remember? And the initial state it has is only the state you set it to have in the first step of our recipe, right? So, yes, you can probably query all of the rows in your database; in this example, it is just one.
\n\nNow it’s done, right? Oh, you have seen it, right? No? Again, let’s do what we said, and we said “set initial state”. Inserting a single row in a database is not stting a known initial state. There may be other rows there. And, hopefully, our rewritten step 4 would detect that and our nice test would fail.
\n\nSo let’s fix that as well. Final solution nowI’ll be considering this to be the good solution. Of course, it may have bugs, in which case I would greatly appreciate any feedback.:
\n\n// 1. Set initial state\nval userId = 23L\nval userName = \"John\"\nconn.execute(\"delete from users\")\nconn.execute(\n \"insert into users(id, name, last_visit), ?, ?, null\", \n userId, \n userName\n)\n// 2 and 3. Execute the method and assert\nval lastVisit = Instant.now()\nval wasUpdated = updateUserLastVisit(userId, lastVisit)\nassertTrue(wasUpdated)\n// 4. Collect the final state and assert\nval endState =\n conn.execute(\"select id,name,last_visit from users order by id\"){ \n (i, n, lv) -> User(id, n, lv)\n }\nassertEquals(listOf(User(userId, userName, lastVisit)), endState)\nNote: I’m assuming the database structure was fixed. We don’t need to create the users table when we run the test because it is something our code does not change.
\n\nThose are the words I used. Test your functions using persistent state using “an actual database instance nobody else will use”. There are two main points here.
\n\nThe first one: Use an actual persistent storage. In fact, use the very same persistent storage technology you use in production. You have PostgreSQL 13.7 in production? Use a PostgreSQL 13.7 to store your tests data. Because, why not? The benefits of using the very same technology are crystal clear: there won’t be any difference that causes false positives (code that fails in production but passes all the tests ) or false negatives (code that works perfectly well in production but fails to pass the tests).
\n\nI won’t go into details, at least not now, but you can use Docker and other virtualization solutions to run such infrastruture. Just, please, don’t make all developers in your team install a ton of software in order to run all tests in the repo.
\n\nThe other part of that suggestion is about not sharing the state with anyone else. I already hinted about some of the problems I’ve seen when sharing the persistent state used in tests. You don’t want your tests behaviour to depend on whether someone else actions. You want them to always give the same results. Known initial state, exercise the program, collect results and assert, collect final state and assert.
\n\nWhat about the tests in your test suite sharing the persistent state? No! Nein! Ni parlar-ne!
\n\nGollum doesn’t want to share its preciousss… persistent state. Me neither.
\n\n![\"Its](\"../assets/its-mine-lotr.png\")
You are probably using the same database for all your tests. But you can’t share state between tests. You need to be able to run a single test. You’ll maintain tests and you may remove some of them, or add one test in between existing tests. You can’t afford the result of one test depending on the state left by other tests executed before it. What do yo do? Known initial state, exercise… You know. I mean, following the recipe I gave in the last article, you guarantee each test starts with a known state, no matter what other tests have done before.
\n\nDon’t simply take whatever initial state you find while trying to cleanup afterwards.
\n\nFor some reason, I’ve seen cases where the recipe is different. Instead of setting a known initial state, tests make the effort to leave things as they found them: they clear whatever state they stored during the test. That tries to solve the same problem of one test affecting other tests. But it is flawled. To begin with, a simple failure to properly clean the state could trigger a failure (or worse, a false positive) in an unrelated test of the same suite. Good luck finding out what just happened. On the other hand, whenever a test fails, sometimes it is very nice to see the end state there for you to query. You are welcome.
\n\n![\"Compiling\"](\"../assets/img/compiling.png\")
And… what about concurrency? I don’t know yours, but my laptop is much, much powerful than any of the nodes in the production environment that run the code I write with it. Heck, it has more cores than I can remember and long are the days when I could drink I coffee while my previous laptop struggled to compile a big Scala project.
\n\nBut concurrency is not a trival one to solve, when you have persistent state in tests. Specially not as an afterthought.
\n\nSo far, we avoided others (developers or the CI server) affecting our builds, by running our persistent storage (e.g. our PostgreSQL) in the same computer where the tests run. And we had the certainty that our tests run in a known state because we set that state ourselves: no test trusted whatever state it found there (from previous test executions or whatever) an it set the wanted, known initial state itself. But now, we may have several concurrent tests doing the very same. So they will affect each other.
\n\nThe solution to this problem usually is to just not run tests concurrently. Let that small chat app eat that CPU as though there were no tomorrow.
\n\nEvery other solution depends on the technology you use. For the sake of giving some hint at it, assuming you know a bit about relational databases, we could make each test generate a random unique identifier and, then, create and use a schema or a table with such name as part of the name. Of course, we would need to be able to create such schemas or tables from our test code.
\n\n// 1. Set initial state\nval userId = 23L\nval userName = \"John\"\nval executionId = UUID.randomUUID()\nconn.execute(\"create users-$executionId(...)\")\nconn.execute(\n \"insert into users-$executionId(id, name, last_visit), ?, ?, null\", \n userId, \n userName\n)\nSo we are now equiped with the perfect recipe to run tests that use persistent state without any hassle. We played a bit with it and saw what we actually mean when we say “set the initial state” and “collect and assert the final state”. We also saw how to run tests concurrently to take advantage of the large number of cores we have in our laptops nowadays.
\n\nBut many open questions from the first article remain unanswered. What about testing a function that gets the current time? One that uses random values? What about something that sends HTTP requests to the world?
\n\nMore on all of that to come. See you soon!
\n\nBack in the day, we used to just hack some code, run it, and give it some inputs to see whether we acomplished what we wanted or not. Naturally, this is no longer the case, and nowadays we use automated tests to automatically check our code behaves as expected.
\n\n\n\n![\"Bored](\"../assets/bored-boring.gif\")
I know, I know… But let’s revisit how do we test a simple program. Bear with me, please, as many people and teams just skiped that part of the explanation when talking about tests to their juniors.
\n\nLet’s imagine we have a simple function like this one:
\n\nfun sum(a: Int, b: Int): Int\nA classical test suite is a collection of test scenarios each of which tests one combination of inputs to the function following these simple steps:
\n\nLike this:
\n\n@Test\nfun testSumPositives() {\n assertEquals(23, sum(21, 2))\n}\n\n@Test\nfun testSumPositiveAndNegative() {\n assertEquals(19, sum(21, -2))\n}\nYou probably already know, but just in case: assertEquals takes the expected value as the first argument and the actual value, the one your code actually returned, as the second argument. Please, please, stop doing the order of those arguments wrong, as failing tests are really confusing when they are switched.
Easy peasy! Isn’t it?
\n\nBut, what about impure “functions”? You know I love pure functions, right?
\n\nFunctional programmer with an agile background
\n\n![\"agile_jordi](\"../assets/agile_jordi.jpg\")
What happens when you want to test something that is not a pure function?
\n\nLet’s start with totality, as that one is easy. A function is total when it returns a result (of the specified type) for every possible input. When a function is not total, you can do 2 different things in your tests:
\n\nThe first approach is valid when you don’t care about such cases. Don’t look at me, I always care, because I don’t want my types to lie to me. But you may not care and that’s a topic for another post.
\n\nThe second one is doable by hand with try ... catch, although every testing library I know offers some higher level solution:
fun div(a: Int, b: Int): Int = a / b\n\n@Test\nfun divBy0() {\n val t = assertThrows<ArithmeticException> { div(23, 0) }\n assertEquals(\"/ by zero\", t.message)\n}\nThere is still a catch. Your function may be non total without you, the poor programming, knowing about it. But if you are lucky enough that one of your tests discovers such a hole in your function, the test library will usually deal with it, showing the thrown exception as a particular case of a test failure:
\n\n@Test\nfun failingTestDueToUnexpectedException() {\n assertTrue(div(23, 0) < 23)\n}\nLike you know, a “function” is not deterministic when, for the same inputs, it may return different outputs. One very good reason for it to not return the same input is having state. If you query the amount in your savings bank account you don’t expect it to be always the same. You expect the result to depend on the state of what is being test.
\n\nTake this simple adder:
\n\nclass MemoryAdder(){\n var lastInput: Int\n fun add(a: Int): Int = (lastInput + a).also{ lastInput = a }\n}\nHow do we test the add method? As the result depends on the state of the MemoryAdder, we need a new algorithm:
Note that many developers and teams forget step number 4. If you don’t check the final state of the system under test, such system can behave as you expected in your test and, still, fail to end in the state you expect it to have at the end.
\n\nOther teams, though, forget about step number 1. If your system uses any kind of persistent state, that means you are testing it with whatever is present in such storage when you launch the test. And that grants you some headache whenever that state is not the one you expected. That can be caused by some other test changing some state you didn’t anticipate it would change, you or some other developer running tests in concurrency with your test execution, someone doing manual tests using the same database you expected no one to use… You know. I’ve seen things you people wouldn’t believe.
\n\nBut I digress… Let’s apply our new algorithm:
\n\n// 1. Set initial state:\nval ma = MemoryAdder()\nma.add(23)\n// 2 and 3. Execute the method and assert\nassertEquals(25, ma.add(2))\n// 4. Collect the final state and assert\nassertEquals(2, ma.lastInput)\nGood job! Done, right? Not quite yet…
\n\nWhat if your function is not deterministic because… it generates random values? Or just because it uses the current system time? And what about it doing some I/O, like reading from the file system, a database or a socket? Even more, what about procedures generating some kind of externally observable effect, like writing to the file system or a databse? 🙀 Yes, it’s a cat screaming. Yes, coming from Scala, that pun is indeed intended. I didn’t find any arrow screaming, sorry about that.
\n\nLet’s talk about these dreaded scenarios in our next post about software testing!
\n\nI hope you enjoyed this one. See you soon!
\n\n