fs2-data - JSON Patch

Module:

The fs2-data-json-diffson module provides some integration with diffson.
It allows for patching a Json stream as it is read to emit the patched value downstream.
Patching stream can be useful in several case, for instance:

it can be used to filter out fields you don’t need for further processing, before building an AST with these fields;
it can be used to make data from an input stream anonymous by removing names or identifiers;
it makes it possible to enrich an input stream with extra data you need for further processing;
many other use cases when you need to amend input data on the fly without building the entire AST in memory.

Currently only JSON Merge Patch is supported.

In order for patches to be applied, you need a Tokenizer for some Json type the patch operates on (see above) and a Jsony from diffson for that same Json type.

Let’s say you are using circe as Json AST library, you can use patches like this:

import fs2.{Fallible, Stream}
import fs2.data.json._
import fs2.data.json.circe._
import fs2.data.json.mergepatch._

import diffson._
import diffson.circe._
import diffson.jsonmergepatch._

import io.circe._

val input = """{
              |  "field1": 0,
              |  "field2": "test",
              |  "field3": [1, 2, 3]
              |}
              |{
              |  "field1": 2,
              |  "field3": []
              |}""".stripMargin
// input: String = """{
//   "field1": 0,
//   "field2": "test",
//   "field3": [1, 2, 3]
// }
// {
//   "field1": 2,
//   "field3": []
// }"""

val stream = Stream.emits(input).through(tokens[Fallible, Char])
// stream: Stream[[A]Fallible[A], Token] = Stream(..)

// a patch that removes `field3`
val mergePatch = JsonMergePatch.Object(Map("field3" -> Json.Null))
// mergePatch: JsonMergePatch.Object[Json] = Object(
//   fields = Map("field3" -> null)
// )

val patched = stream.through(patch[Fallible, Json](mergePatch))
// patched: Stream[[x]Fallible[x], Token] = Stream(..)

patched.compile.toList
// res0: Either[Throwable, List[Token]] = Right(
//   value = List(
//     StartObject,
//     Key(value = "field1"),
//     NumberValue(value = "0"),
//     Key(value = "field2"),
//     StringValue(value = "test"),
//     EndObject,
//     StartObject,
//     Key(value = "field1"),
//     NumberValue(value = "2"),
//     EndObject
//   )
// )