Transformations

Sometimes JSONPath and JSON queries are not sufficient for your use case. In these cases, you can use the transformation pipes provided by the fs2.data.json.ast.transform package.

Selectors

Selectors can be used to select a subset of a JSON token stream. There are several ways to create selectors:

Parsing a string using the selector syntax

For instance, to select and enumerate elements that are in the field3 array, you can create this selector. Only the tokens describing the values in field3 will be emitted as a result.

import fs2._
import fs2.data.json._

type ThrowableEither[T] = Either[Throwable, T]

val selector = ".field3.[]".parseSelector[ThrowableEither].toTry.get
// selector: Selector = PipeSelector(
//   left = NameSelector(
//     pred = Single(name = "field3"),
//     strict = true,
//     mandatory = false
//   ),
//   right = IteratorSelector(strict = true)
// )

The parseSelector method implicitly comes from the import fs2.data.json._ and wraps the result in anything that has an MonadError with error type Throwable to catch potential parsing errors. If you prefer not to have this wrapping and don't mind an extra dependency, you can have a look at the interpolator.

The filter syntax is as follows:

Using the selector DSL

The selector DSL is a nice way to describe selectors without using any string parsing. They also allow for programmatically building selectors. The DSL resides within the fs2.data.json.selector package, and you start a selector using the root builder. The selector above can be written like this with the DSL:

import fs2.data.json.selector._

val selectorFromDsl = root.field("field3").iterate.compile
// selectorFromDsl: Selector = PipeSelector(
//   left = NameSelector(
//     pred = Single(name = "field3"),
//     strict = true,
//     mandatory = false
//   ),
//   right = IteratorSelector(strict = true)
// )

The .compile in the end transforms the previous selector builder from the DSL into the final selector. Builders are safe to reuse, re-compose and compile several times.

You can express the same selectors as with the syntax described above. For instance to make the field mandatory and the iteration lenient you can do:

val selectorFromDsl = root.field("field3").!.iterate.?.compile
// selectorFromDsl: Selector = PipeSelector(
//   left = NameSelector(
//     pred = Single(name = "field3"),
//     strict = true,
//     mandatory = true
//   ),
//   right = IteratorSelector(strict = false)
// )

The DSL is typesafe, so that you cannot write invalid selectors. Any attempt to do so results in a compilation error.

// array index selection cannot be made mandatory
root.index(1).!
// error: Only not yet mandatory field selectors can be made mandatory
// root.index(1).!
//               ^

Parsing a string using the selector interpolator

Module: Maven Central

The fs2-data-json-interpolators module provides users with some useful string interpolators. The interpolators are based on literally and are statically checked.

You can use the selector interpolator to parse a literal string.

The example above can be rewritten as:

import fs2.data.json.interpolators._

val selector = selector".field3.[]"
// selector: Selector = PipeSelector(
//   left = NameSelector(
//     pred = Single(name = "field3"),
//     strict = true,
//     mandatory = false
//   ),
//   right = IteratorSelector(strict = true)
// )

Using the selectors

All the pipes in this package are based on a selector, a Builder, and a Tokenizer.

If you provide an implicit Tokenizer, which describes how a JSON AST is transformed into JSON tokens, you can apply transformations to the JSON stream. For instance, you can apply a function fun to all values in the fields3 array by using this code:

import ast._

trait SomeJsonType

implicit val builder: Builder[SomeJsonType] = ???
implicit val tokenizer: Tokenizer[SomeJsonType] = ???

def fun(json: SomeJsonType): SomeJsonType = ???

val stream: Stream[Fallible, Token] = ???

stream.through(transform[Fallible, SomeJsonType](selector, fun))

For concrete examples of provided Builders and Tokenizers, please refer to the JSON library binding modules documentation.

Sometimes you would like to delete some Json values from the input stream, based o some predicate at a given path, and keep the rest untouched. In this case, you can use the transformOpt pipe, and return None for values you want to remove from the stream.