Composing parsers and decoders, parse overload selection

[kamcma]

Hello.

I am experimenting with trying to compose a pipeline of some existing Decodable-based decoding and some swift-parsing-based parsing. I looked at the Conversions.JSON implementation, but in my scenario, I only have Decodable conformance, so I can't implement a bi-directional conversion.

What I've come up with is this:

extension FooDecodableType {
  static func parser() -> Parsers.AnyParser<Data, Self> {
    .init { (input: inout Data) -> Self in
      try FooDecoder.static.decode(Self.self, from: input)
    }
  }
}

extension BarDecodableType {
  static func parser() -> Parsers.AnyParser<Data, Self> {
    .init { (input: inout Data) -> Self in
      try BarDecoder.static.decode(Self.self, from: input)
    }
  }
}

enum Either {
  case foo(FooDecodableType)
  case bar(BarDecodableType)
}

let eitherParser = OneOf {
  Parse(Either.foo) {
    FooDecodableType.parser()
  }
  Parse(Either.bar) {
    BarDecodableType.parser()
  }
}

At this point, I would like to call:

let myData: Data = // ...
let parsed = try eitherParser.parse(data)

I would like to call the following overload, for the ergonomics and because I'm not interested in unconsumed input:

swift-parsing/Sources/Parsing/Parser.swift

Lines 43 to 67 in 4932348

	/// Parse an input value into an output. This method is more ergonomic to use than
	/// ``parse(_:)-76tcw`` because the input does not need to be inout.
	///
	/// Rather than having to create a mutable input value and feed it to the ``parse(_:)-76tcw``
	/// method like this:
	///
	/// ```swift
	/// var input = ...
	/// let output = try parser.parse(&input)
	/// ```
	///
	/// You can just feed the input directly:
	///
	/// ```swift
	/// let output = try parser.parse(input)
	/// ```
	///
	/// - Parameter input: A nebulous piece of data to be parsed.
	/// - Returns: A more well-structured value parsed from the given input.
	@_disfavoredOverload
	@inlinable
	public func parse(_ input: Input) rethrows -> Output {
	var input = input
	return try self.parse(&input)
	}

But Swift selects this overload, I presume because Data conforms to Collection:

swift-parsing/Sources/Parsing/Parser.swift

Lines 69 to 136 in 4932348

	/// Parse a collection into an output using a parser that works on the collection's `SubSequence`.
	/// This method is more ergnomic to use than ``parse(_:)-76tcw`` because it accepts a
	/// collection directly rather than its subsequence, and the input does not need to be `inout`.
	///
	/// Rather than having to create a mutable subsequence value, such as a `Substring`, and feed it
	/// to the ``parse(_:)-76tcw`` method like this:
	///
	/// ```swift
	/// var input = "123,true"[...]
	/// let output = try Parse {
	/// Int.parser()
	/// ","
	/// Bool.parser()
	/// }
	/// .parse(&input) // (123, true)
	/// ```
	///
	/// You can just feed a plain `String` input directly:
	///
	/// ```swift
	/// let output = try Parse {
	/// Int.parser()
	/// ","
	/// Bool.parser()
	/// }
	/// .parse("123,true") // (123, true)
	/// ```
	///
	/// This method will fail if the parser does not consume the entirety of the input.
	/// For example:
	///
	/// ```swift
	/// let output = try Parse {
	/// Int.parser()
	/// ","
	/// Bool.parser()
	/// }
	/// .parse("123,true ")
	///
	/// // error: unexpected input
	/// // --> input:1:9
	/// // 1 | 123,true␣␣␣␣
	/// // | ^ expected end of input
	/// ```
	///
	/// > Tip: If your input can have trailing whitespace that you would like to consume and discard
	/// > you can do so like this:
	/// > ```swift
	/// > let output = try Parse {
	/// > Int.parser()
	/// > ",".utf8
	/// > Bool.parser()
	/// > Whitespace()
	/// > }
	/// > .parse("123,true ") // (123, true)
	/// > ```
	///
	/// - Parameter input: A nebulous collection of data to be parsed.
	/// - Returns: A more well-structured value parsed from the given input.
	@inlinable
	public func parse<C: Collection>(_ input: C) rethrows -> Output
	where Input == C.SubSequence {
	var input = input[...]
	return try Parse {
	self
	End<Input>()
	}.parse(&input)
	}

Processing my input Data as a Collection like this causes parsing to fail. If I forgo the ergonomic overload, and pass my Data as an inout parameter, parsing suceeds:

var myData: Data = // ...
let parsed = try eitherParser.parse(&data)

So, on one level, I suppose I'm asking about overload selection from libraries and if there's a way to elect the overload I want.

On a broader level, though, I wonder what people think about composing decoding and parsing like this, or if maybe there's another way to approach this that would obviate the overload issue. Maybe I should put the decoding in a map parser's transform rather than wrapping it in a parser per se, or something? Might need a Parsers.TryMap implementation for that.

Thanks for any thoughts!

[kamcma]

Thinking on it some more, I am leaning towards decoding in a transform is indeed the idiomatic integration with swift-parsing. Conversion's apply and unapply are allowed to throw, but Parsers.Map's transform cannot, so I'll take a stab at implementing a Parsers.TryMap and see if that gets me anywhere.

[randomeizer]

I have pitched a variation of map that can throw, which @stephencelis and @mbrandonw are not opposed to in concept. Just hasn't made it into the library yet.

All that said, it does seem to fit into transformation, since map, in and of itself, is a one-way operation.

[stephencelis]

@kamcma Is there a reason your parsers aren't consuming the input? Parsers are supposed to incrementally consume the data handed to them, so you might want to call input.removeAll() after successfully decoding those types.

Alternately, you may want to use a conversion here, which better describe non-incremental transformations (and is what we do with JSON encoding/decoding). I understand you may not care about creating full parser-printer here, though, and maybe there is no encoder available to use conversions instead. But if you do, you could use the conversion in MapConversion.

[kamcma]

@kamcma Is there a reason your parsers aren't consuming the input? Parsers are supposed to incrementally consume the data handed to them, so you might want to call input.removeAll() after successfully decoding those types.

Just me utterly forgetting the basics while I was experimenting with this 🤦. Thanks.

Alternately, you may want to use a conversion here, which better describe non-incremental transformations (and is what we do with JSON encoding/decoding).

Ah, a distinction between parsing per se being incremental and conversions being non-incremental (if I'm interpreting that right) is clarifying, I think. I think that reinforces that I'll try to do this in a conversion.

Thanks for the pointers.