-
Notifications
You must be signed in to change notification settings - Fork 401
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[discussion] make generated JsonCodec re-usable in other code #3070
Comments
I understand the goal of using the same codec at two places. And it is a good use case. I think the first thing I'd change about how to approach this is, that I'd not try to get a While I think there needs to be some R&D to find a nice solution, the feature request is valid. |
I'm new to Based on my research and practice it would seem like However, using those You know, there is no I would admit I don't completely understand, so maybe I'm misplaced but I don't think so. I am very thankful for |
By the way, I understand for certain pristine projects that might be preferred, but for me in my experiences what I usually want is to be able to get something working as quickly as possible. Worrying about revealing json implementation details is usually only a nice-to-have way at the end. |
@decoursin There is no intention and will never be to support non schema based codecs directly in zio-https Endpoint API. The Endpoint API is schema based. But, we use under the hood zio-json. I used both and know both well. And I'd say zio schema is more powerful in the simple use case. And you should not use manual schemas. But please, add more details and let me understand what your use case is. Then we can find a solution. I would assume, that there is either missing understanding or missing documentation for what you want to do. |
@987Nabil Yeah, I didn't mean to hijack the thread, but it seemed to me like there are similarities, and it was labeled as a discussion anyways. I understand that the val myOwnJsonHttpContentCodec =
HttpContentCodec(
ListMap(
MediaType.application.`json` ->
BinaryCodecWithSchema(
myOwnJsonCodec,
schema,
),
),
) The case class SomeCaseClass(field: String)
implicit lazy val zioEncoderSomeCaseClass: JsonEncoder[SomeCaseClass] =
DeriveJsonEncoder.gen[SomeCaseClass]
implicit lazy val zDecodeSomeCaseClass: JsonDecoder[SomeCaseClass] =
DeriveJsonDecoder.gen[SomeCaseClass]
val myOwnJsonCodec = zio.schema.codec.JsonCodec.zioJsonBinaryCodec[SomeCaseClass] When you do that in combination with val somethingRoute = Endpoint(Method.GET / Root / "something")
.out[SomeCaseClass]
.implement(_ => ZIO.succeed(SomeCaseClass("hi"))) This works the same was Tapir. In Tapir, the schema can be defined separately from the json encoding/decoding. This would also solve @myazinn's problem, because then he would be able to seamlessly use his own json codec (as he said himself). So it's really the same problem in my opinion, and I haven't hijacked the thread at all I would say. |
To me it's bewildering that I can't seamlessly use zio-json encoding/decoding with zio HTTP. I can even do with that Tapir. But I don't mean to disrespect the project, I love zio, I just think it would be a very crucial and important addition, to be able to more seamlessly support |
@decoursin why do you want to derive the json codec in the first place? Why not just use the default schema based json codec? |
Tapir is not using zio schema. And I do not understand why you would like to use zio json anyway. There should be no reason |
the goal should be, that there is no need for non schema based codecs in almost all cases |
With With I can do dynamically logging with
This is part of the problem. I need to use manual schemas, and manual schema suck. I don't want to write the names of the fields in a "string" for example and the names of the case classes. I have to write my case classes to match 3rd party APIs that cause all sorts of discrepancies. I'm not going to make all these unnecessary transformations, and multiple case classes for one datatype just to parse json, if that were even possible with zio-schema. I need to not just parse it but also render it as well. |
These goals are fairy tale goals that don't respective real programmer needs, or the convenience and reliability of working directly with the data interchange format (i.e. json). I don't want to be dependent on some other library to generate my json into some other library (from zio-schema to zio-json) that then generates my json, I want to generate it directly from my json library; I don't want middlemen in between my middlemen, I just want to work right with the source. |
Does |
This is not true. Turing complete is a attribute of a programming language. Neither are languages. Neither are are turing complete.
Please make a concrete example. This is an abstract description. Show me the code.
Again, code please. And even if true, the Endpoint API has a clear and narrow purpose. Feel free to use the lowlevel API, if you don't like the constraints.
No idea what this should mean.
Yes, schema has a lot of use cases
Correct. That's why with schema you describe mainly data that is already represented by a case class. Then you get for free codecs for many different formats that represent the same structure as the case class. If you case class does not fit the outbound structure, create a new one and map between those two. Or agin, don't use the Endpoint API but the low level API where you can do what you want. The Endpoint API is opinionated by design
Yes, manual schema generation sucks. Because it is a last resort and not the way to go.
Ofc this transformation is possible. You also mentioned Tapir can do what you need. Tapir is great. Use Tapir 🙂 |
Then don't use the Endpoint API |
You are entitled to your own opinion. No user of the Endpoint API had such issues so far. Also you did not provide me a clear code example of your use case. But it sounds to me, that you just don't want to use the Endpoint API in its intended way. |
I would like to better understand what's being requested. At first glance, what is being requested is:
Okay, before we make that possible, it would be nice to understand, "Why?" I think I have collected some reasons:
Did I miss any reason? Did I miss any specific feature being requested? Once we have a complete list, it will be possible to come up with a solution that balances the tradeoffs involved. |
@decoursin If I understand you correctly, you want to be able to start with a And, furthermore, you want to do this because |
Hi @jdegoes thank you very much for offering your time on this issue. I am very thankful and deeply impressed by the community and libraries that you have built. I wouldn't want this issue to distract from my gratitude towards you and your community including @987Nabil. I don't mean to be aggressive, just purposely constructive, and whether or not this issue gets resolved in a matter that would satisfy me, it wouldn't deter from being sincerely thankful for everything else you've built (which I myself haven't yet materially profited from just fyi but hopefully I may in the future, because I know it's very powerful - I'm in the process of starting my own company with your software.)
Yeah, pretty much, but it wouldn't have to only be like that. I could imagine other solutions, but I don't know the internals of
Yeah, and more powerful actually which I think that's what you mean by For example, I have an API that I work with where I need to output Another example, I'm using Stripe and constantly digesting their webhooks to keep a local copy of their data in perfect synchrony. They're constantly making changes to their data models, which causes parsing problems, I want to be able to log these parsing problems immediately as they appear. Stripe also does things where they use the same field for different types. For example, the subscription.customer is expandable so sometimes it's a string, sometimes it's an object. I really don't know if writing a manual constructor for that using
case class HappyClass(message: String, state: State) and you want to be able to generate multiple fields based on the To me, I can only imagine what other All of these things and others are because of fundamental limitations on |
@jdegoes for me the main reason was 2) So right now we have services that produce to / consume from Kafka, and also provide some request-response features using custom in-house tool. For all of that we use JSON as a data format and Here's the list of features that we were looking for in HTTP libraries
We tried to use Regardless, in a few days after I opened it, we realized that it's not how we want to do things. We decided to stick to schema-first approach. In that case,
With a bit of code we could integrate I don't want to turn it into "X is better than Y", and perhaps |
@987Nabil thank you! Somehow I missed it. That changes a lot for us 🙂 |
@myazinn would you use |
@decoursin we haven't decided fully yet, so yes, probably so. |
That's awesome @myazinn, thanks for sharing! |
Is your feature request related to a problem? Please describe.
Let's say I have a code that looks like this
For my model, schema acts like the source of truth. The problem is that it's not the only place where I'd like to use the same codec for this model. For example, we sent this data to Kafka topic, but also want to provide an HTTP endpoint to manually fetch it if needed. So I would like to keep them as consistent as possible to ensure they're the same everywhere.
Describe the solution you'd like
I'd like to be able to easily and safely "extract" the generated json codec from
HttpContentCodec
.Describe alternatives you've considered
I can think of only three solutions:
myModelHttpContentCodec
with my own. Something like thisThat could also work, but I'd have to copy-paste some code
3) Extract it from HttpContentCodec directly, e.g.
While that involves the least amount of code, there are other issues. First, it's not really safe. While
lookup
is a public API, its internals feel like an implementation detail which I shouldn't rely on. Also, it provides a BinaryCodec fromzio-schema
, even though under the hood it uses a codec fromzio-json
. If I need to usezio-json
somewhere, I'd have to make a wrap fromzio-schema
BinaryCodec tozio-json
BinaryCodec. It's not a huge deal, but still.Additional context
Perhaps it's a stupid question and I'm missing something / doing something wrong. Is there no other easy way to do it? That sound like something that should be quite common and easy to do. I know that this
HttpContentCodec
is supposed to be used not only with json, but I've got a feeling that most of the time it will actually be a simple json. So making a special case for such a common task doesn't sound too bad to me.I guess it could be fixed with addition of something like
updated
toHttpContentCodec
, so that it could be used like thisThough it's still not perfect since I'll have no idea if the codec was there in the first place and therefore will actually be used. And I'll also have to have
myOwnJsonHttpContentCodec
defined manually.The text was updated successfully, but these errors were encountered: