Iterate through every key, value; even inner childs. Get full path of a key

[cptspartano]

Hi! I've got two simple questions:

1.- Imagine you have a JSON with some inner childs being another object, and some others inner inner childs being objects, and so on. Is there any way this package can iterate autonomously to get to all the simple key, value pairs of the JSON (meaning by simple value anyone that is not another object). For example, with a functioncall. I was trying to do it manually but I was ending with a lot of loops inside loops. My main need is to compare those simple key, value pairs between two JSON (in the way that both JSON should follow the same structure, but anyone could have some fields missing.

2.- Followed by the previous one, another question I have is: Given a key, it is possible to recover its full path, especially interested if it is from an inner (or inner inner, or inner inner inner, or...) child? That way I could assure in a way that I'm comparing the same childs between those two JSON.

I don't know if my questions are clear or not. Thanks in advance!

[mihaitodor]

Hey @cptspartano, sorry for the delayed response...

For 1. it sounds like you're looking for something like Bloblang, which makes use of some Gabs functions under the hood. See for example how I used it here: redpanda-data/connect#1036. You can import it into your own project like so: https://pkg.go.dev/github.com/benthosdev/benthos/[email protected]/public/bloblang#example-package-BloblangRestrictedEnvironment

For 2. maybe you can make use of https://pkg.go.dev/github.com/Jeffail/gabs/v2#Container.Flatten or https://pkg.go.dev/github.com/Jeffail/gabs/v2#Container.FlattenIncludeEmpty, but I think you want to get the ancestor path all the way to the root when given a child element, which doesn't seem to fit into the Gabs model, because each element only stores information about its children... What are you trying to achieve? If you need to diff documents, maybe something like https://pkg.go.dev/github.com/r3labs/diff/v3 might be more appropriate.

[cptspartano]

Hi! Right now I have don't have any needs since the question was because of a project I was working in a company. Since the contract ended, so did my relation with that project.

Anyway, just in case someone comes with the same questions, the idea was, given two JSON, compare them field by field, with the caveat (I don't know if that's well said) that both of them were dynamically created, so one could have some fields that the other didn't. So what I was thinking was to iterate through one of the JSON, and with each field, extract it's path, and use it with gabs (Search function) on the other JSON and compare both fields to see if they were the same. I don't know if it is clear.

Anyway, I will have a look when I can to the options you gave, just for personal knowledge. Thanks a lot!

[mihaitodor]

OK, when you say "dynamically created" do you mean that you already have a Go object instead of the actual JSON string? The efficient way to compare them would be to use something like https://pkg.go.dev/github.com/google/go-cmp/cmp#Equal or a similar package. There are many such packages out there so you don't have to roll your own, but yeah, you could if you wanted to. Otherwise, if you're starting with the JSON string, then definitely look into https://pkg.go.dev/github.com/r3labs/diff/v3 or https://pkg.go.dev/github.com/nsf/jsondiff or similar... Here's a collection of various JSON packages: https://github.com/avelino/awesome-go#json

[cptspartano]

Hi, sorry for the delay, I was on a trip and I wasn't paying attention to emails and all that.

I didn't know how to say it, so when I say "dynamically created" I'm saying that I'm not the one creating the JSON files by hand. Instead, they were being created outside the program (and sent to it), more or less in a random manner and, as I said, " one could have some fields that the other didn't". I don't know if that solves your question. Anyway, I don't think it's relevant for this thread, and I don't want to give you more headaches than necessary :D

[mihaitodor]

Oh, OK. No worries! Since you're dealing with JSON strings, I suggested a few libraries above that I think will help you achieve your goal.