Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory leak for using atoms as keys #256

Open
matehat opened this issue Dec 19, 2016 · 4 comments
Open

Memory leak for using atoms as keys #256

matehat opened this issue Dec 19, 2016 · 4 comments

Comments

@matehat
Copy link

matehat commented Dec 19, 2016

Hello,

I've noticed that Tirexs will convert all keys (not just top-level, but deep in a record's structure) to atoms.

tirexs/lib/tirexs/http.ex

Lines 360 to 362 in f798d8c

def decode(json, opts \\ [{:labels, :atom}]) do
JSX.decode!(IO.iodata_to_binary(json), opts)
end

There is no way for a user of the library to indicate that keys should stay as binary, because some part of an indexed document is highly-dynamic.

As mentioned in multiple places around the Erlang community, like in this book,

Don’t use dynamic atoms! Atoms go in a global table and are cached forever. Look for places where you call erlang:binary_to_term/1 and erlang:list_to_atom/1, and consider switching to safer variants (erlang:binary_to_term(Bin, [safe]) and erlang:list_to_existing_atom/1).

Can you change the default behavior, or consider passing an option to opt out of this? Otherwise using this library makes any system vulnerable to memory exhaustion overtime.

@OpakAlex
Copy link
Collaborator

Hi @matehat. It's first good issue since this project started ;) Thanks a lot, i think we need use binary keys. If you want change this by PR will be good, or i can find time on holidays.

@matehat
Copy link
Author

matehat commented Dec 19, 2016

I (or someone else from my team) may create a PR soon to fix this. Thanks for your reply!

@OpakAlex
Copy link
Collaborator

thanks man! I will be happy merge it!

@jdlourenco
Copy link

Has anyone addressed this already?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants