add asymmetric scale functions and some comparison results

[joe-sfx]

Modifies a scale function k_* to a version which is asymmetric about the median by declaring the new scale function agrees with k_* on [p, 1] (for some p between 0 and 1) and is the linearization of k_* at p on [0, p]. (Similarly with the roles of 0, 1 reversed.) The proof that the resulting t-digest is mergeable is contained elsewhere (in a research article).
Empirical results demonstrate the asymmetric t-digest retains the error profile of the original scale function on one side of the distribution and is comparatively less accurate (and less memory/CPU-intensive) on the other side. This allows for a more calibrated tradeoff between computational resources and accuracy.

[tdunning]

Joe, Can you add your proofs of invariance and bounded size to the existing documents?

…

On Wed, May 13, 2020 at 8:47 PM Joe Ross ***@***.***> wrote: Modifies a scale function k_* to a version which is asymmetric about the median by declaring the new scale function agrees with k_* on [p, 1] (for some p between 0 and 1) and is the linearization of k_* at p on [0, p]. (Similarly with the roles of 0, 1 reversed.) The proof that the resulting t-digest is mergeable is contained elsewhere (in a research article). Empirical results demonstrate the asymmetric t-digest retains the error profile of the original scale function on one side of the distribution and is comparatively less accurate (and less memory/CPU-intensive) on the other side. This allows for a more calibrated tradeoff between computational resources and accuracy. ------------------------------ You can view, comment on, or merge this pull request online at: #141 Commit Summary - add asymmetric scale functions and some comparison results File Changes - *M* core/src/main/java/com/tdunning/math/stats/MergingDigest.java <https://github.com/tdunning/t-digest/pull/141/files#diff-918a0eb01eb026c6454f0a518808a086> (7) - *M* core/src/main/java/com/tdunning/math/stats/ScaleFunction.java <https://github.com/tdunning/t-digest/pull/141/files#diff-64faa827235f5a1060b2a1075a1805da> (290) - *M* core/src/test/java/com/tdunning/math/stats/MergingDigestTest.java <https://github.com/tdunning/t-digest/pull/141/files#diff-5099223ec9270096970f83af9798ef98> (99) - *M* core/src/test/java/com/tdunning/math/stats/TDigestTest.java <https://github.com/tdunning/t-digest/pull/141/files#diff-a258b6d2d5959826586c19f463d15726> (76) - *A* docs/asymmetric/README.md <https://github.com/tdunning/t-digest/pull/141/files#diff-d9d2ed65486d8ea557593a8647fe2579> (34) - *A* docs/asymmetric/generate_plots.py <https://github.com/tdunning/t-digest/pull/141/files#diff-fff14ebe623851c74bcbf3f070a457f1> (104) - *A* docs/asymmetric/plots/merging/t_digest_figs_K_0q.png <https://github.com/tdunning/t-digest/pull/141/files#diff-b7430aefbed39255cce547d1380868f2> (0) - *A* docs/asymmetric/plots/merging/t_digest_figs_K_1.png <https://github.com/tdunning/t-digest/pull/141/files#diff-8df63cb9b2e0d197d106645f18a5dc89> (0) - *A* docs/asymmetric/plots/merging/t_digest_figs_K_2.png <https://github.com/tdunning/t-digest/pull/141/files#diff-65be861ebae8c082aa4741a19ab94cb0> (0) - *A* docs/asymmetric/plots/merging/t_digest_figs_K_3.png <https://github.com/tdunning/t-digest/pull/141/files#diff-c04b77e972056c1048cf6be56008a5a4> (0) - *A* docs/asymmetric/plots/tree/t_digest_figs_K_0q.png <https://github.com/tdunning/t-digest/pull/141/files#diff-42b4698a91e65c9b65cd51b7c7d0cf6e> (0) - *A* docs/asymmetric/plots/tree/t_digest_figs_K_1.png <https://github.com/tdunning/t-digest/pull/141/files#diff-9598ff59362d33a5d3479c430f185211> (0) - *A* docs/asymmetric/plots/tree/t_digest_figs_K_2.png <https://github.com/tdunning/t-digest/pull/141/files#diff-2560f0d916bfc94792b790c86a3fa76a> (0) - *A* docs/asymmetric/plots/tree/t_digest_figs_K_3.png <https://github.com/tdunning/t-digest/pull/141/files#diff-cad6790b5a6b791e43073d9598dd349e> (0) Patch Links: - https://github.com/tdunning/t-digest/pull/141.patch - https://github.com/tdunning/t-digest/pull/141.diff — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#141>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB5E6QKV5IVP2J65JS3RIDRRNSWZANCNFSM4NAKF74Q> .

[joe-sfx]

Added preprint with proofs, and fixed some links in the README.

[tdunning]

Joe,

I have thought about this a bit and it seems like a reasonable update. I am clearing the decks for a new release and the release after that would be a good time for this to go in.

Can you incorporate your proof into the existing proof papers in this distro?

[joe-sfx]

Sounds good to me. Do you have an approximate timeline for the release that this would go in?

[tdunning]

I am trying to clear the backlog now for a release. There are some important things that need to go out the door.

…

On Mon, Sep 14, 2020 at 5:13 PM Joe Ross ***@***.***> wrote: Sounds good to me. Do you have an approximate timeline for the release that this would go in? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#141 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB5E6VZFGC43GD5SJPK6W3SF2WQ7ANCNFSM4NAKF74Q> .