Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Split apply combine #45

Draft
wants to merge 53 commits into
base: legacy
Choose a base branch
from
Draft

Split apply combine #45

wants to merge 53 commits into from

Commits on Jun 23, 2024

  1. map reduce yaml mock up

    @cisaacstern
    cisaacstern committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    719202b View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. split apply combine mock up

    @cisaacstern
    cisaacstern committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    6e416b6 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Merge remote-tracking branch 'origin/main' into split-apply-combine

    @cisaacstern
    cisaacstern committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    d48cb1c View commit details
    Browse the repository at this point in the history

  2. fill in annotations for importability

    @cisaacstern
    cisaacstern committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    4fc63f4 View commit details
    Browse the repository at this point in the history

  3. assign from column attribute

    @cisaacstern
    cisaacstern committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    648dd19 View commit details
    Browse the repository at this point in the history

  4. add test for assign from col attr

    @cisaacstern
    cisaacstern committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    1042665 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Merge remote-tracking branch 'origin/assign-from-column-attribute' in… 

    …to split-apply-combine
    @cisaacstern
    cisaacstern committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    313f99c View commit details
    Browse the repository at this point in the history

  2. decorator typing upgrades

    @cisaacstern
    cisaacstern committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    b574b93 View commit details
    Browse the repository at this point in the history

  3. lithops wip cont.

    @cisaacstern
    cisaacstern committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    0899800 View commit details
    Browse the repository at this point in the history

  4. update typevar name

    @cisaacstern
    cisaacstern committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    1a64abe View commit details
    Browse the repository at this point in the history

  5. add return postvalidator typevar

    @cisaacstern
    cisaacstern committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    c370c07 View commit details
    Browse the repository at this point in the history

  6. lithops demo wip., serialize groupby next

    @cisaacstern
    cisaacstern committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    466d54a View commit details
    Browse the repository at this point in the history

  7. lithops demo wip., serialize groupby next

    @cisaacstern
    cisaacstern committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    694304b View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. pull in everything

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    dd91149 View commit details
    Browse the repository at this point in the history

  2. pull in everything

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    9d50eb3 View commit details
    Browse the repository at this point in the history

  3. generate hivekeys

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    205dd9a View commit details
    Browse the repository at this point in the history

  4. time density from synthetic data woohoo

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    0db527e View commit details
    Browse the repository at this point in the history

  5. reformat params

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    4318de0 View commit details
    Browse the repository at this point in the history

  6. aggregate map works

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    18b1d72 View commit details
    Browse the repository at this point in the history

  7. calling map_function on single element works

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    d4d7d65 View commit details
    Browse the repository at this point in the history

  8. wow map reduce works locally in a basic way

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    fb64c2a View commit details
    Browse the repository at this point in the history

  9. ok, _now_ we have an actually collection

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    c050d07 View commit details
    Browse the repository at this point in the history

  10. actually, gather_widget

    @cisaacstern
    cisaacstern committed Jun 27, 2024
    Configuration menu
    Copy the full SHA
    11a5fbb View commit details
    Browse the repository at this point in the history

Commits on Jun 28, 2024

  1. combine map_function into single def

    @cisaacstern
    cisaacstern committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    c1a9291 View commit details
    Browse the repository at this point in the history

  2. clean up reducer

    @cisaacstern
    cisaacstern committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    b3e5ba5 View commit details
    Browse the repository at this point in the history

  3. cleanup and incease number of groups

    @cisaacstern
    cisaacstern committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    e753c4a View commit details
    Browse the repository at this point in the history

  4. move airflow into backends

    @cisaacstern
    cisaacstern committed Jun 28, 2024
    Configuration menu
    Copy the full SHA
    a5d94d0 View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. just use strings for path in script

    @cisaacstern
    cisaacstern committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    7363876 View commit details
    Browse the repository at this point in the history

  2. use fsspec

    @cisaacstern
    cisaacstern committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    f5e7ec0 View commit details
    Browse the repository at this point in the history

  3. add lithops

    @cisaacstern
    cisaacstern committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    30c99c2 View commit details
    Browse the repository at this point in the history

  4. move backends to dotdir

    @cisaacstern
    cisaacstern committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    296c824 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. add gcsfs to dockerfile

    @cisaacstern
    cisaacstern committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    74019d7 View commit details
    Browse the repository at this point in the history

  2. move split groups logic into task

    @cisaacstern
    cisaacstern committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    e21b4c4 View commit details
    Browse the repository at this point in the history

  3. type alias for filter

    @cisaacstern
    cisaacstern committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    6c6dbb0 View commit details
    Browse the repository at this point in the history

  4. type alias for composite filter

    @cisaacstern
    cisaacstern committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    76fed9f View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2024

  1. WIP

    @cisaacstern
    cisaacstern committed Jul 8, 2024
    Configuration menu
    Copy the full SHA
    07002e2 View commit details
    Browse the repository at this point in the history

Commits on Jul 9, 2024

  1. cleanup script for committing it

    @cisaacstern
    cisaacstern committed Jul 9, 2024
    Configuration menu
    Copy the full SHA
    55d86e4 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2024

  1. key groups by widget and then case match

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    2c26ab3 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into split-apply-combine

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    3c339db View commit details
    Browse the repository at this point in the history

  3. use new ecomap

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    1d62304 View commit details
    Browse the repository at this point in the history

  4. merge widgets into dashboard

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    8071c44 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'origin/main' into split-apply-combine

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    3adc825 View commit details
    Browse the repository at this point in the history

  6. add ecoplot placeholder

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    a516563 View commit details
    Browse the repository at this point in the history

  7. parse urls to public https

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    a7cd971 View commit details
    Browse the repository at this point in the history

  8. public results urls

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    dc59b7c View commit details
    Browse the repository at this point in the history

  9. lots of dashboard updates

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    765c898 View commit details
    Browse the repository at this point in the history

  10. new dashboard runs e2e

    @cisaacstern
    cisaacstern committed Jul 10, 2024
    Configuration menu
    Copy the full SHA
    30cb7e2 View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2024

  1. metadata, enumerate widget ids, custom serializer

    @cisaacstern
    cisaacstern committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    de2eb57 View commit details
    Browse the repository at this point in the history

  2. revert plots oops

    @cisaacstern
    cisaacstern committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    b18eb3c View commit details
    Browse the repository at this point in the history

  3. metadata defaults

    @cisaacstern
    cisaacstern committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    bd93bac View commit details
    Browse the repository at this point in the history

  4. parse groupers to filters, layout placeholder

    @cisaacstern
    cisaacstern committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    eb7399d View commit details
    Browse the repository at this point in the history

  5. lint

    @cisaacstern
    cisaacstern committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    1bc504b View commit details
    Browse the repository at this point in the history

  6. dump dashboard to json

    @cisaacstern
    cisaacstern committed Jul 11, 2024
    Configuration menu
    Copy the full SHA
    a698676 View commit details
    Browse the repository at this point in the history