Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: improve memory usage of record import #218

Merged
merged 27 commits into from
Feb 21, 2023
Merged

feat: improve memory usage of record import #218

merged 27 commits into from
Feb 21, 2023

Commits on Jan 31, 2023

  1. wip

    @tasshi-me
    tasshi-me committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    311bf75 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2023

  1. Revert "wip" 

    This reverts commit 311bf75.
    @tasshi-me
    tasshi-me committed Feb 1, 2023
    Configuration menu
    Copy the full SHA
    aea3a4c View commit details
    Browse the repository at this point in the history

Commits on Feb 4, 2023

  1. iterative CSV parser

    @tasshi-me
    tasshi-me committed Feb 4, 2023
    Configuration menu
    Copy the full SHA
    141c544 View commit details
    Browse the repository at this point in the history

Commits on Feb 11, 2023

  1. initialize stream when reader called

    @tasshi-me
    tasshi-me committed Feb 11, 2023
    Configuration menu
    Copy the full SHA
    53935f7 View commit details
    Browse the repository at this point in the history

  2. fix addRecords

    @tasshi-me
    tasshi-me committed Feb 11, 2023
    Configuration menu
    Copy the full SHA
    3829ac2 View commit details
    Browse the repository at this point in the history

  3. fix upsertRecords

    @tasshi-me
    tasshi-me committed Feb 11, 2023
    Configuration menu
    Copy the full SHA
    0cbc06f View commit details
    Browse the repository at this point in the history

  4. debug

    @tasshi-me
    tasshi-me committed Feb 11, 2023
    Configuration menu
    Copy the full SHA
    18705f7 View commit details
    Browse the repository at this point in the history

Commits on Feb 14, 2023

  1. Merge branch 'main' into feat/improve-memory-usage-of-import

    @tasshi-me
    tasshi-me committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    980de5d View commit details
    Browse the repository at this point in the history

  2. Revert "debug" 

    This reverts commit 18705f7.
    @tasshi-me
    tasshi-me committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    7a77a54 View commit details
    Browse the repository at this point in the history

  3. propagate error between steams

    @tasshi-me
    tasshi-me committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    8cede92 View commit details
    Browse the repository at this point in the history

  4. remove recordIndex from LocalRecord

    @tasshi-me
    tasshi-me committed Feb 14, 2023
    Configuration menu
    Copy the full SHA
    28a002d View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2023

  1. apply review: don't use generics for NodeJS.ReadableStream because it… 

    … is IF
    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    19652c8 View commit details
    Browse the repository at this point in the history

  2. separate asyncGeneratorWithIndexAndNextValueFromStream to two helpers

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    fe1bbd6 View commit details
    Browse the repository at this point in the history

  3. remove unnecessary async

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    8e3a726 View commit details
    Browse the repository at this point in the history

  4. create and refactor iterator utils

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    0336314 View commit details
    Browse the repository at this point in the history

  5. add types to fields of LocalRecordRepository impls

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    3f446be View commit details
    Browse the repository at this point in the history

  6. remove debug console.log

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    8968307 View commit details
    Browse the repository at this point in the history

  7. remove file format validation from openFsStreamWithEncode()

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    341b77d View commit details
    Browse the repository at this point in the history

  8. fix typo

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    fb0b2ea View commit details
    Browse the repository at this point in the history

  9. remove unused imports

    @tasshi-me
    tasshi-me committed Feb 15, 2023
    Configuration menu
    Copy the full SHA
    262505d View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2023

  1. remove unnecessary callback

    @tasshi-me
    tasshi-me committed Feb 16, 2023
    Configuration menu
    Copy the full SHA
    68b10e5 View commit details
    Browse the repository at this point in the history

  2. remove unnecessary input from LocalRecordRepositoryMock

    @tasshi-me
    tasshi-me committed Feb 16, 2023
    Configuration menu
    Copy the full SHA
    70cb9bf View commit details
    Browse the repository at this point in the history

  3. use withIndex() on validateUpdateKeyInRecords

    @tasshi-me
    tasshi-me committed Feb 16, 2023
    Configuration menu
    Copy the full SHA
    d8b6abb View commit details
    Browse the repository at this point in the history

  4. remove unknow as the default type of generics

    @tasshi-me
    tasshi-me committed Feb 16, 2023
    Configuration menu
    Copy the full SHA
    c2f3133 View commit details
    Browse the repository at this point in the history

Commits on Feb 21, 2023

  1. Merge remote-tracking branch 'origin/main' into feat/improve-memory-u… 

    …sage-of-import
    @tasshi-me
    tasshi-me committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    ec4882d View commit details
    Browse the repository at this point in the history

  2. move iterator utils to common directory

    @tasshi-me
    tasshi-me committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    e5721db View commit details
    Browse the repository at this point in the history

  3. remove unused import

    @tasshi-me
    tasshi-me committed Feb 21, 2023
    Configuration menu
    Copy the full SHA
    9d44cb7 View commit details
    Browse the repository at this point in the history