-
Notifications
You must be signed in to change notification settings - Fork 80
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Utilities to assist in retrieving data from tables #2857
Open
rbasralian
wants to merge
45
commits into
deephaven:main
Choose a base branch
from
rbasralian:raffi_record_adapter
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
45 commits
Select commit
Hold shift + click to select a range
f971e58
Initial commit of table-to-record adapter/listener utility
rbasralian 6fb85ac
add QueryUtil to build
rbasralian 4b70eff
Update variable names
rbasralian 45902c1
Fix retrieval of multiple records with single keys
rbasralian 4f3ba84
Add Python version of KeyedRecordAdapter. (Currently only works with …
rbasralian afacf3d
spotless
rbasralian 6d567aa
remove dependency
rbasralian 591ceca
add dependency
rbasralian 4c96038
work on support for primitive keys from Python
rbasralian 0749ec0
fix bad 'isRefreshing' parameter in GetDataLockType. add not-yet-func…
rbasralian 3952474
fix bug for missing single records
rbasralian a626e19
fix(?) some snapshot source stuff
rbasralian 00851cd
Updates after merging `main`
rbasralian fe9cfc3
reduce visibility where possible
rbasralian b994f30
remove some outdated classes. simplify package structure.
rbasralian 422a74e
javadocs update
rbasralian ceb0bfd
remove an extra layer of method dispatch from RecordUpdater
rbasralian 68313eb
Fix DateTime -> Instant after rebase
rbasralian 62a630e
Handle multiple update graphs after rebase
rbasralian b6643cb
Don't use terminal listeners. (And switch from InstrumentedTableUpdat…
rbasralian fd1efca
Misc. test cleanup
rbasralian 6c640b0
Task to run select py tests
rbasralian 05daad2
rename module
rbasralian 14d8d1a
rename package
rbasralian 4dfd3d7
spotless
rbasralian be5f0bc
Support building jpy from local sources when running python tests
rbasralian e672c1b
fix array conversion after rebase
rbasralian c7f8223
Use as_jobj to convert Python values to the expected Java datatypes. …
rbasralian c79949c
Merge remote-tracking branch 'origin/main' into raffi_record_adapter
rbasralian b7aaae2
Merge remote-tracking branch 'origin/main' into raffi_record_adapter
rbasralian 4916a57
spotless (copyright)
rbasralian ec8309b
jpy.as_jobj renamed to jpy.convert
rbasralian 2f204c7
Merge remote-tracking branch 'origin/main' into raffi_record_adapter
rbasralian e9b1d5d
javadoc fix
rbasralian 95bcdf6
Merge remote-tracking branch 'origin/main' into raffi_record_adapter
rbasralian e59b117
Merge remote-tracking branch 'refs/remotes/origin/main' into raffi_re…
rbasralian 590b855
fix for QueryCompiler changes
rbasralian 26ac7b0
Merge remote-tracking branch 'refs/remotes/origin/main' into raffi_re…
rbasralian 9294dc3
Merge remote-tracking branch 'refs/remotes/origin/main' into raffi_re…
rbasralian 6745f95
Merge remote-tracking branch 'refs/remotes/origin/main' into raffi_re…
rbasralian d9a12bc
Merge remote-tracking branch 'refs/remotes/origin/main' into raffi_re…
rbasralian 81a4c71
update dependencies
rbasralian b58c7db
Merge remote-tracking branch 'refs/remotes/origin/main' into raffi_re…
rbasralian 2b09ce5
fix python test
rbasralian 3c2ad4f
Merge remote-tracking branch 'refs/remotes/origin/main' into raffi_re…
rbasralian File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
plugins { | ||
id 'io.deephaven.project.register' | ||
id 'java-library' | ||
} | ||
|
||
dependencies { | ||
api project(':engine-api') | ||
api project(':engine-table') | ||
api project(':engine-function') | ||
|
||
api 'com.fasterxml.jackson.core:jackson-databind:2.13.4' | ||
|
||
implementation project(':log-factory') | ||
implementation libs.commons.lang3 | ||
|
||
testRuntimeOnly project(path: ':configs') | ||
testRuntimeOnly project(path: ':test-configs') | ||
|
||
testImplementation TestTools.projectDependency(project, 'engine-table') | ||
testImplementation project(path: ':Base', configuration: 'tests') | ||
|
||
testRuntimeOnly project(':log-to-slf4j') | ||
} | ||
|
||
repositories { | ||
mavenCentral() | ||
} | ||
|
||
TestTools.addEngineOutOfBandTest(project) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
io.deephaven.project.ProjectType=JAVA_PUBLIC |
147 changes: 147 additions & 0 deletions
147
data-adapter/src/main/java/io/deephaven/dataadapter/ChunkRetrievalUtil.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,147 @@ | ||
// | ||
// Copyright (c) 2016-2024 Deephaven Data Labs and Patent Pending | ||
// | ||
package io.deephaven.dataadapter; | ||
|
||
import io.deephaven.chunk.*; | ||
import io.deephaven.chunk.attributes.Values; | ||
import io.deephaven.engine.rowset.RowSequence; | ||
import io.deephaven.engine.table.ChunkSource; | ||
import io.deephaven.engine.table.ColumnSource; | ||
import org.jetbrains.annotations.NotNull; | ||
|
||
import java.util.Objects; | ||
|
||
class ChunkRetrievalUtil { | ||
|
||
@NotNull | ||
static <T> ObjectChunk<T, ?> getObjChunkForKeys( | ||
final ColumnSource<T> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asObjectChunk(); | ||
} | ||
|
||
@NotNull | ||
static CharChunk<? extends Values> getCharChunkForKeys( | ||
final ColumnSource<Character> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asCharChunk(); | ||
} | ||
|
||
@NotNull | ||
static ByteChunk<? extends Values> getByteChunkForKeys( | ||
final ColumnSource<Byte> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asByteChunk(); | ||
} | ||
|
||
@NotNull | ||
static ShortChunk<? extends Values> getShortChunkForKeys( | ||
final ColumnSource<Short> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asShortChunk(); | ||
} | ||
|
||
@NotNull | ||
static IntChunk<? extends Values> getIntChunkForKeys( | ||
final ColumnSource<Integer> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asIntChunk(); | ||
} | ||
|
||
@NotNull | ||
static FloatChunk<? extends Values> getFloatChunkForKeys( | ||
final ColumnSource<Float> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asFloatChunk(); | ||
} | ||
|
||
@NotNull | ||
static LongChunk<? extends Values> getLongChunkForKeys( | ||
final ColumnSource<Long> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asLongChunk(); | ||
} | ||
|
||
@NotNull | ||
static DoubleChunk<? extends Values> getDoubleChunkForKeys( | ||
final ColumnSource<Double> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asDoubleChunk(); | ||
} | ||
|
||
@NotNull | ||
static BooleanChunk<? extends Values> getBooleanChunkForKeys( | ||
final ColumnSource<Boolean> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, | ||
final boolean usePrev) { | ||
return getChunkForKeys( | ||
columnSource, | ||
rowSequence, | ||
context, | ||
usePrev).asBooleanChunk(); | ||
} | ||
|
||
|
||
@NotNull | ||
static <T> Chunk<? extends Values> getChunkForKeys( | ||
final ColumnSource<T> columnSource, | ||
final RowSequence rowSequence, | ||
final ChunkSource.GetContext context, boolean usePrev) { | ||
return Objects.requireNonNull(usePrev ? columnSource.getPrevChunk(context, rowSequence) | ||
: columnSource.getChunk(context, rowSequence)); | ||
} | ||
|
||
|
||
private ChunkRetrievalUtil() {} | ||
|
||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All of these new methods need javadocs.