Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

got "error: The JSON element does not have the requested type" while following the guide to load data #14

Open
zaghost opened this issue Feb 20, 2024 · 0 comments

Comments

@zaghost
Copy link

zaghost commented Feb 20, 2024

When I followed the Kafka-connector-starrocks guide to load data to StarRocks, I got the error message below.

ERROR [starrocks-kafka-connector|task-0] catch exception, wait rollback  (com.starrocks.data.load.stream.v2.StreamLoadManagerV2:424)
com.starrocks.data.load.stream.exception.StreamLoadFailException: Transaction prepare failed, db: example_db, table: test-sqlite-jdbc-accounts, label: -0aba9ed9-fd83-41c3-afcf-9be12bbc926f, 
responseBody: {
    "Status": "DATA_QUALITY_ERROR",
    "Message": "Failed to iterate document stream as object. error: The JSON element does not have the requested type.",
    "ErrorURL": "http://10.5.0.3:8040/api/_load_error_log?file=error_log_174479fdeb882113_29549abd35d3bdb2"
}
errorLog: Error: Data quality error: Failed to iterate document stream as object. error: The JSON element does not have the requested type.. Row: parser current location: [{"id":19,"name":"cathy"}]

        at com.starrocks.data.load.stream.TransactionStreamLoader.prepare(TransactionStreamLoader.java:221)
        at com.starrocks.data.load.stream.v2.TransactionTableRegion.commit(TransactionTableRegion.java:247)
        at com.starrocks.data.load.stream.v2.StreamLoadManagerV2.lambda$init$0(StreamLoadManagerV2.java:210)
        at java.lang.Thread.run(Thread.java:750)

I can get the same error message if I run the command below.

curl --location-trusted -u root             \
    -d '[{"id":19,"name":"cathy"}]'                \
    -H "label:test-3"                  \
    -H "format: JSON"    \
    -H "columns:id,name" \
    -H "Expect:100-continue" \
    -XPUT http://localhost:8030/api/example_db/test-sqlite-jdbc-accounts/_stream_load

But if I add a header by -H "strip_outer_array:true", it will succeed.
So I fix the error by adding "sink.properties.strip_outer_array=true" to the properties file of Kafka-connector-starrocks plugin, because the configuration item will cause kafka-connector-starrocks adding "strip_outer_array:true" to http request header when do stream load.
My question is, is it possible to add the "strip_outer_array:true" header automatically by kafka-connector-starrocks, or do not wrap the data with '[ ]'?

BTW, I submitted a PR to modify the guide by adding "sink.properties.strip_outer_array=true" to the properties file.
StarRocks/starrocks#41218

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant