-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add unit test for graceful failed executions of write.socrata() #131
Comments
Thank you @levyj for creating that private test dataset earlier, it was helpful for testing the username / password again. In doing that I realized that authentication errors fail gracefully, but invalid domains error out. Timeout also causes an error (see below for a test) @tomschenkjr / @nicklucius In #213 I mentioned that I thought it would be a mistake to fail gracefully when writes fail, but we already do that in some cases. I don't think we'll ever have tests for writing again though because there's no way to get a token / user and password to write to a test dataset. @levyj is considering reaching out to support on this. Timeout replication:
|
@levyj created a test dataset and I wrote some manual tests, which I wanted to copy here for reference. # Jon created the following private data set for temporary testing:
https://data.cityofchicago.org/dataset/Dataset-for-Gene-to-test-something-in-RSocrata/6wga-czt4
# The JSON endpoint:
https://data.cityofchicago.org/resource/6wga-czt4.json
# For testing, I created a reated a temporary file with my credentials in
# ./ignore/credentials.yaml
testdf <- dataframe = data.frame(x = sample(-1000:1000, 5),
y = sample(-1000:1000, 5))
# Expected successful write to Jon's sample, private dataset
{
datasetToReplaceUrl <- "https://data.cityofchicago.org/resource/6wga-czt4.json"
creds <- yaml::read_yaml("ignore/credentials.yaml")
res <- write.socrata(testdf,
dataset_json_endpoint = datasetToReplaceUrl,
update_mode = "REPLACE",
email = creds$socrataEmail,
password = creds$socrataPassword)
print(res$status_code)
}
# Expected unsuccessful write to Jon's sample, private dataset
# Authentication error -- GRACEFUL FAIL
{
datasetToReplaceUrl <- "https://data.cityofchicago.org/resource/6wga-czt4.json"
creds <- yaml::read_yaml("ignore/credentials.yaml")
creds$socrataPassword <- "not the real password"
res <- write.socrata(testdf,
dataset_json_endpoint = datasetToReplaceUrl,
update_mode = "REPLACE",
email = creds$socrataEmail,
password = creds$socrataPassword)
print(res$status_code)
}
# Expected unsuccessful write to Jon's sample, private dataset
# Bad URL -- UNHANDLED ERROR
{
datasetToReplaceUrl <- "https://Xdata.cityofchicago.org/resource/6wga-czt4.json"
creds <- yaml::read_yaml("ignore/credentials.yaml")
res <- write.socrata(testdf,
dataset_json_endpoint = datasetToReplaceUrl,
update_mode = "REPLACE",
email = creds$socrataEmail,
password = creds$socrataPassword)
print(res$status_code)
}
# Expected unsuccessful write
# Timeout -- UNHANDLED ERROR
{
datasetToReplaceUrl <- "httpstat.us/200?sleep=5000000"
creds <- yaml::read_yaml("ignore/credentials.yaml")
res <- write.socrata(testdf,
dataset_json_endpoint = datasetToReplaceUrl,
update_mode = "REPLACE",
email = creds$socrataEmail,
password = creds$socrataPassword)
print(res$status_code)
} |
write.socrata()
is tested to ensure that it can correctly write to the portal, but it does not test to ensure there is a consistent way of failing-to-write a data set. Specifically, to ensure when a write-process fails that it returns an error or warning.It would be useful if the error itself would return specifics on why the write failed.
The text was updated successfully, but these errors were encountered: