Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit test for graceful failed executions of write.socrata() #131

Open
tomschenkjr opened this issue Mar 10, 2017 · 2 comments
Open

Add unit test for graceful failed executions of write.socrata() #131

tomschenkjr opened this issue Mar 10, 2017 · 2 comments

Comments

@tomschenkjr
Copy link
Contributor

write.socrata() is tested to ensure that it can correctly write to the portal, but it does not test to ensure there is a consistent way of failing-to-write a data set. Specifically, to ensure when a write-process fails that it returns an error or warning.

It would be useful if the error itself would return specifics on why the write failed.

@geneorama
Copy link
Member

Thank you @levyj for creating that private test dataset earlier, it was helpful for testing the username / password again.

In doing that I realized that authentication errors fail gracefully, but invalid domains error out. Timeout also causes an error (see below for a test)

@tomschenkjr / @nicklucius In #213 I mentioned that I thought it would be a mistake to fail gracefully when writes fail, but we already do that in some cases.

I don't think we'll ever have tests for writing again though because there's no way to get a token / user and password to write to a test dataset. @levyj is considering reaching out to support on this.

Timeout replication:

  datasetToReplaceUrl <- "httpstat.us/200?sleep=5000000"
  res <- write.socrata(dataframe = data.frame(x = sample(-1000:1000, 5),
                                              y = sample(-1000:1000, 5)),
                       dataset_json_endpoint = datasetToReplaceUrl,
                       update_mode = "REPLACE",
                       email = "",
                       password = "")
  print(res$status_code)

@geneorama
Copy link
Member

@levyj created a test dataset and I wrote some manual tests, which I wanted to copy here for reference.

# Jon created the following private data set for temporary testing: 
https://data.cityofchicago.org/dataset/Dataset-for-Gene-to-test-something-in-RSocrata/6wga-czt4

# The JSON endpoint:
https://data.cityofchicago.org/resource/6wga-czt4.json

# For testing, I created a reated a temporary file with my credentials in 
# ./ignore/credentials.yaml


testdf <- dataframe = data.frame(x = sample(-1000:1000, 5),
                                 y = sample(-1000:1000, 5))


# Expected successful write to Jon's sample, private dataset
{
  datasetToReplaceUrl <- "https://data.cityofchicago.org/resource/6wga-czt4.json"
  creds <- yaml::read_yaml("ignore/credentials.yaml")
  res <- write.socrata(testdf,
                       dataset_json_endpoint = datasetToReplaceUrl,
                       update_mode = "REPLACE",
                       email = creds$socrataEmail,
                       password = creds$socrataPassword)
  print(res$status_code)
}


# Expected unsuccessful write to Jon's sample, private dataset
# Authentication error -- GRACEFUL FAIL
{
  datasetToReplaceUrl <- "https://data.cityofchicago.org/resource/6wga-czt4.json"
  creds <- yaml::read_yaml("ignore/credentials.yaml")
  creds$socrataPassword <- "not the real password"
  res <- write.socrata(testdf,
                       dataset_json_endpoint = datasetToReplaceUrl,
                       update_mode = "REPLACE",
                       email = creds$socrataEmail,
                       password = creds$socrataPassword)
  print(res$status_code)
}

# Expected unsuccessful write to Jon's sample, private dataset
# Bad URL -- UNHANDLED ERROR
{
  datasetToReplaceUrl <- "https://Xdata.cityofchicago.org/resource/6wga-czt4.json"
  creds <- yaml::read_yaml("ignore/credentials.yaml")
  res <- write.socrata(testdf,
                       dataset_json_endpoint = datasetToReplaceUrl,
                       update_mode = "REPLACE",
                       email = creds$socrataEmail,
                       password = creds$socrataPassword)
  print(res$status_code)
}


# Expected unsuccessful write
# Timeout -- UNHANDLED ERROR
{
  datasetToReplaceUrl <- "httpstat.us/200?sleep=5000000"
  creds <- yaml::read_yaml("ignore/credentials.yaml")
  res <- write.socrata(testdf,
                       dataset_json_endpoint = datasetToReplaceUrl,
                       update_mode = "REPLACE",
                       email = creds$socrataEmail,
                       password = creds$socrataPassword)
  print(res$status_code)
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants