Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Export windows functions directly from hermod #18

Merged
merged 8 commits into from
Dec 13, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,5 @@ export(hermod_task_eval)
export(hermod_task_result)
export(hermod_task_status)
export(hermod_task_submit)
export(windows_credentials)
export(windows_path)
54 changes: 54 additions & 0 deletions R/windows.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
##' Register DIDE windows credentials.
##'
##' In order to be able to communicate with the Windows DIDE HPC
##' system, we need to be able to communicate with the HPC portal
##' (<https::mrcdata.dide.ic.ac.uk/hpc>), and for this we need your
##' **DIDE** password and username. This is typically, but not always,
##' the same as your Imperial credentials. We store this information
##' securely using the [keyring](https://keyring.r-lib.org/) package,
##' so when unlocking your credentials you will be prompted for your
##' **computer** password, which might differ from either your DIDE or
richfitz marked this conversation as resolved.
Show resolved Hide resolved
##' Imperial password if you are not on a windows machine.
richfitz marked this conversation as resolved.
Show resolved Hide resolved
##'
##' @title DIDE windows credentials
##'
##' @return Nothing, these functions are called for their side effects.
##'
##' @export
windows_credentials <- function() {
ensure_package("hermod.windows")$dide_credentials()
}


##' Describe a path mapping for use when setting up jobs on the cluster.
##' @title Describe a path mapping
##'
##' @param name Name of this map. Can be anything at all, and is used
##' for information purposes only.
##'
##' @param path_local The point where the drive is attached locally.
##' On Windows this will be something like "Q:/", on Mac something
##' like "/Volumes/mountname", and on Linux it could be anything at
##' all, depending on what you used when you mounted it (or what is
##' written in `/etc/fstab`)
##'
##' @param path_remote The network path for this drive. It
##' will look something like `\\\\fi--didef3.dide.ic.ac.uk\\tmp\\`.
##' Unfortunately backslashes are really hard to get right here and
##' you will need to use twice as many as you expect (so *four*
##' backslashes at the beginning and then two for each separator).
##' If this makes you feel bad know that you are not alone:
##' https://xkcd.com/1638 -- alternatively you may use forward
##' slashes in place of backslashes (e.g. `//fi--didef3.dide.ic.ac.uk/tmp`)
##'
##' @param drive_remote The place to mount the drive on the cluster.
##' We're probably going to mount things at Q: and T: already so
##' don't use those. And things like C: are likely to be used.
##' Perhaps there are some guidelines for this somewhere?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future to-dos -

  • Should we have a ticket to write this guidance and improve that last sentence!
  • Should we outlaw I: here?
  • I slightly feel we should drift away from the temp drive, but perhaps not quite ready to shout that yet.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the temp drive we can probably just softly move away from it?

##'
##' @export
##' @author Rich FitzJohn
windows_path <- function(name, path_local, path_remote, drive_remote) {
ensure_package("hermod.windows")$windows_path(name, path_local,
path_remote, drive_remote)
}
7 changes: 2 additions & 5 deletions drivers/windows/NAMESPACE
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
# Generated by roxygen2: do not edit by hand

S3method(as.character,password)
S3method(as.character,path_mapping)
S3method(as.character,windows_path)
S3method(format,dide_clusterload)
S3method(print,dide_clusterload)
S3method(print,password)
S3method(print,path_mapping)
export(dide_authenticate)
export(dide_credentials)
export(path_mapping)
S3method(print,windows_path)
6 changes: 3 additions & 3 deletions drivers/windows/R/batch.R
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ template_data <- function(config, path_root) {

## Semicolon delimited list on windows; see "Managing libraries" in
## https://cran.r-project.org/doc/manuals/r-release/R-admin.html
hermod_library <- paste(unix_path(config$path_lib),
unix_path(config$path_bootstrap),
hermod_library <- paste(unix_path_slashes(config$path_lib),
unix_path_slashes(config$path_bootstrap),
sep = ";")

list(hostname = hostname(),
Expand All @@ -51,7 +51,7 @@ template_data <- function(config, path_root) {
network_shares_create = paste(network_shares_create, collapse = "\n"),
network_shares_delete = paste(network_shares_delete, collapse = "\n"),
hermod_root_drive = hermod_root$drive_remote,
hermod_root_path = paste0("\\", windows_path(hermod_root$rel)),
hermod_root_path = paste0("\\", windows_path_slashes(hermod_root$rel)),
hermod_library = hermod_library,
cluster_name = config$cluster)
}
6 changes: 0 additions & 6 deletions drivers/windows/R/dide_auth.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
##' Deal with DIDE credentials
##'
##' @title DIDE credentials
##' @export
dide_authenticate <- function() {
if (keyring::keyring_is_locked()) {
cli::cli_text(paste(
Expand Down Expand Up @@ -52,8 +48,6 @@ dide_authenticate <- function() {
}


##' @rdname dide_authenticate
##' @export
dide_credentials <- function() {
tryCatch({
username <- keyring::key_get("hermod/dide/username")
Expand Down
2 changes: 1 addition & 1 deletion drivers/windows/R/driver.R
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ windows_submit <- function(id, config, path_root) {
path_batch <- write_batch_task_run(id, config, path_root)

path_batch_dat <- prepare_path(path_batch, config$shares)
path_batch_unc <- windows_path(
path_batch_unc <- windows_path_slashes(
file.path(path_batch_dat$path_remote, path_batch_dat$rel))

client <- get_web_client()
Expand Down
8 changes: 4 additions & 4 deletions drivers/windows/R/mounts.R
Original file line number Diff line number Diff line change
Expand Up @@ -125,14 +125,14 @@ dide_check_shares <- function(shares) {
if (length(shares) == 0) {
return(NULL)
}
if (inherits(shares, "path_mapping")) {
if (inherits(shares, "windows_path")) {
shares <- list(shares)
}
if (!is.list(shares)) {
stop("Invalid input for 'shares'")
}
if (!all(vlapply(shares, inherits, "path_mapping"))) {
stop("All elements of 'shares' must be a path_mapping")
if (!all(vlapply(shares, inherits, "windows_path"))) {
stop("All elements of 'shares' must be a windows_path")
}

remote <- vcapply(shares, "[[", "drive_remote", USE.NAMES = FALSE)
Expand Down Expand Up @@ -172,7 +172,7 @@ dide_add_extra_root_share <- function(shares, path_root,
}
drive <- available_drive(shares, mounts[i, "local"])
c(shares,
list(path_mapping("root", mounts[i, "local"], mounts[i, "remote"], drive)))
list(windows_path("root", mounts[i, "local"], mounts[i, "remote"], drive)))
}


Expand Down
38 changes: 5 additions & 33 deletions drivers/windows/R/paths.R
Original file line number Diff line number Diff line change
@@ -1,32 +1,4 @@
##' Describe a path mapping for use when setting up jobs on the cluster.
##' @title Describe a path mapping
##'
##' @param name Name of this map. Can be anything at all, and is used
##' for information purposes only.
##'
##' @param path_local The point where the drive is attached locally.
##' On Windows this will be something like "Q:/", on Mac something
##' like "/Volumes/mountname", and on Linux it could be anything at
##' all, depending on what you used when you mounted it (or what is
##' written in `/etc/fstab`)
##'
##' @param path_remote The network path for this drive. It
##' will look something like `\\\\fi--didef3.dide.ic.ac.uk\\tmp\\`.
##' Unfortunately backslashes are really hard to get right here and
##' you will need to use twice as many as you expect (so *four*
##' backslashes at the beginning and then two for each separator).
##' If this makes you feel bad know that you are not alone:
##' https://xkcd.com/1638 -- alternatively you may use forward
##' slashes in place of backslashes (e.g. `//fi--didef3.dide.ic.ac.uk/tmp`)
##'
##' @param drive_remote The place to mount the drive on the cluster.
##' We're probably going to mount things at Q: and T: already so
##' don't use those. And things like C: are likely to be used.
##' Perhaps there are some guidelines for this somewhere?
##'
##' @export
##' @author Rich FitzJohn
path_mapping <- function(name, path_local, path_remote, drive_remote) {
windows_path <- function(name, path_local, path_remote, drive_remote) {
assert_scalar_character(name)
assert_scalar_character(path_local)
assert_scalar_character(path_remote)
Expand Down Expand Up @@ -55,14 +27,14 @@ path_mapping <- function(name, path_local, path_remote, drive_remote) {
path_remote = path_remote,
path_local = clean_path_local(path_local),
drive_remote = drive_remote)
class(ret) <- "path_mapping"
class(ret) <- "windows_path"

ret
}


##' @export
as.character.path_mapping <- function(x, ...) {
as.character.windows_path <- function(x, ...) {
if (is.null(x$rel)) {
sprintf("(local) %s => %s => %s (remote)",
x$path_local, x$path_remote, x$drive_remote)
Expand All @@ -74,15 +46,15 @@ as.character.path_mapping <- function(x, ...) {


##' @export
print.path_mapping <- function(x, ...) {
print.windows_path <- function(x, ...) {
cat(paste0("<path mapping>: ", as.character(x), "\n"))
invisible(x)
}


remote_path <- function(x, shares) {
x <- prepare_path(x, shares)
windows_path(file.path(x$path_remote, x$rel, fsep = "/"))
windows_path_slashes(file.path(x$path_remote, x$rel, fsep = "/"))
}


Expand Down
2 changes: 1 addition & 1 deletion drivers/windows/R/provision.R
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ windows_provision <- function(method, config, path_root, environment, ...) {
path_batch <- write_batch_provision_script(id, config, path_root)

path_batch_dat <- prepare_path(path_batch, config$shares)
path_batch_unc <- windows_path(
path_batch_unc <- windows_path_slashes(
file.path(path_batch_dat$path_remote, path_batch_dat$rel))

client <- get_web_client()
Expand Down
4 changes: 2 additions & 2 deletions drivers/windows/R/util.R
Original file line number Diff line number Diff line change
Expand Up @@ -98,12 +98,12 @@ clean_path <- function(x) {
}


windows_path <- function(x) {
windows_path_slashes <- function(x) {
gsub("/", "\\", x, fixed = TRUE)
}


unix_path <- function(x) {
unix_path_slashes <- function(x) {
gsub("\\", "/", x, fixed = TRUE)
}

Expand Down
10 changes: 10 additions & 0 deletions drivers/windows/R/zzz.R
Original file line number Diff line number Diff line change
@@ -1 +1,11 @@
cache <- new.env(parent = emptyenv())


.onAttach <- function(...) {
# nocov start
msg <- paste("You have called 'library(hermod.windows)' but you don't need",
"to; this package just needs to be installed to work, and you",
"only need to call 'library(hermod)'")
packageStartupMessage(paste(strwrap(msg), collapse = "\n"))
# nocov end
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

14 changes: 0 additions & 14 deletions drivers/windows/man/dide_authenticate.Rd

This file was deleted.

2 changes: 1 addition & 1 deletion drivers/windows/tests/testthat/helper-config.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ example_root <- function(mount_path, sub = "b/c") {
path <- file.path(mount_path, sub)
root <- suppressMessages(hermod::hermod_init(path))
path <- normalize_path(path)
shares <- path_mapping("home", mount_path, "//host/share/path", "X:")
shares <- windows_path("home", mount_path, "//host/share/path", "X:")
config <- hermod::hermod_configure("windows", shares = shares, root = root)
root
}
Expand Down
4 changes: 2 additions & 2 deletions drivers/windows/tests/testthat/test-config.R
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ test_that("Can create configuration", {
mount <- withr::local_tempfile()
path <- file.path(mount, "b", "c")
fs::dir_create(path)
shares <- path_mapping("home", mount, "//host/share/path", "X:")
shares <- windows_path("home", mount, "//host/share/path", "X:")
config <- withr::with_dir(path, windows_configure(shares, "4.3.0"))
expect_setequal(
names(config),
Expand Down Expand Up @@ -33,7 +33,7 @@ test_that("can configure a root", {
mount <- withr::local_tempfile()
path <- file.path(mount, "b", "c")
root <- suppressMessages(hermod::hermod_init(path))
shares <- path_mapping("home", mount, "//host/share/path", "X:")
shares <- windows_path("home", mount, "//host/share/path", "X:")
cmp <- withr::with_dir(path, windows_configure(shares, "4.3.0"))

hermod::hermod_configure(driver = "windows",
Expand Down
2 changes: 1 addition & 1 deletion drivers/windows/tests/testthat/test-driver.R
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ test_that("can submit a task", {
mockery::expect_called(mock_get_client, 1)
expect_equal(mockery::mock_args(mock_get_client)[[1]], list())

batch_path <- windows_path(file.path(
batch_path <- windows_path_slashes(file.path(
"//host.dide.ic.ac.uk/share/path/b/c/hermod/tasks",
id,
"run.bat"))
Expand Down
10 changes: 5 additions & 5 deletions drivers/windows/tests/testthat/test-mounts.R
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ test_that("can locate root path among paths", {
fs::dir_create(paths)
paths <- clean_path_local(paths)
mounts[, "local"] <- clean_path_local(mounts[, "local"])
shares <- Map(path_mapping,
shares <- Map(windows_path,
basename(mounts[, "local"]),
mounts[, "local"],
mounts[, "remote"],
Expand All @@ -19,7 +19,7 @@ test_that("can locate root path among paths", {
## their working directory
expect_equal(dide_add_extra_root_share(shares, paths[[1]], mounts),
shares)
result <- path_mapping("root", mounts[1, "local"], mounts[1, "remote"],
result <- windows_path("root", mounts[1, "local"], mounts[1, "remote"],
"V:")

## More commonly, we work out where the working directory is from
Expand Down Expand Up @@ -226,7 +226,7 @@ test_that("Find an available drive", {
test_that("Validate additional shares", {
path <- withr::local_tempfile()
mounts <- example_mounts(path)
shares <- Map(path_mapping,
shares <- Map(windows_path,
c("other", "home", "project", "temp", "sk"),
mounts[, "local"],
mounts[, "remote"],
Expand All @@ -235,7 +235,7 @@ test_that("Validate additional shares", {
expect_silent(dide_check_shares(shares))
expect_equal(dide_check_shares(shares[[1]]), shares[1])
expect_error(dide_check_shares(c(shares, TRUE)),
"All elements of 'shares' must be a path_mapping")
"All elements of 'shares' must be a windows_path")
expect_error(dide_check_shares(TRUE),
"Invalid input for 'shares'")
expect_null(dide_check_shares(list()))
Expand All @@ -246,7 +246,7 @@ test_that("Validate additional shares", {
test_that("Prevent duplicated drives", {
path <- withr::local_tempfile()
mounts <- example_mounts(path)
shares <- Map(path_mapping,
shares <- Map(windows_path,
c("a", "b", "c"),
mounts[1:3, "local"],
mounts[1:3, "remote"],
Expand Down
Loading