fixing mlflow logging to Databricks workspace file paths with /Shared/ prefix

[JackZ-db]

In mlflow logging, users that wanted to log into the "/Shared/" file directory had an extra slash in their modified experiment names, leading to Mlflow not recognizing existing experiments, trying to create them, and then claiming they already exist.

This PR accounts for that edge case and removes the extra slash added.

Prior to this PR, there was a bug where users who directly wanted to mlflow-log to the /Shared/ workspace would be able to do so after the first run of an experiment name, but would not be able to for following runs of the same experiment name. We found that an extra '/' was prepended to the experiment name (yielding an experiment name of //Shared/... after '/' + os.path.join('Users', databricks_username, experiment_name)), leading to get_experiment_by_name not finding the existing experiment, and create_experiment claiming that it already exists (which it does). There are probably ways in which get_experiment_name and create_experiment name handle file directories differently, also contributing to this bug.

In the original code prior to this PR, inputting an experiment name starting with /Shared/ would throw this error: mlflow.exceptions.RestException: RESOURCE_ALREADY_EXISTS: Node named ___ already exists

Now, mlflow logger correctly logs to the /Shared/... folder path as an absolute path.

Before:

• /Shared/... -> /Users/Shared/...
• /Users/... -> /Users/...
• path -> /Users/path

After:

• /Shared/... -> /Shared/... (no change)
• /Users/... -> /Users/...
• path -> /Users/path

[mvpatel2000]

Im a little confused, can you give examples of initial user specified arg and transformed arg before and after this PR as an example?

[JackZ-db]

Im a little confused, can you give examples of initial user specified arg and transformed arg before and after this PR as an example?

@mvpatel2000
Prior to this PR, there was a bug where users who directly wanted to mlflow-log to the /Shared/ workspace would be able to do so after the first run of an experiment name, but would not be able to for following runs of the same experiment name. We found that an extra '/' was prepended to the experiment name (yielding an experiment name of '//Shared/...' after '/' + os.path.join('Users', databricks_username, experiment_name)), leading to get_experiment_by_name not finding the existing experiment, and create_experiment claiming that it already exists (which it does). There are probably ways in which get_experiment_name and create_experiment name handle file directories differently, also contributing to this bug.

In the original code prior to this PR, inputting an experiment name starting with '/Shared/' would throw this error: mlflow.exceptions.RestException: RESOURCE_ALREADY_EXISTS: Node named ___ already exists

Now, mlflow logger correctly logs to the '/Shared/...' folder path as an absolute path. As mentioned above, there are probably some choices we want to make surrounding how we can most intelligently interpret absolute vs relative paths, but this serves as a functioning bandaid solution, because there are only '/Users/' and '/Shared/' under workspaces.

[mvpatel2000]

Lets move the skip on Shared to be where Users is checked

[mvpatel2000]

LGTM