Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for git deployments #501

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
and `*_app.py` are now considered. However, if the directory contains more than
one file matching these new patterns, you must provide rsconnect-python with an
explicit `--entrypoint` argument.
- Added support for deploying directly from remote git repositories. Only
Connect server targets are supported, and the Connect server must have git
configured with access to your git repositories. See the
[Connect administrator guide](https://docs.posit.co/connect/admin/content-management/git-backed/)
and
[Connect user guide](https://docs.posit.co/connect/user/git-backed/) for details.

## [1.20.0] - 2023-09-11

Expand Down
41 changes: 35 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,7 @@ you will need to include the `--cacert` option that points to your certificate
authority (CA) trusted certificates file. Both of these options can be saved along
with the URL and API Key for a server.

> **Note**
> **Note**
> When certificate information is saved for the server, the specified file
> is read and its _contents_ are saved under the server's nickname. If the CA file's
> contents are ever changed, you will need to add the server information again.
Expand All @@ -135,7 +135,7 @@ rsconnect add \
--name myserver
```

> **Note**
> **Note**
> The `rsconnect` CLI will verify that the serve URL and API key
> are valid. If either is found not to be, no information will be saved.

Expand Down Expand Up @@ -407,6 +407,35 @@ library(rsconnect)
?rsconnect::writeManifest
```

### Deploying from Git Repositories
You can deploy content directly from from hosted Git repositories to Posit Connect.
The content must have an existing `manifest.json` file to identify the content
type. For Python content, a `requirements.txt` file must also be present.

See the [Connect user guide](https://docs.posit.co/connect/user/git-backed/)
for details on how to prepare your content for Git publishing.

Once your git repository contains the prepared content, use the `deploy git` command:
```
rsconnect deploy git -r https://my.repository.server/repository
```

To deploy from a branch other than `main`, use the `--branch/-b` option.

To deploy content from a subdirectory, provide the subdirectory
using the `--subdirectory/-d` option. The specified directory
must contain the `manifest.json` file.

```
rsconnect deploy git -r https://my.repository.server/repository -b my-branch -d path/within/repo
```

These commands create a new git-backed deployment within Posit Connect,
which will periodically check for new commits to your repository/branch
and deploy updates automatically. Do not run the
`deploy git` command again for the same source
unless you want to create a second, separate deployment for it.

### Options for All Types of Deployments

These options apply to any type of content deployment.
Expand All @@ -430,7 +459,7 @@ filename referenced in the manifest.

### Environment variables
You can set environment variables during deployment. Their names and values will be
passed to Posit Connect during deployment so you can use them in your code. Note that
passed to Posit Connect during deployment so you can use them in your code. Note that
if you are using `rsconnect` to deploy to shinyapps.io, environment variable management
is not supported on that platform.

Expand Down Expand Up @@ -985,9 +1014,9 @@ xargs printf -- '-g %s\n' < guids.txt | xargs rsconnect content build add
```
## Programmatic Provisioning

Posit Connect supports the programmatic bootstrapping of an administrator API key
Posit Connect supports the programmatic bootstrapping of an administrator API key
for scripted provisioning tasks. This process is supported by the `rsconnect bootstrap` command,
which uses a JSON Web Token to request an initial API key from a fresh Connect instance.
which uses a JSON Web Token to request an initial API key from a fresh Connect instance.

> **Warning**
> This feature **requires Python version 3.6 or higher**.
Expand All @@ -998,7 +1027,7 @@ rsconnect bootstrap \
--jwt-keypath /path/to/secret.key
```

A full description on how to use `rsconnect bootstrap` in a provisioning workflow is provided in the Connect administrator guide's
A full description on how to use `rsconnect bootstrap` in a provisioning workflow is provided in the Connect administrator guide's
[programmatic provisioning](https://docs.posit.co/connect/admin/programmatic-provisioning) documentation.

## Server Administration Tasks
Expand Down
79 changes: 62 additions & 17 deletions rsconnect/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,36 @@ def task_get(self, task_id, first_status=None):
self._server.handle_bad_response(response)
return response

def deploy_git(self, app_name, repository, branch, subdirectory, app_title, env_vars):
app = self.app_create(app_name)
self._server.handle_bad_response(app)

resp = self.post(
"applications/%s/repo" % app["guid"],
body={"repository": repository, "branch": branch, "subdirectory": subdirectory},
)
self._server.handle_bad_response(resp)

if app_title:
resp = self.app_update(app["guid"], {"title": app_title})
self._server.handle_bad_response(resp)
app["title"] = app_title

if env_vars:
result = self.app_add_environment_vars(app["guid"], list(env_vars.items()))
self._server.handle_bad_response(result)

task = self.app_deploy(app["guid"])
self._server.handle_bad_response(task)

return {
"task_id": task["id"],
"app_id": app["id"],
"app_guid": app["guid"],
"app_url": app["url"],
"title": app["title"],
}

def deploy(self, app_id, app_name, app_title, title_is_default, tarball, env_vars=None):
if app_id is None:
# create an app if id is not provided
Expand Down Expand Up @@ -300,7 +330,6 @@ def wait_for_task(
poll_wait=0.5,
raise_on_error=True,
):

if log_callback is None:
log_lines = []
log_callback = log_lines.append
Expand Down Expand Up @@ -741,6 +770,28 @@ def deploy_bundle(
}
return self

@cls_logged("Deploying git repository ...")
def deploy_git(
self,
app_name: str = None,
title: str = None,
repository: str = None,
branch: str = None,
subdirectory: str = None,
env_vars: typing.Dict[str, str] = None,
):
app_name = app_name or self.get("app_name")
repository = repository or self.get("repository")
branch = branch or self.get("branch")
subdirectory = subdirectory or self.get("subdirectory")
title = title or self.get("title")
env_vars = env_vars or self.get("env_vars")

result = self.client.deploy_git(app_name, repository, branch, subdirectory, title, env_vars)
self.remote_server.handle_bad_response(result)
self.state["deployed_info"] = result
return self

def emit_task_log(
self,
app_id: int = None,
Expand Down Expand Up @@ -1121,14 +1172,9 @@ def create_application(self, account_id, application_name):
return response

def create_output(self, name: str, application_type: str, project_id=None, space_id=None, render_by=None):
data = {
"name": name,
"space": space_id,
"project": project_id,
"application_type": application_type
}
data = {"name": name, "space": space_id, "project": project_id, "application_type": application_type}
if render_by:
data['render_by'] = render_by
data["render_by"] = render_by
response = self.post("/v1/outputs/", body=data)
self._server.handle_bad_response(response)
return response
Expand Down Expand Up @@ -1341,10 +1387,7 @@ def prepare_deploy(
app_mode: AppMode,
app_store_version: typing.Optional[int],
) -> PrepareDeployOutputResult:

application_type = "static" if app_mode in [
AppModes.STATIC,
AppModes.STATIC_QUARTO] else "connect"
application_type = "static" if app_mode in [AppModes.STATIC, AppModes.STATIC_QUARTO] else "connect"
logger.debug(f"application_type: {application_type}")

render_by = "server" if app_mode == AppModes.STATIC_QUARTO else None
Expand All @@ -1362,11 +1405,13 @@ def prepare_deploy(
space_id = None

# create the new output and associate it with the current Posit Cloud project and space
output = self._rstudio_client.create_output(name=app_name,
application_type=application_type,
project_id=project_id,
space_id=space_id,
render_by=render_by)
output = self._rstudio_client.create_output(
name=app_name,
application_type=application_type,
project_id=project_id,
space_id=space_id,
render_by=render_by,
)
app_id_int = output["source_id"]
else:
# this is a redeployment of an existing output
Expand Down
55 changes: 49 additions & 6 deletions rsconnect/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ def wrapper(*args, **kwargs):

return wrapper


# This callback handles the "shorthand" --disable-env-management option.
# If the shorthand flag is provided, then it takes precendence over the R and Python flags.
# This callback also inverts the --disable-env-management-r and
Expand All @@ -252,7 +253,7 @@ def wrapper(*args, **kwargs):
# which is more consistent when writing these values to the manifest.
def env_management_callback(ctx, param, value) -> typing.Optional[bool]:
# eval the shorthand flag if it was provided
disable_env_management = ctx.params.get('disable_env_management')
disable_env_management = ctx.params.get("disable_env_management")
if disable_env_management is not None:
value = disable_env_management

Expand Down Expand Up @@ -486,7 +487,6 @@ def bootstrap(
@cloud_shinyapps_args
@click.pass_context
def add(ctx, name, server, api_key, insecure, cacert, account, token, secret, verbose):

set_verbosity(verbose)
if click.__version__ >= "8.0.0" and sys.version_info >= (3, 7):
click.echo("Detected the following inputs:")
Expand Down Expand Up @@ -1081,11 +1081,11 @@ def deploy_manifest(
name="quarto",
short_help="Deploy Quarto content to Posit Connect [v2021.08.0+] or Posit Cloud.",
help=(
'Deploy a Quarto document or project to Posit Connect or Posit Cloud. Should the content use the Quarto '
"Deploy a Quarto document or project to Posit Connect or Posit Cloud. Should the content use the Quarto "
'Jupyter engine, an environment file ("requirements.txt") is created and included in the deployment if one '
'does not already exist. Requires Posit Connect 2021.08.0 or later.'
'\n\n'
'FILE_OR_DIRECTORY is the path to a single-file Quarto document or the directory containing a Quarto project.'
"does not already exist. Requires Posit Connect 2021.08.0 or later."
"\n\n"
"FILE_OR_DIRECTORY is the path to a single-file Quarto document or the directory containing a Quarto project."
),
no_args_is_help=True,
)
Expand Down Expand Up @@ -1436,6 +1436,49 @@ def deploy_help():
click.echo()


@deploy.command(
name="git",
short_help="Deploy git repository with exisiting manifest file",
help="Deploy git repository with exisiting manifest file",
)
@server_args
@click.option("--app_name", "-a")
@click.option("--repository", "-r", required=True)
@click.option("--branch", "-b", default="main")
@click.option("--subdirectory", "-d", default="/")
@click.option("--title", "-t", help="Title of the content (default is the same as the filename).")
@click.option(
"--environment",
"-E",
"env_vars",
multiple=True,
callback=validate_env_vars,
help="Set an environment variable. Specify a value with NAME=VALUE, "
"or just NAME to use the value from the local environment. "
"May be specified multiple times. [v1.8.6+]",
)
@cli_exception_handler
def deploy_git(
name: str,
server: str,
api_key: str,
insecure: bool,
cacert: typing.IO,
verbose,
app_name: str,
repository: str,
branch: str,
subdirectory: str,
title: str,
env_vars: typing.Dict[str, str],
):
subdirectory = subdirectory.strip("/")
kwargs = locals()
set_verbosity(verbose)
ce = RSConnectExecutor(**kwargs)
ce.validate_server().deploy_git().emit_task_log()


@cli.group(
name="write-manifest",
no_args_is_help=True,
Expand Down
Loading