[wip] facade charm

.gitignore

@@ -6,5 +6,9 @@ dist/
 .ruff_cache
 .pytest_cache
 .idea/
+/facade_charm/mocks/provide/
+/facade_charm/mocks/require/
+/facade_charm/charmcraft.yaml
+
 venv
-*.charm
+*.charm

facade_charm/README.md

@@ -0,0 +1,69 @@
+# Facade charm
+
+The facade charm is somewhat similar to the any-charm, but is less generic and has a sharper focus on relation data.
+
+Usage:
+`juju deploy facade`
+`juju integrate facade:provide-your-interface <your charm>`
+
+## update databags with jhack eval
+
+`jhack eval facade/0 "self.set('provide-your-interface', app_data={'hello': 'world'}, unit_data={'lets foo': 'those bars'})"`
+
+## update databags with actions
+
+```yaml
+# in params.yaml
+endpoint: provide-tempo_cluster
+app_data: '{"foo":"bar"}'
+unit_data: '{"foo":"baz"}'
+```
+
+`juju run facade/0 update --params params.yaml`
+
+## update databags from mock files
+
+`cd facade-charm`
+`jhack sync -S facade/0 --include-files=".*\.yaml" --source mocks`
+
+`cd facade-charm/mocks/provide`
+
+edit `provide-your-interface.yaml`
+
+`juju run facade update`
+
+# using custom interfaces
+
+Your interface isn't in `charm-relation-interfaces` or on charmhub (yet)? No problem.
+Add it to `custom_interfaces.yaml`, and it shall be picked up when you `tox -e pack`.
+
+
+## interface conflicts
+The operator framework translates `"-"` to `"_"` when generating event names.
+So if you have an endpoint called `foo-bar`, you'll listen to relation events like: `self.on.foo_bar_relation_changed`.
+
+So if you have two endpoints, one called `foo-bar` and another called `foo_bar`, the operator framework will complain that the event is already defined.
+
+It turns out, there are some conflicts of this kind. 
+In this case, instead of generating `provides-foo-bar` and `provides-foo_bar`, our approach is to map `foo-bar` to `foo__bar`.
+
+So the facade charm will have:
+
+```yaml
+provides:
+  provides-foo__bar:
+    interface: foo-bar
+  provides-foo_bar:
+    interface: foo_bar
+```
+
+And same for `requires`.
+
+The list of such conflicts, and how they've been resolved, at the time of writing, is:
+
+- `vault-kv & vault_kv` --> **vault__kv**
+- `nginx-route & nginx_route` --> **nginx__route**
+- `grafana-dashboard & grafana_dashboard` --> **grafana__dashboard**
+- `tls-certificates & tls_certificates` --> **tls__certificates**
+
+

facade_charm/charmhub_interfaces.yaml

@@ -0,0 +1,239 @@
+interfaces:
+- ephemeral-backend
+- nova-vgpu
+- secrets
+- glance
+- pgbouncer-extra-config
+- ceph-mds
+- elasticsearch-datastore
+- gnocchi
+- ubuntu
+- lxd-bgp
+- local-monitors
+- keystone-domain-backend
+- ceph-admin
+- ldap
+- docker-registry
+- aar
+- jenkins-slave
+- rest
+- lldp
+- kapacitor
+- script-provider
+- tokens
+- grafana_auth
+- quantum
+- user-group
+- temporal
+- swift
+- oauth
+- pod-defaults
+- jenkins-extension
+- hydra_endpoints
+- elastic-beats
+- block-storage
+- tls-certificates
+- ceph-osd
+- prometheus
+- db
+- squid-auth-helper
+- ovsdb-subordinate
+- dockerhost
+- keystone-middleware
+- grafana-source
+- radosgw-user
+- kubeflow_dashboard_links
+- nova-ceilometer
+- swift-gw
+- karma_dashboard
+- shards
+- fluentbit
+- object-storage
+- websso-fid-service-provider
+- ovsdb-manager
+- glance-simplestreams-sync
+- anbox-stream-gateway
+- openstack-loadbalancer
+- ingress_per_unit
+- monitors
+- logs
+- kafka
+- pgsql
+- telegraf-exec
+- neutron-load-balancer
+- smtp
+- redis
+- slurmdbd
+- statistics
+- lxd
+- prometheus_scrape
+- grafana_datasource
+- cinder
+- keystone-fid-service-provider
+- loadbalancer
+- service-mesh
+- certificate_transfer
+- manila-plugin
+- magma-orchestrator
+- gcp-integration
+- elasticsearch
+- lxd-https
+- httpd
+- grafana_cloud_config
+- cos_agent
+- ceph-radosgw
+- mount
+- prometheus_remote_write
+- keystone-admin
+- external_cloud_provider
+- apache-vhost-config
+- rabbitmq
+- giraph
+- mysql-root
+- vsd-rest-api
+- mongodb
+- mongodb_client
+- ceph-iscsi-admin-access
+- keystone-notifications
+- swift-global-cluster
+- vsphere-integration
+- s3
+- ntp
+- monitor
+- kratos_info
+- neutron-api
+- zuul
+- http
+- mysql-monitor
+- kratos_endpoints
+- external_provider
+- baremetal
+- juju-info
+- ovsdb-cms
+- barbican-hsm
+- xlc-compiler
+- containers
+- prolog-epilog
+- mysql-router
+- postgresql_client
+- slurmrestd
+- istio-gateway-info
+- service-control
+- register-application
+- glauth_auxiliary
+- odl-controller-api
+- cinder-backend
+- keystone-credentials
+- sdn-plugin
+- alertmanager_remote_configuration
+- lxd-metrics
+- prometheus-manual
+- nova-cell
+- oidc-client
+- placement
+- event-service
+- radosgw-multisite
+- prometheus-rules
+- ceph-bootstrap
+- slurmd
+- ranger_client
+- openstack-integration
+- bind-rndc
+- arangodb
+- thruk-agent
+- etcd
+- glance-backend
+- nova
+- mysql-async-replication
+- mysql_client
+- landscape-hosted
+- bgp
+- websso-trusted-dashboard
+- mysql-shared
+- untrusted-container-runtime
+- ftn-compiler
+- ovsdb-cluster
+- vault-kv
+- kubernetes-cni
+- login_ui_endpoints
+- cinder-gw
+- infoblox
+- neutron-plugin
+- loki_push_api
+- swift-proxy
+- ceph-client
+- mysql
+- keystone
+- neutron-plugin-api-subordinate
+- jenkins_agent_v0
+- stun-server
+- autoscaling
+- dashboard-plugin
+- openfga
+- zookeeper
+- barbican-secrets
+- sentry-metrics
+- nats
+- aws-iam
+- ingress-auth
+- syslog
+- nova-vmware
+- azure-integration
+- grafana_dashboard
+- hacluster
+- agent-auth
+- memcache
+- cassandra
+- pacemaker-remote
+- apache-website
+- guacd
+- nginx-route
+- config-server
+- ingress
+- nova-compute
+- ceph-dashboard
+- grafana-dashboard
+- ssl-termination
+- juju-dashboard
+- lxd-dns
+- kube-dns
+- oathkeeper_info
+- udldap-userdata
+- kafka_client
+- cinder-backup
+- aws-integration
+- heat-plugin-subordinate
+- etcd-proxy
+- nrpe
+- ceph-rbd-mirror
+- saml
+- postfix-metrics
+- db2
+- munin-node
+- cinder-nedge
+- kube-control
+- lte-core
+- grpc
+- forward_auth
+- auth_proxy
+- catalogue
+- nrpe-external-master
+- dashboard
+- neutron-plugin-api
+- web-publish
+- logstash-client
+- midonet
+- traefik_route
+- container-runtime
+- public-address
+- ceilometer
+- influxdb-api
+- generic-ip-port-user-pass
+- java
+- nedge
+- ovsdb
+- k8s-service
+- cinder-ceph-key
+- designate
+- tracing
+- alertmanager_dispatch

facade_charm/custom_interfaces.yaml

@@ -0,0 +1,3 @@
+interfaces:
+  - your-interface-here
+  - tempo_cluster

facade_charm/fetch_interfaces_from_charmhub.py

@@ -0,0 +1,61 @@
+"""Scrape charmhub to grab all interfaces for all registered charms.
+"""
+import asyncio
+import json
+import logging
+from pathlib import Path
+
+import aiohttp
+import requests
+import tenacity
+import yaml
+
+logger = logging.getLogger("update-endpoints")
+
+FACADE_CHARM_ROOT = Path(__file__).parent
+CH_INTERFACES_PATH = FACADE_CHARM_ROOT / 'charmhub_interfaces.yaml'
+
+
+def _get_all_registered_charms():
+    print('fetching store...')
+    resp = requests.get("https://charmhub.io/packages.json")
+    return resp.json()['packages']
+
+
+@tenacity.retry(stop=tenacity.stop_after_attempt(10), wait=tenacity.wait_fixed(1))
+async def get_endpoints(charm_name, session: aiohttp.ClientSession):
+    url = f"https://charmhub.io/{charm_name}/integrations.json"
+    async with session.get(url=url, timeout=4) as response:
+        raw = await response.read()
+        return json.loads(raw)['grouped_relations']
+
+
+async def get_all_integrations(charms_pkg_info):
+    async with aiohttp.ClientSession() as session:
+        ret = await asyncio.gather(*(get_endpoints(charm['name'], session) for charm in charms_pkg_info))
+    return ret
+
+
+def _gather_interfaces(charms_pkg_info):
+    logger.info(f"gathering interfaces from {len(charms_pkg_info)} charms...")
+    all_integrations = asyncio.run(get_all_integrations(charms_pkg_info))
+
+    interfaces = set()
+    # discard any failed ones
+    for integrations in filter(None, all_integrations):
+        provides = integrations.get('provides', [])
+        requires = integrations.get('requires', [])
+        interfaces.update(endpoint['interface'] for endpoint in provides + requires)
+
+    logger.info(f"gathered {len(interfaces)} interfaces")
+    return interfaces
+
+
+def main():
+    charms_pkg_info = _get_all_registered_charms()
+    interfaces = _gather_interfaces(charms_pkg_info)
+    CH_INTERFACES_PATH.write_text(yaml.safe_dump({"interfaces": list(interfaces)}))
+
+
+if __name__ == '__main__':
+    main()

facade_charm/mocks/__DO_NOT_TOUCH.md

@@ -0,0 +1 @@
+the contents of this directory may be overridden by the update_endpoints script. 

facade_charm/requirements.txt

@@ -0,0 +1 @@
+ops ~= 2.5