Allow MQTT device based auto discovery #118757
Conversation
|
Hey there @emontnemery, @bdraco, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
|
Since this is difficult to read for someone not involved. Does this still support specifying topics device-wide (ie. same for all entities), and then overriding them on a per-entity basis if required? |
Effectively the PR is the same is the previous PR, but we love to seem some test results. We will try to update this branch as good as possible. A documentation PR is linked. |
jbouwh
force-pushed
the
mqtt-device-based-discovery
branch
from
9d516dd
to
317bb4b
Compare
jbouwh
force-pushed
the
mqtt-device-based-discovery
branch
2 times, most recently
from
bdb0bf7
to
85d2546
Compare
jbouwh
force-pushed
the
mqtt-device-based-discovery
branch
from
ffd153b
to
51fda9a
Compare
jbouwh
force-pushed
the
mqtt-device-based-discovery
branch
from
fbc9829
to
d23973b
Compare
![home-assistant[bot]](https://avatars.githubusercontent.com/in/97978?s=60&v=4){kind=small}
|
Please take a look at the requested changes, and use the Ready for review button when you are done, thanks 👍 |
jbouwh
force-pushed
the
mqtt-device-based-discovery
branch
from
d23973b
to
98a6812
Compare
| async def test_discovery_migration_to_device_base( | ||
| hass: HomeAssistant, | ||
| device_registry: dr.DeviceRegistry, | ||
| mqtt_mock_entry: MqttMockHAClientGenerator, | ||
| tag_mock: AsyncMock, | ||
| single_configs: list[tuple[str, dict[str, Any]]], | ||
| device_discovery_topic: str, | ||
| device_config: dict[str, Any], | ||
| ) -> None: | ||
| """Test the migration of single discovery to device discovery.""" | ||
| await mqtt_mock_entry() | ||
|
|
||
| # Discovery single config schema | ||
| for discovery_topic, config in single_configs: | ||
| payload = json.dumps(config) | ||
| async_fire_mqtt_message( | ||
| hass, | ||
| discovery_topic, | ||
| payload, | ||
| ) | ||
| await hass.async_block_till_done() | ||
| await hass.async_block_till_done() | ||
|
|
||
| await help_check_discovered_items(hass, device_registry, tag_mock) | ||
|
|
||
| # Migrate to device based discovery | ||
| payload = json.dumps(device_config | {"migrate_discovery": True}) | ||
| async_fire_mqtt_message( | ||
| hass, | ||
| device_discovery_topic, | ||
| payload, | ||
| ) | ||
| await hass.async_block_till_done() | ||
| # Check we still have our mqtt items | ||
| await help_check_discovered_items(hass, device_registry, tag_mock) |
There was a problem hiding this comment.
The integration that provides the entities and migrates them, is also resposible for cleaning up.
jbouwh
force-pushed
the
mqtt-device-based-discovery
branch
from
5c7ee65
to
b280d9a
Compare
| def _merge_common_options( | ||
| component_config: MQTTDiscoveryPayload, device_config: dict[str, Any] | ||
| ) -> None: | ||
| """Merge common options with the component config options.""" |
There was a problem hiding this comment.
Please explain this too
| @@ -590,6 +591,7 @@ async def cleanup_device_registry( | |||
| entity_registry = er.async_get(hass) | |||
| if ( | |||
| device_id | |||
| and device_id not in device_registry.deleted_devices | |||
There was a problem hiding this comment.
Why is this check added?
There was a problem hiding this comment.
If a device based discovery is triggered to be removed and all components removed, lets say 2 components, then this check will be done twice, as we do not await anything and the actual cleanup is processed later.
From the tests it seemed this would cause issues. With single components the updates have more time to process, and this race does not occur.
Co-authored-by: Erik Montnemery <[email protected]>
Proposed change
Rework #109030 which introduces a new device based MQTT discovery schema.
The original PR was reverted during the beta of HA Core 2024.6.0. With PR and branch we aim to test and optimize large MQTT setups that run via MQTT discovery.
Context
The current MQTT discovery model only allows to setup per component, if a device has multiple entities to published, the device context, availability and origin information needs to be duplicated.
This PR reduces IO overhead in the paho client on processing discovery packages: The baseline was 10000 packets, with device discovery it 6250 packet reads so a 37.5% reduction in I/O. Processing less I/O reduced the paho client run time by ~18%. The grouped discovery reduced the run time in the HA client by ~12%.
This PR adds a way to discover all components for a device with one discovery. The device, availability and origin mapping is only submitted once. Also the following options are allowed in the shared context:
state_topiccommand_topicencodingqosExcept for
deviceandoriginoptions, supported shared options can be overridden in the component config.Discovery updates and removal is fully supported.
The device based discovery coexists with the component based discovery, so no deprecation is planned for discovery with the single component schema to guarantee older devices keep working.
Example
The example below is of a device based auto discovery that supplies a
sensor, abinary_sensorand atag.Discovery topic:
Payload:
{ "dev": { "identifiers": [ "mysensors123" ] }, "avty": { "topic": "avty-topic" }, "cmp": { "bins1": { "platform": "binary_sensor", "name": "Beer", "state_topic": "test-topic" }, "sens1": { "platform": "sensor", "name": "Milk", "state_topic": "test-topic" }, "tag1": { "platform": "tag", "topic": "foobar/tags/tag1/see" } }, "o": { "name": "My org", "sw": "bla2mqtt" }, "qos": 0 }Note that a required
platformoption is added to identify the component platform. The keys under thecomponents(abbreviatedcmp), are treated asobject_idand are used to create a unique devicediscovery_id. In this case thediscover_id'saretest123 bins1andtest123 sens1.In case a
node_idis specified (e.g.node123) in the device discovery topic ...... this will become part of the
object_id.The
discovery_id's becometest123 node123 bins1andtest123 node123 sens1.Migration from single topic to device based discovery
To allow a smooth migration from single topic discovery to device based discovery, the
discovery_idfor anmqttitem must be the same. Note that migration is only supported if the single configuration topic has both anode_idand aobject_id. After migration theobjectid moves inside the discovery payload and the previousnode_idbecomes the newobject_idof the device discovery topic.The config to migrate to must have a
migrate_discoveryconfig option set toTrue. This will allow clearing the migrated discovery topics without removing the entities. So"migrate_discovery": trueneeds to be added to the JSON discovery payload(s) to migrate to. Migration is supported in both directions.Example (device automation and 2 sensors):
Discovery topic single:
homeassistant/device_automation/0AFFD2/bla1/configDiscovery id:
0AFFD2 bla1Discovery payload single:
{ "automation_type": "trigger", "device": { "identifiers": [ "0AFFD2" ] }, "o": { "name": "foobar" }, "payload": "short_press", "topic": "foobar/triggers/button1", "type": "button_short_press", "subtype": "button_1" }Discovery topic single:
homeassistant/sensor/0AFFD2/bla2/configDiscovery id:
0AFFD2 bla2Discovery payload single:
{ "automation_type": "trigger", "device": { "identifiers": [ "0AFFD2" ] }, "o": { "name": "foobar" }, "state_topic": "foobar/bla2/state", }Migrated to a one device config payload:
Discovery topic device:
homeassistant/device/0AFFD2/configDiscovery id:
0AFFD2Discovery payload device:
{ "device": { "identifiers": [ "0AFFD2" ] }, "o": { "name": "foobar" }, "cmp": { "bla1": { "automation_type": "trigger", "payload": "short_press", "topic": "foobar/triggers/button1", "type": "button_short_press", "subtype": "button_1", "platform": "device_automation" }, "bla2": { "state_topic": "foobar/bla2/state", "platform": "sensor" } }, "migrate_discovery": true }The source integration, responsible for the entities and the migration, is also responsible for cleaning up the old discovery topics after the migration has been completed. This the source integration should publish empty payloads to the old configuration topics.
This is safe to do after the new device based configuration has been published and processed.
Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all..coveragerc.To help with the load of incoming pull requests: