Home Assistant MQTT Template

[ferazambuja]

Currently, Home Assistant users use the Openhab MQTT template changing the Controller Subscribe and Publish.
As stated in home assistant documentation:
https://www.home-assistant.io/components/sensor.mqtt/#get-sensor-value-from-a-device-with-espeasy

Controller Subscribe: home/%sysname%/# (instead of /%sysname%/#)
Controller Publish: home/%sysname%/%tskname%/%valname% (instead of /%sysname%/%tskname%/%valname%)

Since this is such a simple change wouldn't be possible to add the template?

Besides naming is just this two lines from _C005.ino:

    case CPLUGIN_PROTOCOL_TEMPLATE:
      {
        event->String1 = F("/%sysname%/#");
        event->String2 = F("/%sysname%/%tskname%/%valname%");
        break;
      }

[Grovkillen]

It seems that openHAB now have gone from having a starting / to have %sysname% in as the base topic. I would suggest not to change to home/%sysname%/... for the openHAB controller. We need to find a more generic way of using the same MQTT core and only apply templates.

https://docs.openhab.org/addons/bindings/mqtt1/readme.html

So based on these new info I'd say we could at least change the default to %sysname%/%tskname%/%valname% (with no leading front slash).

[TD-er]

If OpenHAB and HomeAssistant are different, we could add a new controller, which will share a lot of code between them, but have their own implementation where they differ.
I guess that will be better than having some solution that's sub-optimal for both.

So please try to think of the best topic template for Home Assistant. (and for OpenHAB if the current one could be better)

[davidgraeff]

Related to this topic. Eclipse Smarthome (OpenHab is based on this) gets a shinny new MQTT stack while I'm writing these lines. Some research went into auto-discovery of (IoT) devices and published sensors/actuators. The need for devices to describe themselves and their capabilities is inherent.

To form some sort of standard (convention on top of MQTT), members of the OpenHab community collaborated with the creator of the Homie convention.

It would be very convenient if ESPeasy could speak Homie 3.x and allows easy point and click integration in future OpenHab versions.

[TD-er]

Sounds like a good idea.
I have to read it first to let the concept shape in my head.
The general idea to have a single description of the device you have to allow low level discovery is very good and I hope this description does not place too many limitations on the current (and future) ESPeasy design itself.
I will have a look into this.

Too bad you have to read documents like this. An audiobook/podcast would be great, since I'm behind the wheels about 10 - 15 hours/week ;)

[bkpsu]

Has there been any discussion/progress among the team in implementing one of these standards (Homie or HomeAssistant) within the ESPEasy Controller? @davidgraeff 's newest MQTT binding in openHAB is now live (https://community.openhab.org/t/mqtt-binding-version-2-4/53811/55) and supports auto-discovery of MQTT topics, as long as they're defined in the Homie/HomeAssistant format, as I understand it. It would be excellent if I could convert all of my sensors to use this, rather than having to manually build them into my openHAB configuration...

[TD-er]

Do you have an easy to read document on how such an auto-discovery should look like?

[bkpsu]

This is the official convention that I could find for Homie. There seems to be a lot of topics that need to be populated for auto-discovery, but, I did see an issue (homieiot/convention#45) opened that aims to thin down those topics to a minimal set of required ones.

[TD-er]

It looks like well documented and elaborate.
Maybe a bit too elaborate at this moment to get an idea on where to start.

So can you give an example, for me to get a feeling about this discovery, to for instance let a switch input be discovered.
What should the node do and what settings can be derived and what should be given by the user?

[davidgraeff]

A "in a nutshell" paragraph is missing, indeed.

• The homie version is required: "homie/device123/$version" → 3.0
• The device state is of interest: "homie/device123/$state" → ready (can be init, ready, sleeping, lost and some more)
• A device name is helpful: "homie/device123/$name" → My cool device
• We need an explit comma separated list of available nodes:
  • "*homie/device123/$nodes" → thermostat
• We need an explit comma separated list of available properties:
  • "*homie/device123/thermostat/$properties" → temperature
• Each property need a name and a type, eg:
  • "homie/device123/thermostat/temperature/$name" = Temperature
  • "homie/device123/thermostat/temperature/$type" = integer (can be integer, float, boolean, string, enum, color)
• The property topic itself contains the state.
• The command topic is "homie/device123/thermostat/temperature/set".

That's it!

[davidgraeff]

@bkpsu Please refer to the web-page https://homieiot.github.io/ instead of the repository. The repo contains the latest/in-development specification, and is not meant to be used for implementers.

[Christian-Me]

I currently start my home automation new from scratch. The "old" system was based on openHAB 1.8 and mosquitto MQTT server and a lot of openHAB code. I waited a wile until openHAB 2.x settled. Recently I discovered that with Version 2.4 openHAB committed to MQTT by integrating a MQTT server inside as a binding - great!. With some rules and a little mod of the MQTT-import plugin to receive true and false payloads It is possible to get sensors and actors running ESPEasy autodiscovered and working with no line of code, rules or conf files.

I'm planning to implement a controller plugin for homie and/or home assistant. For a "proof of concept" I did a test implementation which works out fine
Basically a rule is triggered every minute to propagate the features. In the future this should be sent only on mqtt connect with retained flag set but for a quick test this works.

on System#Boot do
 Debug 4
 timerSet,1,60
endon

On Rules#Timer=1 do  //Update autodiscover
 Publish homie/%sysname%/$homie,3.0.0
 Publish homie/%sysname%/$name,%sysname%
 Publish homie/%sysname%/$localip,%ip%
 Publish homie/%sysname%/$mac,%mac% // not working but not used by openhab
 Publish homie/%sysname%/$fw/name,ESP Easy Mega
 Publish homie/%sysname%/$fw/version,2.0.0
 Publish homie/%sysname%/$imlementation,ESP8266
 Publish homie/%sysname%/$stats/interval,90 // publish a longer interval helps to avoid timeout messages
 Publish homie/%sysname%/$nodes,Durchfluss,switchValve // to devices, one sensor one actor

 Publish homie/%sysname%/Durchfluss/$name,Durchflussrate
 Publish homie/%sysname%/Durchfluss/$type,Flowcounter
 Publish homie/%sysname%/Durchfluss/$properties,Count,Total,Time // sensor offers 3 values

 Publish homie/%sysname%/Durchfluss/Count/$name,Zaehler // Value 1
 Publish homie/%sysname%/Durchfluss/Count/$datatype,integer
 Publish homie/%sysname%/Durchfluss/Count/$unit,#
 Publish homie/%sysname%/Durchfluss/Total/$name,Gesamt // Value 2
 Publish homie/%sysname%/Durchfluss/Total/$datatype,integer
 Publish homie/%sysname%/Durchfluss/Total/$unit,#
 Publish homie/%sysname%/Durchfluss/Time/$name,Zeit // Value 3
 Publish homie/%sysname%/Durchfluss/Time/$unit,sec
 Publish homie/%sysname%/Durchfluss/Time/$datatype,integer

 Publish homie/%sysname%/switchValve/$name,Stellventil
 Publish homie/%sysname%/switchValve/$type,Valve
 Publish homie/%sysname%/switchValve/$properties,Valve
 Publish homie/%sysname%/switchValve/Valve/$name,Ventil
 Publish homie/%sysname%/switchValve/Valve/$settable,true // because it is an actor
 Publish homie/%sysname%/switchValve/Valve/$datatype,boolean // needs a modified flow2string function to convert true/false to 1/2

 Publish homie/%sysname%/$state,ready //all up and ready
 timerSet,1,60
endon

on Valve do // handle the incomming /set msg via MQTT import plugin
TaskValueSet 4,1,%eventvalue% // store value for debug reasons
if [switchValve#Valve]=1
  gpio,5,1
  Publish homie/%sysname%/switchValve/Valve,true // confirm back to homie
else
  gpio,5,0
  Publish homie/%sysname%/switchValve/Valve,false
endif
endon

after a few seconds a new thing is detected

So with the little mod of the source code to handle incomming true/false msgs by the MQTT import plugin the relais work. And it is usable after a few clicks.

Now I'm planning to implement this as a controller plugin to do most things automatically.

• in the CPLUGIN_INIT section the header should be sent. looping through all devices/tasks and values
• in the CPLUGIN_PROTOCOL_RECV section the %value%/set commands should be handeled and feedback sent automatically
• CPLUGIN_PROTOCOL_SEND should work as is with the "homie/%sysname%/%tskname%/%valname%" template
• messages for the $state topic should send "sleeping" (before falling asleep), "init" (during boot), "ready" and "alert" (when the user should handle errors) messages.

My "only" concern is adding/deleting devices as the retained flag should be set. There will be the need of a little bit of housekeeping to get rid of unused devices/nodes: With the above rules "New Thing" and "Thing updated" is working inside paper UI.

• openhab should delete all devices/nodes if they are not listed in the $nodes message any more (prefered)
• openhab mqtt server could accept homie/sysname/# messages with no payload to erase all messages
• or subscribe to the topic and loop through the retained messages an looking for outdated entries and deleting them (most complicated but certainly possible)

so with every reboot the header and node informations should be send/updated. If a user do modifications a reboot once he is happy is better than having openHAB handle "half backed" configurations with every change.

Finally I like the home assistant spec a little bit better because of less topics with JSON formatted payload but I first took a look on the homie as recomended by @davidgraeff

It cannot be stressed enough, to consider changing existing MQTT client devices to an MQTT convention like the mentioned Homie 3.x convention. That might not be possible in some cases though.

Davids blog post

So my questions are:

• is something like this already in the works and I will waste my time?
• are there any traps I should consider - beside reading the wiki?
• anybody with the same demand for making ESPeasy homie and/or homeassistant compatible?
  or is there a existing way I not found or thought of?

BTW: I'm totally new in contributing to an open source project and so please be partitioned.

[TD-er]

These look like they can be incorporated in an extension to the existing OpenHab controller.
How often do these messages have to be sent to the broker?
At least on a change I assume? We can create an event when settings are saved, then a rule can be triggered. Maybe later on also a command to publish the discovery texts, so it can also be queried from OpenHab I guess? Is there an existing format for?

Maybe we must also add something to the plugins themselves, or to a group of plugins.
And about the info you may need, just ask here and someone will answer :)

[davidgraeff]

I would not extend the openHAB controller, which should only stay as a legacy controller (or be removed, depending on your backwards compatibility policies).

openHAB is now fully compliant to the MQTT Homie specification and for OH 2.5 we aim to be fully compatible to the HomeAssistant MQTT-Components specification.

A Homie convention controller and a HA-controller is fully sufficient.

I like the home assistant spec a little bit better because of less topics with JSON formatted payload

Homie tries to stay json free, that's true. But HA MQTT has the issue that they are not generic. They have specified a set of components and that's it. If your device has more features or does not fit into one of the components you are out of luck with their specification. That's why I think both conventions should be considered.

openhab should delete all devices/nodes if they are not listed in the $nodes message any more

That is actually happening. But the OH core does not remove a once-seen thing or channel, no matter what the binding requests. There is an issue on our issue tracker and also a PR. Might be fixed for OH 2.5.

[Christian-Me]

I agree to start with a new "homie" controller and perhaps a extra homeassistant controller later when openHAB 2.5 is ready. Looking into homeassistants definition I agree that there are some limitations.

as tools like mqtt-forget are available and openhab ignores old $nodes there should be no problem with "housekeeping"

[Christian-Me]

Hello
@davidgraeff is it possible that the build in mqtt server do not set the retain flag as expected? I spend a few hours poking around in the ESPeasy source to find out why mqttFX don't show the retained flag until i found that messages I send by mqttFX with the retained flag set also not show up as retained. Sending the same message to my "old" mosquitto server work as expected,
openHAB server:

mosquitto server:

Retained flag always set:

or doese the build in mqtt server recognizes the retained flag and "consumes" it?

[davidgraeff]

The mqtt specification allows MQTT servers to ignore the "retained" flag if QoS is 0. Might that be your problem?

[Christian-Me]

The question is if "retained" is nessesary for autodiscover? And if so should I use a higher QoS?
The homie convention says that most messages should be sent with retain flag set.
Or can I simply ignore that the retain flag here. I plan to send all the information only once after reboot.

[davidgraeff]

Retained is not necessary and only affects how the server processes and stores messages. openHAB will use whatever message it receives.

[Christian-Me]

Fine, than I‘ll not wasting my time on this any further.

[Christian-Me]

I finished 1st draft version of the homie protocol/controller plugin (I called it c014 for now if it's ok)

currently the basic discovery for sensors is implemented. Incomming messages are the next task.

I ran into several issues:

1. I added a new event called CPLUGIN_CONNECTED which should be called by the framework as soon a mqtt connection is established but only once.
2. I used the MQTTpublish() function and found out later that this only puts the messages into the queue. processMQTTdelayQueue(); helps but I think there should/must be a better way. (handling all mqtt messages completely by the plugin itself directly to the pubsub lib(?)
3. Did not found good way to find information about the expected units (°C, A, V ...) - Used the value Names for testing
4. Don't know how to determine settable values for displays, motors, neopixels .... and to handle them (should be read and settable. i.e homie/esp_test/oled/line1 for sending and homie/esp_test/oled/line1/set for setting. can be done for each devive "hardcoded" but I'm looking for a general solution.
5. Tried to mimize incomming traffic (every message gets echoed with the default publish / subcribe topic masks because the subscribe is a subset of the publish mask. Thought "homie/%sysname%/+/+/#" for subscription and "homie/%sysname%/%tskname%/%valname%" for sending whould help but still every message is echoed.
6. for sending commands a scheme perhaps should be defined like "homie/%sysname%/cmd/%command%" should be defined. But again the complete echoed stream must be analysed. Pausing subscribing is not a solution because according to the conventions the messages should be flagged as retained.
7. Handling of GPIOs like Relays or LEDs without rules to match the convention homie/%sysname%/GPIO_Plugin_Name/GPIO_A for reading and /set for writes. Perhaps there should be a simple GPIO plugin that a Name can be associated to a GPIO (I learned that a plugin can handle 3 GPIOs)
8. proper error handling is not done to make sure that finally all messages are sent because the queue is limited.
9. and may more ...

Every comments and help welcome. I committed the plugin and some changes in the framework to my repro.

[davidgraeff]

@Christian-Me : Could you please link your repo here (with the correct branch preselected)? That saves people time, who are willing to have a look.

I also suggest that you create small PRs for required framework changes separately and try to get those in first. Your repo/branch should at the end only contain your new controller.

[TD-er]

For now it is only possible to have 1 MQTT controller active in ESPeasy.
So please be aware of that when testing yourself.
This is still a legacy decision and I plan on removing that limitation, but that needs a lot of code changes.

About the queue system used for controllers.
Each controller has its own delay queue system and I advice you have a short look at it to understand the idea behind it.
The main purpose for writing this queue is to make sure handling messages is not stalling other tasks on the ESP.
The MQTT controllers all use the same queue mechanism, which is also the simplest of all, since PubSub client is also using some buffer.
But still that buffer is limited, so the MQTT delay queue is making ESPeasy behave a lot faster with a lot less delays since the PubSubClient buffer is quite small.

For testing purposes you can increase the limits of those queues, but keep in mind they tend to use a lot of memory if they retain the entire MQTT message.
See also controller documentation about the available parameters for that buffer.
If you need to extend these, please let me know, so other controllers can benefit from it too.
But please don't try to do things different from other controllers, unless it is really needed.
I've been trying very hard to make the code for all controllers look as much the same as possible so we can move stuff into separate helpers instead of the severe code duplication it was in the past.

If you need to process the items in the queue, please have a look at the function scheduleNextMQTTdelayQueue and processMQTTdelayQueue.

As can be seen there, the scheduler is having an entry for handling the MQTT delay queue and processing incoming messages.
For example by calling setIntervalTimerOverride(TIMER_MQTT, 10); you will schedule to handle the MQTT flow at 10 msec from now. (don't set it to '0' or else you may cause lots of other scheduled tasks to become delayed)
By setting that timer, it will schedule a task to run runPeriodicalMQTT(), which does also call the loop() function of PubSubClient.
That loop function does the actual MQTT work.

So please try to use the existing MQTT functions as much as possible and if there's something missing, don't try to interact directly with PubSubClient, but try to find a way to use or extend the code in ESPeasy.
If you're trying to interact directly with PubSubClient, you may cause other issues when someone is running another setup than the one you've been testing with.

Also note that we're not using the same code as on the official PubSubClient Github.
I've made a few fixes myself and combined ideas of others that made pull requests to that repo. Currently there is a newer version than the one our lib is based on, but I didn't have time to check the differences.

[Christian-Me]

you can find my 1st attempt here complete fork
or the controller plugin directly here: _C014.ino

The autodiscover information for my test setup are more than 20 topics (10-12 for the system information, 3 for every device and 2 for every value) so expanding the queue is not an option I think. I used the following code for each topic (example for the system name):

        // $name	Device → Controller	Friendly name of the device	Yes	Yes
        tmppubname = pubname;
        tmppubname.replace(F("#"), "$name");
        log = F("C014: $name: ");
        log += Settings.Name;
        if (MQTTpublish(event->ControllerIndex, tmppubname.c_str(), Settings.Name, 1)) {addLog(LOG_LEVEL_INFO, log+" success!");}
            else addLog(LOG_LEVEL_ERROR, log+" ERROR!");
        processMQTTdelayQueue();

This works fine in most cases.

The plugin will need 4 new events

// new messages for autodiscover plugins (experimental)
#define CPLUGIN_CONNECTED                  52 // call after connected to mqtt server to publich device autodicover features
#define CPLUGIN_FALLING_ASLEEP             53 // call before going to sleep
#define CPLUGIN_AWAKE                      54 // call after wakeup (not sending autodicover feed) 
#define CPLUGIN_SEND_STATS                 55 // call every interval loop

I'm not sure were to place the function calls.
CPLUGIN_CONNECTED should be called after the MQTT connection is established but not after deep sleep. Currently placed in bool MQTTCheck(int controller_idx)
CPLUGIN_AWAKE should be called instead after deep sleep
CPLUGIN_FALLING_ASLEEP should be called before deep sleep
CPLUGIN_SEND_STATS should be called periodical Currently placed inside void runEach30Seconds()

[TD-er]

Well at first I would suggest to make a separate function to show the extra plugin functions needed.
Then we can call those later from the right points.
In the loop function in ESPEasy.ino there is a function which is called right before going to sleep.
You can add a call to this function from there.
Later we will move these calls to the right place if we know more about their use. (it is now still in flux)

About the queue for all those topics.
Do you think we can define a fixed set of topics? Then you could make some enum to keep these along with the values you try to send.
Then we only store an enum and the values and generate the whole strings from the queue again as soon as they are being read from the queue.
That's also being done in some other controller queues.
Only store what is needed to recreate the message when needed.

Edit:
I guess those topics are depending on the plugin, right?
Then the right place would eventually be to ask the plugin for the topic. But if it can be done via some helper to create generic topics for a type of sensor value, then we would have less code duplication.
So that means we should query the plugin for a specific type of value it is trying to send. (e.g. temp/humidity/distance/time/CO2/etc)

[Christian-Me]

The plan is to create a controller plugin to be as close as possible to the homie convention without changing code inside plugins and as less as possible in the firmware. In the beginning I thought I could uses the INIT message but this is to early in the boot process.

1. There are 11 messages to describe the Device Attributes (see point 5.2.1) marked as required
2. There are 3 messages to describe the Node (See point 5.3.1) in ESPEasy terms a plugin / Sensor
3. There are 7 (not required) for every value (for sensors) (5.4.1) and minimum 1 required for actors ( the settable attribute). I think unit and a friendly name is interesting too.
4. Device Statistics (5.2.3) only Uptime is required as a heartbeat. Think for WIFI devices signal strength is good too.
5. Device Behavior (5.2.2) is good to implement to aware of sleep states or low battery or sensor failures

All required messages for sensors can be acquired without any change in plugins leaving the unit open.
For actors a final solution is open. I think for test reasons i will "had code" LCD/OLED and simple NEOPixel because I have them here. Perhaps the two motor plugins too because I have both somewhere in my desk. Most interesting could be GPIOs for Relais. They can be handled by the controller directly catching incoming messages like homie/systemName/GPIO/Number/set 0 or 1 so no extra plugin necessary. Problem is how to determine when they should be in use. Perhaps if the set in the hardware tab to anything else than default or input. When set as input they can be estimated as buttons (homie/systeminfo/GPIO/Number without the settable flag set and handled by the controller.

Can you point me to an good example controller how a enum could be implemented?

Every topic acording to this scheme homie/%deviceName%/%nodeName%/%valueName%
ie. homie/sensorDevice1/BMP280/Temperature

Sending Values work "of the shelf" by sending to "homie/%sysname%/%tskname%/%valname%"
Receiving works by appending /set to set a value. i.e. "homie/motorDevice/motorController1/Speed/set". It is expected that the value will be set and sent back as a confirmation.

It would be good that beside the value name a value Unit can be selected currently °C Degree Celsius, °F Degree Fahrenheit, ° Degree, L Liter, gal Galon, V Volts, W Watt, A Ampere, % Percent, m Meter, ft Feet, Pa Pascal, psi PSI, # Count or Amount. I can thing about some more Hz, Ah, Wh, VA, LUX, ....
Perhaps a set of fixed Units define by a byte can be enough. If I look in openHAB there are many more units defined.

Hey this started looking like a quick hack and going to be a real task, Its the first time in contribute to an opensource project ...

[Grovkillen]

Totally agree on that. It should be protocol first. Den tors 11 juli 2019 23:28Misiu <[email protected]> skrev:

…

@Grovkillen <https://github.com/Grovkillen> if this can be set via front end then even better, but please another option to Protocol dropdown. I'm aware that we can just edit Controller Subscribe and Controller Publish but why should we choose openHAB if we want Home Assistant, that's confusing for new users. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1314?email_source=notifications&email_token=AGLYCYTYAPN66HFT5FWDZPDP66QYRA5CNFSM4E4LF2MKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZYBBBA#issuecomment-510660740>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGLYCYUV4FZ4K4H5LVWZ4JTP66QYRANCNFSM4E4LF2MA> .

[Misiu]

@Grovkillen I see all the HTML comes from WebServer.ino file.
The easiest thing (for me) would be to create a new Controller (and change https://github.com/letscontrolit/ESPEasy/blob/mega/src/_C005.ino#L42), but as You suggested this should be done using C005 (OpenHAB MQTT), but how should I duplicate controller on list and change its settings when I select it? Basically, I think I should add C005 with a new name (Home Assistant MQTT) and when the user selects it I should update its settings by changing subscribe and publish topics.

[TD-er]

Currently there can be only one MQTT controller active.

[Misiu]

@TD-er so the easiest way would be to add new controller (Home Assistant MQTT), but as @Grovkillen suggested here we should reuse existing controller.
If you agree I can create a new controller (_C018?) for Home Assistant.

[TD-er]

What are the exact differences between Home Assistant MQTT and OpenHAB MQTT?
If they are almost the same, I would suggest to add some kind of selector in the existing one.
I don't like code duplication and have already moved a lot of code duplication from the controllers a few months ago.

[Misiu]

@davidgraeff could You please help me with this one?
The current version of openHAB is using Homie convention, which is supported by https://github.com/letscontrolit/ESPEasy/blob/mega/src/_C014.ino.
The old versions of openHAB were supported by C005.

The changes are minimal, @ferazambuja posted them in the first post here.
Take a look at official HA docs.

Controller Subscribe: home/%sysname%/# (instead of /%sysname%/#)
Controller Publish: home/%sysname%/%tskname%/%valname% (instead of /%sysname%/%tskname%/%valname%)

If C005 isn't used maybe it can be renamed to Home Assistant MQTT?
I was confused that I must use openHAB and tweak some settings if I want to connect to Home Assistant.

Maybe name like openHAB/Home Assistant MQTT would be better? and a selector that would prefill topics according to selection.

[TD-er]

Maybe name like openHAB/Home Assistant MQTT would be better? and a selector that would prefill topics according to selection.

I think this is the way to go if the only difference is the default topic.

[Misiu]

@TD-er I'm relatively new to both Home Assistant and openHAB so I'd like to ask more advanced users for confirmation. I base my answer on official HA docs and my short experience.
@ferazambuja any comments? Did You notice any other differences?

[davidgraeff]

openHABs preferred MQTT convention is Homie. But openHAB also supports the HomeAssistant MQTT convention. A name like this "Home Assistant (openHAB) MQTT" will do. Because it's their standard I would suggest to name Home Assistant first for the label and openHAB only in parentheses.

[Misiu]

if @TD-er agrees I will rename C005 controller to Home Assistant (openHAB) MQTT, but how/where I should add selector that will prefill topics? is it in WebServer.ino?
How can I add two buttons and a couple of lines of JS to this controller Web UI?
Something like this.

[TD-er]

Please remind me if I didn't reply this evening.
I am on the road now.

[davidgraeff]

I don't see a reason for this javascript or UI part. The openHAB implementation mimics the Home Assistant implementation. Right now it expects "homeassistant" to be the first topic part as seen in the sources:

https://github.com/openhab/openhab2-addons/blob/master/bundles/org.openhab.binding.mqtt.homeassistant/src/main/java/org/openhab/binding/mqtt/homeassistant/internal/discovery/HomeAssistantDiscovery.java#L75

This could be configurable, but I do not see why. The HomeAssistant guys should instead settle on a good default (like for example "homeassistant" so what we can all agree on how to perform discovery).

I propose to change the HA documents and swap the suggested "home" with "homeassistant" or get a confirmation from a HA dev, that they want to keep "home" and I will change the openHAB implementation. No matter what, a UI or different topics for both systems are not necessary.

[Christian-Me]

@Misiu
The default topics are defined in the controller plugin

    case CPLUGIN_PROTOCOL_TEMPLATE:
      {
        event->String1 = F("/%sysname%/#");
        event->String2 = F("/%sysname%/%tskname%/%valname%");
        break;
      }

The current (old) templates are not (strictly) correct because mqtt toppics do not start with an "/". This results in an empty root topic: Here my last device running with C005

@davidgraeff : When can we expect a new openHAB milestone build or a snapshot where the mqtt issues are fixed. I currently still run into several problems (during testing C014 & P086).

• openHAB stops updating values (disable / enable this thing helps)
• openHAB shows things as offline (but they are not - again disable and enable the thing helps)
• openHAB do not autodetect (sometimes) new things (restart openHAB or adding things manually helps)

All problems seems to be a result of openHAB stops receiving messages. Still have to use Mosquitto because of the QoS problem (see above).
I cannot recommend M1 because of other problems without an easy workaround. So I`m desperately waiting for M2 or a "good" snapshot. Do not want to test random snapshots because I have to run it on my live environment to do the "real life tests".

[davidgraeff]

I'm not involved in the release process of openHAB. Could happen at any time when someone presses the release button. The mqtt update problems are related to the used paho mqtt java library which is shit. A migration to the hivemqtt library is probably required but I do not have time for that.

But back to the topic: The protocol template would need a "homeassistant" base topic as well.

[TD-er]

Can you describe what is happening when OpenHAB stops updating values?
Would it help if ESPeasy has a disconnect/reconnect MQTT option in the controller?
How should something like that be initiated?

Also ESPeasy is not perfect regarding network stability. For example a WiFi disconnect may not always be detected in ESPeasy itself, which may lead to strange behavior. Most of the times a WiFi reconnect does lead to WDT reboots, which is -in this use case- the preferable solution since it forces a proper reconnect.

[Christian-Me]

Currently performing many tests on "real hardware" with the Homie Plugins together with MQTT Explorer (which is a great help) showing that ESPEasy even when running complex rules is quite stable and can perform well even after a reboot (using RTC memory / retained mqtt messages).
I found out the sometimes openHAB do not react on value updates any more. Values can still be sent over #/set topics to ESPEasy so the MQTT connection to the broker is still alive but it seams that messages get lost within openHAB. This correlates with openHAB stopping to autodetect new or updated things. I did not dig into the logs ... as a workaround I use node-red (witch is my main logic engine) ... where all communication with ESPEasy via MQTT is flawless.

[Misiu]

@TD-er @davidgraeff I've created PR that renames C005 plugin.
I'll also create an issue in HA repo to get more info about the first topic.

Should I also change topics in C005 and add home to the beginning?

[TD-er]

At least some topic descriptions/examples in the documentation would be fine.

[Misiu]

@TD-er, as mentioned in my PR Home Assistant, doesn't have a base topic (prefix).
Please take a look at https://www.home-assistant.io/components/sensor.mqtt/
I can add a link to HA documentation.

@davidgraeff can openHAB ignore first part? Currently homeassistant is required as the base topic, but this isn't required by HA in any way (I might be wrong on that one, so please correct me if so)

[davidgraeff]

but this isn't required by HA in any way

Might be correct. But than there is no proper way to detect HA components. In Homie we have "something/{deviceid}/$homie" to recognise the Homie convention.

[Misiu]

@davidgraeff because of Homie there is the ability to do autodiscovery in openHAB. HA doesn't support Homie (hope this will change), but what about the old way? Was there some kind of auto-discovery?
In Home Assistant I'm adding every sensor manually by following docs.
There are scripts for specific devices (like this one for Shelly), but there isn't any standard I can think of.

[davidgraeff]

MQTT doesn't enforce any topic layout. There is no way of performing auto discovery. That's why those two conventions emerged. But I thought that HA had a fixed default topic. Without that, it is not really suitable for auto-discovery, I guess.

[Misiu]

Can #2517 get merged? @TD-er @davidgraeff any comments? Something missing?

[TD-er]

@Misiu I guess it can be merged, but not right now, for several reasons.

• I'm a few days away ;)
• The current state of the WiFi related code is something wrong.

That last point is mainly why I will not yet merge code into the mega branch. If I merge code, it will trigger a rebuild and that one has about 50% chance of failing to connect to WiFi. A gamble which is unique for each and every bin file in the ZIP.
Something is just too time critical (or some complete other reason) which does result in perfectly working WiFi connect and just adding something completely unrelated => not able to connect to WiFi.
And since I'm not home right now, I cannot work on that part, nor delete the build as early as possible tomorrow morning.

[Misiu]

@TD-er no worries 🙂
It can get merged later. It's not a critical change.
The WiFi part is more important and should be addressed/fixed first.

[Misiu]

#2517 is merged so I guess this can be closed?

[TD-er]

Yep.
I should use references in commits more often ("Fixes #issuenr" does automatically close issues when merged)