[SUGGESTION] Team up with "Zigbee for Domoticz" developers who is developing a zigpy based plugin for Domoticz #17
Comments
Hi @Hedda great to hear that zigpy is growing. We can send free samples to the developers, I only need shipping addresses send to [email protected] (just reference me and this issue).
Indeed I think this brings benefits like projects using zigpy don't have to fully reinvent and maintain all the Zigbee complexity on their own and join forces. |
|
Thanks @Hedda for that, but I wonder what would be the benefit of such integration while there is already a deConz dedicated plugin for Domoticz available https://github.com/Smanar/Domoticz-deCONZ . Maybe @manup you could give your point of view ? I also don't see much activity on the zigpy-deconz github , does that mean it is mature and stable ? accept my naive question, but is the serial protocol available on all deConz firmware version, or is that only available on specific one ? |
|
mail sent to @manup . |
To compare situation there was already deCONZ dedicated plugins/integrations for Home Assistant and Jeedom before their zigpy based implementations came to be and now there are many users of those that use ConBee and RaspBee adapters with them. Also note that today it is no longer only Dresden Elektronik's official deCONZ/Phoscon application and the zigpy based integrations like Home Assistant's ZHA integration component and Jeedom's Zigbee Plugin that has native support for ConBee and RaspBee adapter via this deconz serial protocol, but also zigbee-herdsman based projects like Zigbee2MQTT and IoBroker as well. Both the Zigbee2MQTT developers and developers of the zigpy based Jeedom Zigbee Plugin are listing ConBee II adapters to their users. See https://www.zigbee2mqtt.io/guide/adapters/#recommended and https://doc.jeedom.com/en_US/plugins/automation%20protocol/zigbee/ and https://www.home-assistant.io/integrations/zha Supporting an additional hardware type would of course mean more work but I believe that there could be several different good reasons and benitis with your zigpy based implementation also supporting ConBee and RaspBee adapter via zigpy-deconz;
Yes the zigpy-deconz library should be considered mature and stable which is the main reason why there are not many updates for that GitHub repository. Jeedom developers is even listing it as the foremost recommended adapter for their implementation before any other adapters -> https://doc.jeedom.com/en_US/plugins/automation%20protocol/zigbee/ deconz based adapters are still also listed at the top of the compatibility list for Home Assistant's ZHA integration (however I do not believe that any of the zigpy developers that are most active today are currently using ConBee/RaspBee adapters in their own personal "main-production" Zigbee setups) -> https://www.home-assistant.io/integrations/zha
All deconz firmware versions are supported by zigpy-deconz but it is highly recommend that users first update their ConBee/RaspBee adapters to the latest deconz firmware to avoid bugs that have been fixed in newer firmware releases, see- > https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Firmware-Changelog |
|
@Hedda thanks , but as already reported there are some issues which seems to be only fixed on the new-radio-API which looks for me and for the Zigbee for Domoticz plugin and handicap.
Anyway I have sent an email to @manup (as mentioned earlier). |
|
Thanks for the mail the samples should arrive quickly :)
The serial protocol allows to specify Extended PANID ("APS Use Extended PANID" parameter) and to configure custom endpoints/clusters. These need be set while the network is disconnected and become active after network start. |
|
thanks @manup , that was more less my understanding, just need to get that implemented on the zigpy-deconz side. |
Big thanks for the sponsorship @manup , will keep you posted. |
@manup @pipiche38 So I understand that all those are functions that do already exist in deCONZ Serial Protocol but they have not yet been added to zigpy-deconz so should be feature request or pull requests for https://github.com/zigpy/zigpy-deconz/ ? |
|
the Extended PAN Id seems to be implemented in the new Radio Api, however the force_form new network I have issues with. For adding Ep, I didn't find any api for that (probably because we are the only one having Schneider Wiser eco-system running) |
@pipiche38 That is not completley true. Jeedom developer @zoic21 explained that do share the source code of the plugin but not directly via a Git repository and instead just to those who bought their Zigbee plugin via the Jeedom Market, which while maybe is not in the spirit of FOSS (Free and Open Source Software) is technically allowed according to the GPL license used. However since you can not simply download the Jeedom plugin from a Git repository they do make you jump through some hops to get the source code, as instead you need to first install Jeedom and register account for their Jeedom Market, then buy and install the Zigbee Plugin from their Jeedom Market, after that the plugin can be installed on your computer. As the plugin is coded in Python it is also source code and as such it is now available to you on your local computer from the installation directory: https://www.jeedom.com/market/index.php?v=d&p=market_display&id=4050 [EDIT]: Update! Re-posted this @zoic21 Git repo request for Jeedom Zigbee plugin source code to here -> zigpy/zigpy#725 PS: I do not personally agree with their practice of having a partially closed system which do not make code directly available via Git to other developers as I believe in the spirit of FOSS (Free and Open Source Software) when depending on other projects code, so in the spirit of open sourcve I think that they should instead have them put their Zigbee Plugin in a public GitHub repository under https://github.com/jeedom as they have done with Jeedom itself and other plugins for Jeedom so that developers from other projects like yourself could take advantage by learning from their implementation without having to buy their plugin and install the software. That would allow all developers and projects depending on zigpy to collaborate much easier. |
|
@manup I tried to use the old RasBee shield you sent to me 2 years back. Unfortunately I'm not able to make it running The ConBee II works fine. I have been able to update to the latest firmware. Here after is the zigpy error I'm getting 2022-03-15 18:01:22,265 WARNING :No response to 'Command.read_parameter' command with seq id '0x02' The above exception was the direct cause of the following exception: cc: @puddly |
Note that to use RaspBee (or any other Zigbee shield on a Raspberry Pi) you will need to first disable and/or enable some things depending on which exact Raspberry Pi model that you have; so either; disable Bluetooth + hciuart service and "uncomment to enable primary UART console" to enable UART via https://www.abelectronics.co.uk/kb/article/1035/serial-port-setup-in-raspberry-pi-os https://www.google.com/search?q=raspbee+AND+%22dtoverlay%3Ddisable-bt%22+AND+%22enable_uart%3D1%22 If got older Raspberry Pi 2 or the original Raspberry Pi or Raspberry Pi Zero then configure user access rights of serial interface:
If got the later Raspberry Pi 3 and Raspberry Pi Zero W or newer then instead edit the config.txt inside boot directly on storage:
Then edit end of that file where it says to "uncomment to enable primary UART console" to also add Some instructions say it should also work if re-enable BT via miniUART on newer Raspberry Pi with the latest firmware: Anyway, then run commands to disable hciuart service Changes to access only become active after a full reboot.
|
|
@Hedda, thanks my question is more to 1/ RasBee firmware versus RaspBee II ? |
|
Looks to me like the is "deCONZ_Rpi" firmware images for it here -> https://deconz.dresden-elektronik.de/deconz-firmware/ deCONZ_Rpi-latest.bin.GCF is today currently the same as deCONZ_Rpi_0x26400500.bin.GCF from 18-Aug-2021 21:43 So assume that "deCONZ_Rpi" firmware images is for "old" first-generation shield and "deCONZ_RaspBeeII" is for RaspBee II? As I understand the application firmware feature and function parity is the same for the first/old RaspBee and new RaspBee II? Believe that the only relative practical difference is that the RaspBee II hardware has a newer radio chip MCU model == better? |
|
Still no chance rpi4 - bullseye cat /boot/cmdline.txt /boot/config.txt dmesg | grep tty ps -ef | grep tty sudo GCFFlasher_internal -l sudo GCFFlasher_internal -f deCONZ_Rpi_0x26400500.bin.GCF -d /dev/ttyAMA0 -x 3 |
|
If not using deCONZ did you try https://github-wiki-see.page/m/dresden-elektronik/deconz-rest-plugin/wiki/Update-deCONZ-manually or if updating from deCONZ Docker container then should use https://github.com/deconz-community/deconz-docker#updating-conbeeraspbee-firmware Alternative methods to using GCFFlasher could be to install deCONZ app or Phoscon App Beta and update firmware from there? That is, I think that both the deCONZ app and Phoscon App Beta has built-in functions to update RaspBee and ConBee firmware. Also, always make sure you got a very good power supply for RaspBerry Pi 4 and RaspBerry Pi 3 respectively as they are infamously known for not work properly with peripherals if do not correctly powered. As well as disable hciuart service if did not do so already. Other than those I have no more tips myself, though hope you manage to flash the old RaspBee adapter and get it working :) PS: To not go too much off-topic that first-gen RaspBee firmware upgrade problem now maybe belongs in some other forum(s)?
|
That was maybe a wrong assumption? See -> https://github.com/dresden-elektronik/deconz-rest-plugin/wiki/Firmware-Changelog "Note: The number differences between I and II series don't represent feature or bugfix equality as there are different underlying Zigbee stacks in use." As I understand the older first-generation ConBee and RaspBee products are based on some older NXP 8-bit AVR based MCU chip with NXP Zigbee stack(?) and then they added their own proprietary "deCONZ" application firmware (application layer) with custom APIs(?). While the newer ConBee II and RaspBee II are base on a Microchip ATSAMR21E18A 32-bit Arm Cortex-M0 MCU, so presumable that instead uses Microchip Stack for the Zigbee Protocol and then Dresden Elektronik ported their own proprietary "deCONZ" application firmware with custom APIs to that too(?). It is also possible to run DRS ZBOSS Zigbee Stack on both older NXP chips as well as the newer Microchip ATSAMR21G18A chips. Regardless it would explain the APIs being the same but there not being feature or bugfix parity between the older older first-generation firmware for first-generation ConBee and RaspBee hardware and the the newer ConBee II and RaspBee II.
PS: I believe this is a similar practice to ZiGate with the exception that their application firmware (application layer) is open source. |
@manup could you kindly clarify which API needs to be use to configure custom endpoints/clusters that ? |
|
From feature perspective RaspBee I and II are comparable, with the difference that the 1. gen also implements the unholy Touchlink function. RaspBee I doesn't use NXP but Atmels (now Microchip) ATmegaRFR2, the R21 chip in RaspBee II is newer, but this makes no real difference for Zigbee, both have a power amplifier (PA) and low noise amplifier (LNA) to strengthen sending and receiving. The next gen is based on Silabs and will also use the same serial protocol.
To configure the endpoints the parameter 0x13 need to be written via WRITE_PARAMETER (0x0B) request. I just noticed this is currently missing in the docs, I've put it on the todo list. Example: simple endpointThe following configures endpoint 0x01 as first endpoint at index 0x00. Note that the U16 values need to be written as little endian byte order 0x0019 → 0x19 0x00. Hope that helps. |
|
@manup when you say 3 endpoints are supported: 0-2, is that in addition of the standard ZCL 0x01 EndPoint, or does it includes 0x01, and so we can add only 2 End Points ? |
|
@puddly is that enough to make a similar API as you did for znp ? |
|
Is this necessary? Only ZNP requires explicit endpoint registration as far as I can tell: zigbeefordomoticz/Domoticz-Zigbee#1090 (comment) |
Seems that my test was not relevant. I made the same test on ZNP without ep registration and this device paired without problem. |
|
Providing certain clusters is necessary for a few devices which query the coordinator for these via Match Descriptor Request. The actual ZCL cluster implementation is done in the application. The OTA, Time and IAS clusters are a good choice to handle a wide range of devices. For example many Xiaomi devices query the coordinators Time cluster and act up if they don't receive an response. Here is the default configuration that deCONZ currently sets:
The default endpoints which are initially set by the firmware depend on the firmware version, I think it contains Basic, Time and OTA. For that reason we check in deCONZ if everything needed is configured and extend the cluster list if necessary to be independent of the firmware version in use. The endpoint 0xF2 is needed if you want to support Zigbee Green Power Proxy support. |
@puddly yes it is necessary for pairing the Livolo switch, getting the Wiser Thermostat (touch) working. This was the case with ZNP and we have added those Ep and it works like a charm |
@manup is that "next-gen" as in the new Silicon Labs EFR32MG24 chip will be used in upcoming "ConBee III" and "RaspBee III"? |
|
The EFR32MG24 is under consideration but not available yet, more likely we go for for EFR32MG21 which we already use in other products to consolidate chip diversity. As a side node, contrary to common believe there isn't a real difference from a users perspective in comparison for the R21 in ConBee II / RaspBee II. All these chips are insanely fast in terms of Zigbee performance and the workload and memory intensive stuff is handled in the host application. :) The real fixed bottlenecks are the same for all chips, that is the 250 kbps of the radio which only can transmit one message in serialized manner, and the legal restrictions for max. transmit power of ~10 dBm. Quite interesting here is the OpenThread stack which we'll support also for ConBee II / RaspBee II as Border Router. It has configurable architecture and allows the radio firmware to be simple and out source the heavy parts to the host application. |
FYI, I understand Silicon Labs FW in "RCP" mode (instead of "NCP" mode) can also offload parts of their Zigbee stack to host, see:
https://github.com/home-assistant/addons-development/tree/master/silabs-multiprotocol |
|
@manup Thanks for the details. Is it worth considering increasing the endpoint registration number limit ? On each above endpoint, requests are made from the device on another endpoint than 0x01. I've made some further test and for example, the wiser thermostat have requests (Read Attributes and Report Attributes) from 0x0b to 0x0b. Other coordinators are not impacted by this kind of limit, the more surprising is EZSP with silabs chips, all the endpoints are driven to the application level without registration. |
|
I'd highly recommend to just use one endpoint, perhaps two, for outgoing messages. Having one for each vendor scales not so well considering the growing number devices and vendors. Note you can send from one source endpoint (for example 0x01) to any destination endpoint, they don't need to be the same. On the incoming side it we have seen only one device in the wild which did expect a specific endpoint number on the coordinator, for that there was fix in the firmware to forward the command to the application. |
|
Manu,
This is not about sending from an ep it is about receiving a command to a specific ep in a client -> server communication ( server on coordinateur).
For now I don’t know if the issue is on the conbee or the zigpy layers which is not bringing the message to the appli a level
Envoyé de mon iPhone
… Le 3 avr. 2022 à 02:17, Manuel Pietschmann ***@***.***> a écrit :
I'd highly recommend to just use one endpoint, perhaps two, for outgoing messages. Having one for each vendor scales not so well considering the growing number devices and vendors. Note you can send from one source endpoint (for example 0x01) to any destination endpoint, they don't need to be the same.
On the incoming side it we have seen only one device in the wild which did expect a specific endpoint number on the coordinator, for that there was fix in the firmware to forward the command to the application.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.
|
|
Ah ok understand, you may check if the outgoing command from coordinator uses an existing source endpoint from the coordinator, the device the coordinator talks to should always pickup the right source endpoint from the message and use it as destination endpoint for replies. For commands which are send from a device to the coordinator without prior request the devices query endpoints and clusters first via ZDP Match_Descr_req (e.g. OTA, Time). For ZCL reporting the bindings need to be configured with a destination endpoint existing on the coordinator. |
|
We have reverse-engineered the all Wiser Legacy and despite how the Coordinator is configured there are some hard coded Request Attribute Request from Wiser End Device which send to a dedicated Ep on the coordinator. I'm talking about a ZCL command Read Attribute Request ( not a report, not a response) |
|
Interesting might be a good issue to forward to Wiser, do you have a modelid? |
FYI, zigpy does not yet support ZGP (Zigbee Green Power) so for zigpy ZGP discussion see zigpy/zigpy#656 and zigpy/zigpy#341 |
Manu this is Legacy Wiser from Schneider. Not supported anymore , so I don't think this make sense to forward the issue to them. Basically , we have (on the plugin side) do a full reverse engineering of the Wiser legacy equipement ( Thermostat, Valve, Actuator, and also Wiser sMeter . https://www.se.com/fr/fr/product/EER51000/wiser-thermostat/ So for me the ALL eco-system works really great on Zigate , ZNP, ENZP and unfortunately not on deConz . This is similar for the Livolo switch which send a Read Attribute Request to "report" its current state. |
|
@manup Thank you for the parameter definition! I have a question regarding endpoint "index" |
|
@manup have you been able to progress on the firmware able to manage more than 2 Eps ? |
|
@pipiche38 Can I suggest that you also post many new separate issues to https://github.com/dresden-elektronik/deconz-serial-protocol/issues for each specific issue so that to help @manup and others track different issues here in their upstream repository? |
|
@manup have you been able to progress on the firmware able to manage more than 2 Eps ? |
@manup (Manuel Pietschmann) Can I suggest that you (and @dresden-elektronik / Phoscon) try to reach out to @pipiche38 (Patrick Pichon) and his fellow developers about possible collaborating and teaming up to add compatibility for your ConBee USB stick and RaspBee Raspberry Pi Shield Zigbee Coordinator adapters together with the new "Zigbee for Domoticz Plugin" implementation for Domoticz home automation software that they are developing with zigpy as a new dependency for hardware-independency so that it can make use of different radio libraries like the zigpy-deconz radio library to add deCONZ Serial Protocol compatibility with your Zigbee Coordinator adapters in addition to Zigbee Coordinators from other manufacturers?
Also posted a matching request to that project here -> zigbeefordomoticz/Domoticz-Zigbee#1081
You can also read about that projects long back-story about utilizing zigpy radio libraries here -> zigpy/zigpy#865
Their new zigpy based implementation was recently merged into their "beta6" branch of their "Zigbee for Domoticz" plugin project as they have so far added support the zigpy radio libraries for both Texas Instruments (TI Z-Stack ZNP) and Silicon Labs EZSP (EmberZNet Serial Protocol) support via the zigpy-znp and bellows radio libraries respectivly, which are both together with the zigpy-deconz radio library and some other zigp radio libraries already used in by both Home Assistant's ZHA integration component and Jeedom's Zigbee plugin implementations as well, making this the third popular home automation software to publicly make use of the zigpy library and zigpy radio libraires to support Zigbee Coordinator adapters from different manufacturers:
https://github.com/zigbeefordomoticz/Domoticz-Zigbee/tree/beta6
https://github.com/zigbeefordomoticz/Domoticz-Zigbee/blob/beta6/readme.md#tested-hardware-zigbee-adaptersdonglesstickskeys
https://github.com/zigbeefordomoticz/wiki/blob/zigpy/en-eng/Home.md
https://www.domoticz.com/wiki/ZigbeeForDomoticz
https://www.domoticz.com/forum/viewforum.php?f=68
@manup If dresden-elektronik / Phoscon would help sponsor them with hardware by donating a few of your ConBee USB sticks and RaspBee Raspberry Pi Shields to @pipiche38, @badzz @SylvainPer and @jp-keros then maybe they would consider prioritizing adding support for and testing zigpy-deconz radio library integration next in order to support even more types of Zigbee Coordinators from different manufacturers. Perhaps they can mention that they use ConBee and RaspBee adapters as reference adapters for Phoscon/deCONZ hardware compatibility. No strings attached of course as these are still hobby projects at their core with only a few developers volunteering on that project so far.
Support for zigpy based radio libraries "Zigbee for Domoticz Plugin" for Domoticz is still in Beta development stage, but those guys have long been developing a previous "ZiGate Plugin for Domoticz" for Domoticz, (with Domoticz itself also being a popular free and open-source DIY home automation software). So at least you know that these guys already have the experience, skills, and personal interest to develop a new zigpy based Zigbee plugin for Domoticz.
Again, as I understand it, zigpy, bellows, Domoticz, and this new "Zigbee for Domoticz Plugin" are all only spare-time hobby projects but still, I think that teaming up might give all involved projects some more positive recognition in the Domoticz community and the whole home automation scene in general which might attract more developers and willing alpha/beta-testers from the community to come to help out with all these involved projects which in the long run could benefit all users wanting Zigbee in Domoticz to become more mature and working with more Zigbee hardware dongles than it is today.
The text was updated successfully, but these errors were encountered: