My perspective on Zigbee is one of a DIY-er and Maker; I’m primarily interested in making my own devices, rather than doing things with commercial devices. This means, among other things, that I definitely do not want to be reliant on proprietary “hub” software and want to keep control over complexity and cost. What follows are some brief notes on the technology choices which worked for me. The XBee platform looks a bit pricey, especially if the ultimate plan is for multiple DIY devices. This post condenses quite a bit of desk research and experimentation (I didn’t already have a Zigbee network, so started from zero); I hope it provides other people with a “jump start”. I won’t explain basics, so if you don’t already know what “coordinator”, “router”, and “end device” signify in a Zigbee network, look here.
The E18 modules from EByte are available in a number different models and I opted for the quite economical E18 MS1 PCB (this has a PCB antenna, whereas the similarly-priced E18 MS1 IPX has an IPX socket for an external antenna). Current (July 2023) prices on Aliexpress for E18 MS1 PCB, are: £2.61 or $3.18 + tax. Other options are available with power amplifiers, but for experimenting, the MS1 is just fine and I’ll only pay the extra (and suffer lower battery life) if I find a need. When I say E18 below, I will be particularly referring to the E18 MS1, although the comments may apply to other models.
The basic E18 module is in the form of a surface mount “stamp”, so I made some breakout boards to expose its many GPIOs. I made lots, so am selling on ebay. I’ve written a separate post about these boards, which contains a link to the ebay listing and to a github repository with Eagle CAD files (if you want to get your own boards made).
The big challenge with using the E18 is firmware. The factory firmware is quite specific and limited in what it can achieve, although it takes quite a while to work this out from studying the manual; it is intended for “transparent” data exchange between devices. This is not a good match for the kind of home automation devices which I’m interested in, and which I suppose most DIY/Maker types are. Another issue is that it is impractical for hobbyists to compile source code for the CC2530, the Texas Instruments system-on-chip which is the heart of the E18, because you need the IAR IDE costing several thousand dollars. However: the good news is that someone has produced a configurable firmware generator: PTVO. Another option is to use the Z-Stack “Zigbee Network Processor” (aka ZNP). This also avoids the need to compile for the CC2530 but is nowhere near as easy to use as the PTVO configurable firmware generator, so I will write a separate article on using ZNP.
PTVO Configurable Firmware is extremely helpful in that you set up how the GPIOs should be used as inputs and outputs, including use of multiple I2C sensors and more. It does all this without needing you to use a separate microcontroller; a single E18 (or similar) module interfaced to switches, sensors, indicators, etc is all you need. There are a few things which the free version cannot do, but the “premium” version is not at all expensive. Practical battery-powered end devices will require the premium version for the power saving mode; without this, I have measured PTVO firmware devices consuming over 70mA continuously.
The PTVO tool can also generate the converter JS files needed by Zigbee2MQTT (see below), which is very helpful indeed, since the documentation on writing these youreself is rather sketchy. While these converter files seem to mostly work “out of the box”, the one generated for a basic switch is not compatible with the firmware which is generated. I raised an issue on github but the developer didn’t seem to consider incompatible output to be a bug. One limitation of the PTVO firmware is that you can only use the devices which it supports and in the way it supports them (e.g. you cannot control over-sampling and other noise-reduction techniques). If these are issues then a resort to ZNP and a MCU with your own code to interface with the sensor/display/actuator is the easiest way ahead.
In addition to the E18s, which will be the heart of my Zigbee devices, I also obtained:
- The Ebyte E18 USB dongle (CC2531 chip), model E18-2G4U04B, for use as a Zigbee message sniffer. The pre-installed firmware works with the Texas Instruments Sniffer software (version 1, not version 2!) but this is not much use as it cannot decrypt encrypted messages. So: I installed the ZBOSS sniffer firmware and use the ZBOSS extension for Wireshark (see setup notes on Zigbee2MQTT site). This works mostly nicely and is quite informative. You DO need to select the right networking channel (the default in Zigbee2MQTT is 11, but I changed to 19 to reduce WiFi interferrence). Note that this hardware is not well suited to running the Zigbee 3 coordinator firmware or supporting many devices, and is quite low power (specifics are documented on the Zigbee2MQTT site).
- A bundle comprising another CC2531 USB dongle (but this one with an external antenna, and intended to be the network coordinator), a CCDebugger and associated connection cables AND an adapter PCB with 0.1″ and 0.05″ headers. There are plenty of these on ebay, Aliexpress, and Amazon. I installed the Z-Stack 1.2 HA coordinator firmware on this dongle using the CCDebugger and TI Flashing software (this is where the 0.1″ to 0.05″ adapter board and cables come in). I also found that the Ebyte dongle had a single row of 0.05″ spacing holes for programming and was able to slide these over 5 of the 10 pins on the adapter board and then make ad-hoc connections to the programmer. This worked first time, and is much cleaner than solding “Du Pont” cables as suggested on the web.
On the question of programmers: I think the CCDebugger is the easiest way and, when bundled with the adapter and dongle is a “no brainer” purchase if you’re getting set up from scratch. I have also used other options with a Raspberry Pi, Flash-CC2530 (see my other post), and if you conduct a web-search for “CCLoader” + “ESP8266” or “Arduino”, you will easily find ways of using those MCU platforms as programmers. Some of these are for the CC254x devices, but the communication protocol is the same. Use “.hex” (Intel Hex) files with TI SmartRF Flash Programmer and “.bin” files with the other options. If you need to create bin from hex files, try using hex2bin (noting that you may need to use padding pad). The CCDebugger and Flash-CC2530 also show you the chip id, which can be helpful to find out whether there is a CC2530 or CC2531 inside (or not). Chip Id helped discover that multiple BLE modules I bought had mis-labelled chips as well as incorrect firmware: total fakes! Whatever you use for programming, only the DD, DC, and Reset signals are needed (in addition to Gnd and supply). If you use the CCDebugger, note that it provides a 3.3V power supply but also has a Vsens (read the manual!). I tend to program “out of circuit”, using the CCDebugger’s power supply and simply connect Vsens to the debugger’s power supply.
Now to the important question of the coordinator, and the bridge between planet-Zigbee and the rest of the universe. Zigbee2MQTT is freely available and non-proprietary and has some nice features:
- It provides a bridge to a MQTT broker. This is a smart move for integration of different sense-and-control technologies. In my case, I already have a MQTT broker running as part of my Open Energy Monitor (inside it has a Raspberry Pi3) and I have co-opted the broker to be a hub for several air quality and temp+humidity sensors around my house and garden based around the ESP8266 Wifi MCU. There are numerous PC and Android (and doubtless Apple) Apps which can subscribe and publish to MQTT and I’ve written several Python dashboards using Plotly Dash and the Paho MQTT package. Using MQTT allows you to integrate Zigbee and non-Zigbee “worlds”, although the interactions will not be as rapid as a pure Zigbee system. If you know nothing about MQTT: find out!
- It can run on a Raspberry Pi v3 or v4 (but not my old v1). I run it in a RPi v3 with its WiFi turned off, and have not had to resort to a USB extension cable for the coordinator dongle (some people recommend such an extension to avoid radio frequency interferrence).
- It has a nice web interface which can be used to monitor the Zigbee network, inspect device status, control devices, and “bind” devices together.
- Kantor Koen (the brains behind Zigbee2MQTT) provides firmware for the coordinator.
- Mostly good documentation, although I could not get it to install on Windows 11 by following the instructions and the documentation on writing converter JS files is not adequate (not that this is a problem when the PTVO JS files work).
The question of radio frequency interferrence is referred to on the Zigbee2MQTT website, forums, and elsewhere but much of the information is unhelpfully inaccurate due to omission of caveats. The core issue is that WiFi and Zigbee share the same frequency band. I have widely seen that the assertion that there are only three non-overlapping WiFi channels: 1, 6, and 11. The central neglected caveat is that in the USA, unlicenced use is limited to channels 1-11. Much of the rest of the world can use channels 12 and 13. So, while 1, 6, and 11 are the sensible choice for optimal separation in the USA, there are actually 4 non-overlapping channels elsewhere: 1, 5, 9, 13 (a separation of 4 channels is quite adqeuate). This gives scope for operating 3 WiFi channels AND having uncontested space for a Zigbee network. NXP also has an application note (pdf) which shows the results of experiments on WiFi/Zigbee interference which suggests that concerns about wide “sideband” noise are not really warranted if you are using “802.11n” WiFi.
Summary
Ebyte E18 USB (CC2531) dongle and ZBOSS for a Zigbee sniffer.
Another CC2531 USB dongle with an antenna running Z-Stack 1.2 HA coordinator firmware attached to a Raspberry Pi v3. This is a bit low powered for serious use, but is fine for experimenting, prototyping, and small-scale use; I’ll upgrade if needed.
E18 modules on my breakout PCB runing PTVO firmware as “terminal” devices.
[optionally also E18 modules with PTVO “router” firmware, although I might end up using yet more USB CC2531 dongles as routers on the grounds that they are easy to power]
Zigbee2MQTT running on a Raspberry Pi v3 (in my case I used the MQTT broker on my Open Energy Monitor, rather than running on the same Pi).
For monitoring and publishing MQTT while experimenting I tend to use MQTT Explorer on my Windows PC. There are several Android apps which can do plotting, dashboards, and control widgets; some are better than others, depending on what you want…