After upgrade 7.11.3->7.14.2 Zerotier no longer connects

After upgrading an RutX11 from 7.11.3 to 7.14.2,
Zerotier on this RutX11 no longer connects to any peers
and on Status /Services its Status is shown as Enabled but Down.
Also, my main console at Zerotier shows it is not connecting.

The Events log doesn’t seem to show anything,
just my rebooting and re-saving the settings.
Zerotier settings have not changed.

I have tried disabling it, rebootiong then re-enabling then rebooting again,
but no change - still doesn’t connect.
All my other Zerotier nodes connect.

What has changed between 7.11.3 and 7.14.2 to stop Zerotier from connecting?
Any suggestion for how I can debug this?

Update: just updated an RutX10 from 7.12 → 7.14.2 - Zerotier connects with no problem.

So the problem is only with an RutX11 from 7.11.3 to 7.14.2.

Hello,

Could you please confirm whether you performed the update with “Keep settings” enabled or not?

Best regards,

Hi Marija

Apologies, I should have mentioned, I performed the update with “Keep Settings”.
I always use “Keep Settings” as redoing the settings takes a few hours,
and I’m never sure if I get everything right.

Much More Useful Diagnstic Information

I ssh’d into a router on which zerotier is working (syd5)
and the one it is not working (syd1)
and compared them.

—syd5—
:>zerotier-cli status
200 info [node-id] 1.14.0 ONLINE
:>which zerotier-one
/usr/local/usr/bin/zerotier-one
:>ls /var/lib
misc zerotier-one zerotier-one_[node-id]_2
where zerotier-one is just a link to zerotier-one_[node-id]_2

—syd1—
:>zerotier-cli status
zerotier-cli: missing port and zerotier-one.port not found in /var/lib/zerotier-one
:>which zerotier-one
/usr/local/usr/bin/zerotier-one
:>ls /var/lib
misc

So obviously zerotier-one is not being started on syd1.

So two Questions:

  1. why isn’t zerotier being started? I checked the config file
    /etc/config/zerotier
    and it seems correct.

  2. It seems likely that the best solution is to uninstall and reinstall zerotier,
    but making sure all old configuration files are purged by doing a manual check.
    Do you agree?

It seems the problem is more serious.
I tried to uninstall zerotier hoping an uninstall/ reinstall would fix the problem.

I removed the zerotier configuration in Services/vpn/zerotier,
then tried three times to remove zerotier with the package manager.
It says “Package Remove Action Started Successfuly”
but I then reboot - and it still shows zerotier in the package manager as Installed.

If I ssh into the router after trying to uninstall zerotier, the following files are still there:
:>find . -iname zerotier*

./etc/config/zerotier-opkg
./etc/config/zerotier
./etc/init.d/zerotier
./overlay/etc/upper/init.d/zerotier
./overlay/etc/upper/config/zerotier-opkg
./overlay/etc/upper/config/zerotier
./overlay/root/upper/usr/bin/zerotier-idtool
./overlay/root/upper/usr/bin/zerotier-cli
./overlay/root/upper/usr/bin/zerotier-one
./overlay/root/upper/usr/lib/lua/api/services/zerotier_network.lua
./overlay/root/upper/usr/lib/lua/api/services/zerotier.lua
./overlay/root/upper/usr/lib/opkg/info/zerotier.postinst-pkg
./overlay/root/upper/usr/lib/opkg/info/zerotier.prerm-pkg
./overlay/root/upper/usr/lib/opkg/info/zerotier.conffiles
./overlay/root/upper/usr/lib/opkg/info/zerotier.prerm
./overlay/root/upper/usr/lib/opkg/info/zerotier.postinst
./overlay/root/upper/usr/lib/opkg/info/zerotier.list
./overlay/root/upper/usr/lib/opkg/info/zerotier.control
./overlay/root/upper/usr/share/rpcd/acl.d/zerotier.json
./overlay/root/upper/usr/share/vuci/menu.d/zerotier.json
./usr/local/usr/lib/lua/api/services/zerotier_network.lua
./usr/local/usr/lib/lua/api/services/zerotier.lua
./usr/local/usr/lib/opkg/info/zerotier.postinst-pkg
./usr/local/usr/lib/opkg/info/zerotier.prerm-pkg
./usr/local/usr/lib/opkg/info/zerotier.conffiles
./usr/local/usr/lib/opkg/info/zerotier.prerm
./usr/local/usr/lib/opkg/info/zerotier.postinst
./usr/local/usr/lib/opkg/info/zerotier.list
./usr/local/usr/lib/opkg/info/zerotier.control
./usr/local/usr/share/rpcd/acl.d/zerotier.json
./usr/local/usr/share/vuci/menu.d/zerotier.json
./usr/local/usr/bin/zerotier-idtool
./usr/local/usr/bin/zerotier-cli
./usr/local/usr/bin/zerotier-one

I can start it with
:>zerotier-one
but this starts it with the wrong configuration file and node-id so it not the correct way to start it.

I can also start/stop zerotier with
:>/etc/init.d/zerotier start
:>/etc/init.d/zerotier stop
but I guess this also starts it with the wrong configuration file and node-id.

It wasn’t installd with opkg as:
opkg remove zerotier shows:
No packages removed
and opkg list_installed doesnt show zerotier.

So why won’t the package manager remove it?
Is there a cli comand I can use as root to try, and maybe see any messages as to why it wont remove?

In a similar fashion wireguard has also stopped working with the current update.

I had to roll back to 7.13.* the other day to be able to set a device up, no doubt related.

Hello @alienheartbeat ,

I’ve sent you a form to fill out. Once completed, I will contact you privately. Please use the ticket ID “13798” when submitting the form.

Best regards,

reply to ewr:

Just tested on two RutX10’s in different countries (both 7.14.2)
running wireguard. both worked fine.

However I note The Zerotier problem is also not universal.
Zerotier is working on two of my RutX10s and an TCR100.

(btw, how do you downgrade firmware? I can’t see any option to restore previous version.
Axis cameras have an option to return to a previous version on the update page).

Thanks Marija, will do - now done.

Thank you. You will receive an email from us shortly.

Best regards,

I have the same issue with wireguard
[ RUTX11 lost after upgrade from 7.13.2 to 7.14.2](RUTX11 lost after upgrade from 7.13.2 to 7.14.2)

Thanks to Teltonika support for their assistance resolving this.

Results / Solutions

1 The lack of response of Zerotier was likely caused by
the fairly large firmware gap upgrading from 7.11.3 → 7.14.2.

2 It turned out this wasn’t the only problem encountered,
so rather than try to identify and resolve each problem separately,
it was best to do a reset to factory defaults
which did resolve all problems.

3 It may be hard to know which large jump upgrades are safe and which are not.
In particular, while 7.11.3 → 7.14.2 on RutX11 went poorly,
7.12 → 7.14.2 on RutX10s went with no problems.
It may also depend on the complexity of the device.
RutX10s, with no modem and the associated settings (eg fallback),
seem more tolerant than RutX11s.

4 However the Teltonika devices may be installed in locations
that can’t be accessed locally for extended periods,
(eg I have them in multiple countries)
so large firmware gap upgrades may be unavoidable.
In this case there are 2 strategies:
a stepped updates, say 7.11.x → 7.12.x → 7.13.x → 7.14.x.
b Reset to Factory Defaults

Other Useful Notes:

5 To remove Zerotier if it can’t be removed from the gui,
we can either ssh in or use the cli, then:
opkg remove Zerotier vuci-app-Zerotier-ui vuci-app-Zerotier-api

6 After a Reset to Factory Defaults,
extra packages installed via the gui, eg Zerotier,
or via ssh/opkg, eg nano, are removed.
The system however remains at the latest installed version
(ie does not revert to the factory installed version).

7 One other technique I found useful:
having set a Users Default Configuration
then when I lost contact with the device
I was able to regain contact by restoring this Users Default Configuration
by holding the reset button 6-11 seconds.
In some cases this may save a full reset to factory default,
or at least enable you to make a full record of all settings (eg port forwards)
prior to doing a factory reset.
I generally update the Users Default Configuration after I have done signifcant changes
and ensured theye are working.

1 Like

Hello @alienheartbeat,

Thank you very much for your detailed feedback regarding this issue — we truly appreciate it!

If you need any further assistance or have additional questions, please don’t hesitate to reach out.

Best regards,

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.