Problems with RMS

[Danny01]

Hi, I’m currently having a problems with RMS, I need to configure important settings on my routers but it just time out, help asap!

[AndzejJ]

Hello,

Indeed, there were some issues with RMS links this night/morning. However, those issue should be resolved now. If you continue to face any issues with the RMS links, please let me know.

Apologies for any inconvenience this may have caused.

Kind Regards,

[slightly-stapler]

We’ve seen some issues with RMS as well these past couple of days.
We have 2 TRB140 devices on TRB1_R_00.07.06.4 which are reported as “offline” in RMS.
Since they’re at a remote location, serving as an OoB mgmt channel for other devices, I can’t easily access the WebUI in this situation.
The managed devices are reachable via the Internet connection the TRBs provide, so I am able to access the CLI of the TRBs themselves in this convoluted way.
I can ping 1.1.1.1 and google.com by hostname, I tried some of the options given in this forum (/etc/init.d/rms_mqtt restart - doesn’t make any difference).
Netstat shows some attempt to connect to RMS, but it seems to go nowhere:
root@Teltonika-TRB140:~# netstat -n -p -t
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 10.54.114.140:58918 3.66.40.246:80 TIME_WAIT -

I’m considering doing a factory reset via CLI, but I’m a bit wary of the end result.
If the TRB keeps providing Internet + LAN IPs via DHCP, the other device will eventually recover and I should in theory be able to SSH into the TRB again.
This is riding on a lot of assumptions, so I’m hesitant to go for this approach if there’s a better way.

I feel like I’m missing something obvious, like maybe some tick or license somewhere.
Did anyone hit this sort of behaviour previously and can share what fixed it?

[slightly-stapler]

Ok, so I’ve spent some more time on troubleshooting our particular issue and am documenting here what I did to fix it.
I did some “/etc/init.d/rms_mqtt restart” commands in one window and did a tcpdump in another.
Turns our there was communication to RMS, but the TRB was receiving a TCP RST from it.
I managed to find in the forum a handy command which I wasn’t able to find previously and sort of found a root cause:

root@Teltonika-TRB140:~# ubus call rms get_status
{
“next_retry”: 1706620202,
“connection_status”: 1,
“error_text”: “Server refused connection. The device may be blocked or unidentified”,
“error_code”: 17,
“error”: 1
}

The fix was to unregister and re-register the device, not sure what went wrong, but if it works, it works.
For one device, I had to do it multiple times, which was a bit strange.
With the latest 07.06.4 firmware we enabled the auto-reboot functionality to hopefully not lose the management to the devices again.
In our experience the older firmware especially tends to go offline in the RMS, even if there is an Internet connection.
There are also all sorts of weird SSH bugs, sessions can’t be established and the like.
I really struggled with finding a way to check via the CLI if RMS management was enabled on the device itself.
Hopefully this can be improved in the future, so that there is feature parity between CLI/HTTPS.
If you found this message while troubleshooting, I hope it helps you with your problem.