Rutx50 has become unreliable

Teltonika Networks
1

I have been using an RUTX50 as my main ISP modem at home for several months. Recently I’ve been having internet services pausing either for a few minutes or getting completely stuck.

My router Ubiquiti UDM Pro SE has the RUTX50 in passthrough mode as WAN 1 and a slow BT DSL as WAN 2. Whenever the internet hangs, the failover does not work.

Digging further - UDM Pro is using ping 8.8.8.8 as the WAN test - as this keeps working there is no failover. Some apps will carry on working - Facebook app for example, but anything trying to establish a connection will fail. Ping seems to be unaffected however.

The Teltonika speed test fails - failed to find provider / failed to find ip.

I can re-establish the service via the RUTX50 web page:
Status → Network → Mobile → Restart Connection

  • sometimes I will also need to restart my browser to get hanging tabs to work again.

Firmware is: RUTX_R_00.07.03.3
I took some diagnostics before and after the restart, but not sure where to send them.

regards
Steve

5

Hello,

Is it correct that the RUTX50 is connected to the WAN port of the Ubiquiti UDM Pro?

Could you please provide more details about how the failover should function in your situation? Does the RUTX50 have any other WAN sources apart from the cellular mobile connection?

Have you considered connecting the BT DSL to a WAN port on the RUTX50 and configuring failover on the RUTX50, where the mobile interface serves as the primary interface and WAN acts as the failover?

Regarding the connectivity issues related to the mobile connection, it’s challenging to determine the exact cause.

Could you please let me know the firmware version of the internal modem? You can find this information under Status → System.

Additionally, could you provide the signal strength and quality parameters? You can find these details under Status → Network → Mobile.

Maybe you had an opportunity to test the connectivity via a different SIM card from another provider?

Have you tried configuring the mobile interface on the RUTX50 in NAT mode instead of bridge mode? Does the same issue persist with this configuration?

For now, I recommend setting up a ping reboot in Services → Auto Reboot → Ping/Wget reboot. You can choose to restart the mobile connection or perform a modem reboot as the action. The former option should be quicker, so I suggest trying it first to see if it resolves the issue.

Furthermore, it appears that you are currently running FW 07.03.3. A newer version is available, and updating it might potentially offer more stability. However, I suggest updating without selecting the ‘keep settings’ option, as there could be some migration issues due to significant differences between your current firmware version and the latest one.

At the moment, there is no secure method to send files privately. If you decide to share information publicly, please ensure that you blur or remove any sensitive information before posting.

Kind Regards,

6

Hello,
the RUTX50 is attached to the UDM Pro SE on Port 8 ethernet (I’ll activate PoE on that port shortly) - this is set at the WAN port. Port 9 Ethernet is set as WAN2 for the BT Hub. The RUTX50 has a single EE unlimited SIM for the wan connection.

The Unifi system performs a regular ping test to check WAN is working - unfortunately ping 8.8.8.8 carries on working so the fail over is never triggered.

I can’t connect the BT DSL to the RUTX due to cabling limitations, but I think the failover test is also a ping that carries on working.

the firmware version is RUTX_R_00.07.03.3

Signal parameters are

Operator EE EE
Operator state Registered, home
Cell ID 5B3F00
Data connection state Connected
Connection type 4G (LTE); VoLTE
Signal strength -57
Carrier Aggregation Single
Connected band LTE B3
RSRP -82
RSRQ -6
SINR 30
Bytes received 97.4 MB
Bytes sent 28.2 MB

There are no other providers nor sims within range.

It was originally in NAT mode and was stable, but have not tried that recently. It probably won’t offer any functional changes as EE uses CGNAT, effectivly blocking all inbound connections.

Ping reboot is set to 8.8.8.8. and that still responds while new connections cannot be established.

I’ll try the firmware update this evening. 07.03.3 to 07.04.3 - or should I download the 97.04.4 - that is not yet on the RUTX50 admin download page?

7

Hello,

The internal modem in RUTX50 has its own firmware. Could you please check the internal modem’s FW version in Status → System? It should start with RG.

You can also check if you have any new internal modem updates available in System → Firmware. If so, you can try updating.

The latest firmware for the device itself is available here.

Also, are you able to access the WebUI of RUTX50 when the issue occurs? If so, and since the pings are successful, have you tried running a speed test or updating the list of packages in the package manager from the webUI (or from CLI via ‘opkg update’)? Are those successful?

Kind Regards,

8

Hello Andzej,
I updated the device firmware to RUTX_R_00.07.04.4 yesterday evening – after it was hanging again ( the problem usually occurs in the evenings).

The internal modem firmware version is shown as:
RG501Q-EU
RG501QEUAAR12A05M4G_04.001.04.001

The download link does not show the version that will be installed (the update button does not give a confirm option either!). So the modem upgrade is now running!

Re: Also, are you able to access the WebUI of RUTX50 when the issue occurs? If so, and since the pings are successful, have you tried running a speed test or updating the list of packages in the package manager from the webUI (or from CLI via ‘opkg update’)? Are those successful?

When the problem occurs, the RUTX50 is accessible, and its speed test fails to get the list of servers. I just ran speedtest and it is working normally at the moment.

After the modem update the version reported is
RG501QEUAAR12A07M4G_04.001.04.001

Oddly, once the update completes, the firmware page still says update available. ??

Regards
Steve

9

Hello,

Please proceed with updating the internal modem’s firmware once again. Since you’ve already updated it to version R12A07, you should be able to further update it to the latest version, R12A08.

However, considering that the issue occurs specifically in the evenings, it’s possible that the problem is related to the mobile network itself or your mobile operator.

When the connection becomes unstable, do you take any actions to restore stability, such as rebooting the device? Alternatively, does the issue resolve on its own after some time?

Let me know if the connection became more stable after the updates.

Kind Regards,

10

Hello,
this morning the network was hanging when I woke up.

I did some more testing - ping still worked, but curl http did not

steve@nutserver:~ $ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=55.6 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=113 time=40.9 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=113 time=49.8 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=113 time=48.9 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 40.866/48.801/55.623/5.258 ms
steve@nutserver:~ $ ping twitter.com
PING twitter.com (104.244.42.129) 56(84) bytes of data.
64 bytes from 104.244.42.129: icmp_seq=1 ttl=52 time=55.1 ms
64 bytes from 104.244.42.129: icmp_seq=2 ttl=52 time=40.9 ms
64 bytes from 104.244.42.129: icmp_seq=3 ttl=52 time=43.8 ms
64 bytes from 104.244.42.129: icmp_seq=4 ttl=52 time=51.7 ms
^C
--- twitter.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 40.869/47.861/55.136/5.760 ms
steve@nutserver:~ $ curl twitter.com
^C
steve@nutserver:~ $ curl -v https://twitter.com
*   Trying 104.244.42.129:443...
* Connected to twitter.com (104.244.42.129) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):

The speedtest failed, but I could access all the RUTX50 web pages. Count not add screenshots due to new member status.

I rebooted the RUTX50 and the connection was working again.

I then went on to update the firmware to the 08 version. Note the update in progress banner does not go away until I refresh the browser - the screen shows the new version but also shows update in progress.

IMG_1105
IMG_11051920×1439 115 KB

I’ll let you know how the latest firmware works.

Steve

11

Connection has been stable the past five days.

:slight_smile:

12

Next time that happens could you check if /tmp contains coredumps from netifd:
ls -al /tmp/netifd*

13

Hello,
after some weeks of stability the problem reoccurred this evening.

Again I could telnet to the RUTX50 and ping 8.8.8.8, but curl https://google.com would hang.

There were no dumps in the tmp folder. I made a list of all the files / folders under /var in case that gives some clues, but my user permissions do not allow me to attach it.

After this I tried shutdown -r, but this was also not successful. Finally I had to physically access the device and pull the power to restore service.

regards
Steve

14

The command to restart the device is simply “reboot”.

15

Hello,

You have mentioned that the restart of the device or restart of the modem brings the connection back. Do you obtain a new IP address each time by any chance?

In your case, I would recommend configuring auto reboot in Services → Auto Reboot → Ping/Wget reboot. Specifically, try using Wget method instead of ping and set the action to restart mobile connection or modem reboot. This way, even if the pings are successful when the issue occurs, Wget should fail. This should result an automatic restart of only the mobile connection/modem. Please, let me know if this works for you.

Kind Regards,

16

I would guess, that I have the same problem with unriliability.
In some moment the mobile internet connection is broken. Always till some reboot. Nothing what I tried over the webinterface solved the problem, including restart of the modem. Sometimes the problem immediately exists after reboot. This is why the scheduled reboot at night not always helps. A new reboot mostly helps 24h later. A reboot on failed ping to 8.8.8.8 seems not to work at all.

Firmware version RUTX_R_00.07.04.4
Firmware build date 2023-06-29 12:15:56
Internal modem firmware version RG501QEUAAR12A08M4G_04.200.04.200
Kernel version 5.4.229
17

Even if the wget might work, it can not be the right solution to regularly need to reboot the device. I was thinking to use other services on it, like MQTT. But with this reliability I would not dare to.

18

Specially It woult start to regularly reboot in areas without mobile service. This is really no solution!

19

Hello,

@Nils

Could you please provide more details about your case? How often this happens?

You mentioned that the connection is not restored on modem reboot. What do you do to restore the connection? Does it come back by itself after some time?

Does the device completely disconnects from the operator or like in the Steve’s case, it is connected and pings are successful, while general traffic fails?

How are you using your RUTX50? Maybe its it bridge/passthrough mode?

Kind Regards,

20

It happens in average every 2nd or 3rd day, under stationary conditions, when the router stays in one place. Sometimes a week without problems, sometimes less then a day. It never came back by itself so far. Once having this disconnect on site, the webinterface showed a crossed out symbol at the signal stregth visualization as far as I remember. The mobile network was perfectly available at that moment. But even a modem reset didn´t regain connection. Restore of connection was always only gained by a full reset or power cycle. If the ping would work, no idea, probably not, I am currently not on site. I can currently only communicate, when the the connection is established. But the restet on failed ping never worked. Only the scheduled reset once per 24h normally reestablishes the connection, but even that not always - then it´s sometimes another 24h offline.
It´s not bridge/passthrough.
By the way: The modbus datapoint “GSM operator name” (addr. 23) does not delivers anything at any time. Just nothing. All other datafields deliver consistent data, but not this one. Exactly this one would be valuable under travel to understand roaming.

21

Hello,

Do you have any other configurations on the device? Like failover, ping-reboot, etc?

When the issue occurs, would it be possible for you to show a screenshot of the Status → System and Status → Network → Mobile pages? Make sure to hide sensitive information, such as IMEI, IMSI, etc.

Additionally, after the issues occurs, navigate to System → Administration → Troubleshoot and ‘view’ system logs. Post the logs around the time when the issue occured (again, hide any sensitive information).

I would also suggest updating to the latest firmware with ‘keep settings’ option disabled, as this has proved to help with stability in some cases as well.

Regarding the ‘GSM operator name’, this is a known issue where nothing is delivered if the device is unable to obtain operators name. This should be fixed with v7.5 firmware.

Kind Regards,

22

Indeed, there is (now was) a fail over/load balancing active to an external AP (which is currently physically not present at the site - my phone).

When I am next time on site and experience such a disconnect, I will do the screenshot. As well the fw update. If I would initiate the fw update now (without keeping settings) then I would loose any contact to the device.

What I could do in the meanwhile, record any remote accessible state via Modbus or MQTT, what might give hints during beeing offline. Any whishes for an external loggin of such a data? What would give us some information?

23

Hello,

Since the device loses connection, you won’t be able to monitor it remotely. It is possible that System logs may provide some insight into the issue. You can view system logs in System → Administration → Troubleshoot. You can also download a troubleshoot file which contains a lot of information. However, since this is a public forum and there is no way to share information privately, it is recommended to avoid sharing this file publicly.

Could you please clarify if the you are using an Auto-APN option, or you entered you APN manually? I would suggest trying to enter it manually in Network → Interfaces → Edit mob1s1a1 interface. Though this is a long shot and may not be the reason of disconnects.

Also, was this issue present when you first started using the device or it started to occur after some time? You can try the bootloader procedure as described here to reinstall the firmware. This would eliminate any potential configuration issues or firmware glitches.

Anyways, please post the logs whenever you will have the opportunity.

Kind Regards,