RUTX12 sporadically reboots after FW upgrade to RUTX_R_00.07.07

After I’ve upgraded FW of my RUTX12 to v RUTX_R_00.07.07.1, it started to reboot sporadically. Same holds with the version RUTX_R_00.07.07.2 as well. I use the router couple of years already and it’s the first time I’ve started seeing “Device startup after unexpected shutdown” system event in its event log. I’m pretty sure (200%) that the power is perfect there, I even additionally tried it with BAT120, same result. No device overheating or something like that. Purely FW stuff. Here is my system event log after upgrade to RUTX_R_00.07.07.1:

ID Date Event Type Event
100 2024-06-23 01:31:08 Startup Device startup after unexpected shutdown
99 2024-06-22 14:37:39 Startup Device startup after unexpected shutdown
98 2024-06-22 08:33:12 Startup Device startup after unexpected shutdown
97 2024-06-21 20:55:58 Startup Device startup after unexpected shutdown
96 2024-06-21 20:32:56 Startup Device startup after unexpected shutdown
95 2024-06-21 19:27:39 Startup Device startup after unexpected shutdown
94 2024-06-21 18:45:27 Startup Device startup after unexpected shutdown
93 2024-06-21 17:55:12 Startup Device startup after unexpected shutdown
92 2024-06-21 12:06:29 Startup Device startup after unexpected shutdown
91 2024-06-20 18:56:48 Startup Device startup after unexpected shutdown
90 2024-06-20 14:07:16 Startup Device startup after unexpected shutdown
89 2024-06-19 23:24:07 Startup Device startup after unexpected shutdown
88 2024-06-18 01:06:49 Startup Device startup after unexpected shutdown
87 2024-06-16 15:06:02 Startup Device startup after unexpected shutdown
86 2024-06-14 08:04:17 Startup Device startup after unexpected shutdown
85 2024-06-13 08:04:31 Startup Device startup after unexpected shutdown
84 2024-06-13 07:51:10 Startup Device startup after unexpected shutdown
83 2024-06-12 13:31:00 Reboot Request after FW upgrade
82 2024-06-12 08:06:24 Startup Device startup after unexpected shutdown
81 2024-06-10 00:22:36 Startup Device startup after unexpected shutdown
80 2024-06-09 22:16:12 Startup Device startup after unexpected shutdown
79 2024-06-08 18:43:31 Startup Device startup after unexpected shutdown
78 2024-06-08 09:48:49 Startup Device startup after unexpected shutdown
77 2024-06-05 15:11:09 Startup Device startup after unexpected shutdown
76 2024-06-04 07:26:00 Startup Device startup after unexpected shutdown
75 2024-06-02 04:22:59 Startup Device startup after unexpected shutdown
74 2024-05-29 12:49:17 Startup Device startup after unexpected shutdown
73 2024-05-29 12:45:52 Reboot Request from Web UI
72 2024-05-29 05:31:37 Startup Device startup after unexpected shutdown
71 2024-05-29 03:36:10 Startup Device startup after unexpected shutdown
70 2024-05-26 19:14:01 Startup Device startup after unexpected shutdown
69 2024-05-22 01:46:42 Startup Device startup after unexpected shutdown
68 2024-05-15 08:58:36 Startup Device startup after unexpected shutdown
67 2024-05-05 02:23:27 Reboot Request after FW upgrade

Do you usually ‚Keep Settings‘ when upgrading over the years as you said?
If so, it would be worth taking a backup on 07.07.2, set the box back to factory settings and either re-configure or restore backup (important: same box, same firmware version it must be!)
If your config is not too complex, just invest a few minutes in re-configuring by hand.

If you ‚Keep Settings‘ over the years you also keep the waste and don’t get changes in the default config made by Teltonika.

You want to give it a try?

I see there is a new FW released, with a “fixed config when upgrading from older firmware with keep settings” in release notes, 07.07.3. I’ll try this one first. If it will not help, then I’ll try to reset with clean settings and then reconfigure.

Good plan.
Let us know how it went!

I tried 07.07.3 and it didn’t help. I’ve got already “Device startup after unexpected shutdown” 2 times since FW upgrade. So now I’ve just tried backup of 07.07.3, then factory reset, then backup restoration. Let’s see how it goes.

Hello dshoot,

In order for us to be able to troubleshoot the root cause of the issue, we will need a troubleshoot file, but as the device is unexpectedly rebooting, we will need to save these troubleshoot files in Flash memory.

In order to save it into flash memory, login to your WebUI → navigate to System → Maintenance → Troubleshoot → " Logging Settings" section, there you will see “Save log in” option and choose “Flash memory”.

As the troubleshoot file will not be accessible that easily, I would want to make a remote session, where we could access the troubleshoot file and see what is the cause of the issue.

For further communication I have sent you a Hubspot instructions to your e-mail, check your inbox and there refer to further steps, your ticket ID will be 6460.

Kindly waiting for your reply,
DziugasS

Upgrade to 07.07.3, then backup, then factory reset, then backup restoration didn’t hep either. I’ve got one more ‘Device startup after unexpected shutdown’ in 4 days after that. I followed your instructions from email for the ticket 6460.

Hello,

Thank you for providing the troubleshoot file.

I see that the wireless interface is spamming error messages, which seems to shutdown the device.

ath10k_ahb a800000.wifi: Invalid VHT mcs 15 peer stats

There is plenty of these logs, after which the IPv4 connectivity starts to fail. Once the logs are flooded, the device shutdowns and reboots. I have tried uploading the wireless settings that you were using and my router started generating the same logs, which is why I think you need to change the settings of your wireless network.

By testing out some configuration changes, I noticed that once I have disabled Short Preamble, the logs stopped generating. Could you try disabling this option and then testing it out and see how it goes? Go to your router’s WebUI Network → Wireless → SSIDs → Edit the interface and choose Advanced Settings and disable Short Preamble

Do the same on both SSIDs and test it out and see if your device shutdowns unexpectedly.

Let me know how it goes,
DziugasS

Thanks!
I configured Wi-Fi as you suggested. Let’s see how it will work now.

I tried this Wi-Fi configuration and it didn’t help. I have 2 more sporadic restarts in logs after that. It seems that the only option to try which is still left is factory reset and then manual reconfiguration. Otherwise I can send you one more fresh troubleshoot file if needed. Should I send new one to you?

Hello Dshoot,

I have contacted you via hubspot once again, where I have provided a more private solution, please check your inbox and continue the conversation there.

In the meantime have a nice day,
DziugasS

Thanks! I decided to switch directly to the newest 07.08. I made a reset and reconfigured all manually from scratch after switching to 07.08. Hopefully it will work.

Hello Dshoot,

Let me know how it goes and if any issues arise with unexpected reboots, please generate a troubleshoot file and provide it via hubspot, so I can look into the issue further.

Best regards,
DziugasS

Hmm, I’m a bit frustrated now. I’ve upgraded to 07.08 without keeping settings, I reconfigured it all after that. Then I still got one more sporadic reboot. After that I turned off short preamble again on both SSIDs. And still got one more sporadic reboot after that. That’s very weird. Not sure what I should do next. Meanwhile I turned logging into flash again.

Hello Dshoot,

Thank you for informing us,
If the unexpected reboot will happen once again, please send us the troubleshoot file (that is saved to flash), so we can look into the issue further, maybe the issue persists in another place.

Best regards,
DziugasS

Hi.
I’ve sent fresh troubleshoot file, last 3 reboots are sporadic there.
That’s 07.08, reconfig from scratch + no short preamble.

Hi. Could you please confirm that you have received my new troubleshoot file? I continue seeing new sporadic failures in logs.

Hello Dshoot,

Yes I have gotten the troubleshoot file, which was generated on 2024-08-01.

By the logs I have noticed that there was DNS-rebind attacks (or maybe you wanted to contact that specific address), could you clarify, do you communicate/interact with “j&&&hammer-&&&&.data&&&ion.ide&&.com?” or is the router targeted and attacked by such interactions? As I can see that rebind protection is turned on, so it is denied. Other than that, I can see that there is mobile modem issues, have you noticed them with older firmware versions, or does the mobile connection in this use case is used as a backup WAN?

Kindly waiting for your reply,
DziugasS

Hi. Thanks for quick reply. That address which you mentioned is in the private VPN network. So it could be that when I have disconnected from VPN and left browser opened, it stilled tried to connect to the back-end. So it’s not an attack for sure. Mobile currently is only a backup for the WAN, so not in use currently. Sits in standby as a backup all the time. I haven’t noticed any mobile related issues with an old firmware. But I actively used it as a primary source more than a year ago, so not sure if there were any issues with it on the old firmware. But what I know for sure is that sporadic reboots came exactly with 07.07. Nothing like that before. And, I also have similar second device, rutx12, with 2 mobiles and no WAN, so mobile is the only internet source there. And there I also got sporadic reboots after upgrade to 07.07! I can’t provide you with the troubleshoot from the second device right away, since it’s located remotely from my current location. But, I get troubleshoot file from that router a bit later. Anyway, I see the same on 2 rutx12 devices starting with 07.07 firmware. I can’t recall anything like that before upgrade to 07.07. What else… Both rutx12 routers have latest modem firmware as well, I upgraded it to latest on both modems in each of 2 devices. Both have 2 sim cards. But one has also WAN as a primary connection. Something like that…

Hello Dshoot,

In such case I would offer then turning off the Rebind protection and see if this resolves the issue, as at the moment I see most of the issues generated are from rebind attacks


Keep in mind from the security side it is not the best practice to turn it off, So maybe it would be better to remember to turn off the browser tabs once the VPN is off.

It would be great if there would be a possibility to get a troubleshoot file where the unexpected reboot happened, as by the provided file I can not find unexpected reboot and what could have caused it.

Best regards,
DziugasS