RUT240/RUT241 Random Delay for initiating Ping/Wget Auto-Reboot

Hello Teltonika Community,

We are working in a Project where we will have a big number of remote sites connected to a central server using 3G/4G connectivity and Wireguard VPN.

To avoid situation where the Cellular Data connection is lost and not restored until manual intervention, we are planning to activate the Auto-Reboot option, and we have tested it successfully in our initial tests. We have configured the Ping/Wget Reboot settings with 3 different tasks:

  1. One Ping is sent every 2 minutes to a well-known public IP address (8.8.8.8) to check Internet connectivity. After several failures ,the router would reset the Cellular Data Session

  2. A second task is created to ping the internal server located in the Headquarter every 15 minutes. This is an internal private IP address that would be reachable only if Wireguard VPN works correctly. After 4 consecutive falures, the router will do a modem reset.

  3. A third task to ping the internal server every hour (again checking full connectivity with the VPN). 4 failures will cause a full reboot of the router.

We are happy with the results. However, observing the behavior of the routers and how the Ping/Wget Reboot tasks are executed, we have the following concern.

It looks like those tasks are executed at the second 0 of every minute. I mean, checking the system logs, the first task scheduled every 2 minutes at 09 h 00 min 00 sec, next time at 09:02:00, next at 09:04:00, etc.

This is not a problem for one router. But if we consider the full system, where will would have potentially several hundreds or thousands of Remote Routers, and all of them synchronized via NTP, we could face the situation that all of them would try to send the Ping Test packets to the Internal Server in the Headquarter exactly at the same time.

And that would create an abrupt increase of incoming traffic in the server, and unintentionally we could be creating a Denial of Service Attack by ourselves, overloading the central server with a lot of ICMP Echo Requests (pings) at the same time.

So, it would be very useful to be able to apply a random delay to the startup of those Auto-reboot Ping tasks, so that the ping requests of all remote routers will be distributed in some interval, and would not arrive all at the same time to the Internal Server.

I noticed that after configuring the Ping/Wget Auto-reboot settings in the Web GUI, several lines are added to the file /etc/crontabs/root inside the router file system.

*/2 * * * * /usr/sbin/ping_reboot.sh cfg01c21d
*/5 * * * * /usr/sbin/ping_reboot.sh cfg02c21d
0 */1 * * * /usr/sbin/ping_reboot.sh cfg03c21d

So, I guess that on the lower level, the RUT240/RUT241 routers are using cron in the linux kernel to schedule the auto-reboot tasks.

My question is, would it be possible to edit this /etc/crontab/root file and include some modifications to add a random delay to the start of each task?

Searching on other development forums about cron and crontab configuration, I’ve read that it might be possible to delay the start of the script to be executed using the sleep command, followed by a random number of seconds that we want to delay.
For instance, for the first task to send one ping every 2 minutes, we could rewrite the entry in crontab like this, to sleep first a random number of seconds between 0 and 60.

*/2 * * * * sleep $(( RANDOM % 60 )); /usr/sbin/ping_reboot.sh cfg01c21d

Would that be supported by the cron implemented inside the firmware of RUT240/RUT241 routers? Or that would create a syntax error?

If this is not supported directly in the crontab file, maybe another alternative would be creating a modified script that will first do the random sleep, and then call the original ping_reboot.sh script.
Example in root crontab file

*/2 * * * * /usr/sbin/ping_reboot_v2.sh cfg01c21d

And the ping_reboot_v2.sh script would be manually created like this:

#!/bin/sh
sleep $(( RANDOM % 60 ))
/usr/sbin/ping_reboot.sh %1

Would that work? Is that feasible in the RUT240/RUT241 devices?

Thank you in advance for your help.

Hello,

Teltonika Networks routers use RutOS. RutOS is built upon OpenWRT, and OpenWRT itself is derived from Linux. Hence, the crontab functionality in RutOS is similar with what you’d expect from a Linux system.

If you wish to implement your own tweaks, you can do so. For instance, you have the option to manually adjust the ping reboot script located at /usr/sbin/ping_reboot.sh.

However, please keep in mind that the RANDOM function is a feature of the Bash shell. If you intend to use this function, you can install bash using the opkg package manager with the following commands:

opkg update 
opkg install bash

Once Bash is installed, you can use the RANDOM function in your crontab entries. For example (logger just to check the time, you can use ping_reboot):

* * * * * /bin/bash -c 'sleep $((RANDOM % 30)); logger "random time delay $(date +\"%T\")"' 

Or you can write an alternative in shell to skip Bash installation and save space on the device.

Kind Regards,

Thank you for your reply, AndzejJ. We will try the different options suggested.

This topic was automatically closed after 15 days. New replies are no longer allowed.