Hello Teltonika Community,
We are working in a Project where we will have a big number of remote sites connected to a central server using 3G/4G connectivity and Wireguard VPN.
To avoid situation where the Cellular Data connection is lost and not restored until manual intervention, we are planning to activate the Auto-Reboot option, and we have tested it successfully in our initial tests. We have configured the Ping/Wget Reboot settings with 3 different tasks:
One Ping is sent every 2 minutes to a well-known public IP address (184.108.40.206) to check Internet connectivity. After several failures ,the router would reset the Cellular Data Session
A second task is created to ping the internal server located in the Headquarter every 15 minutes. This is an internal private IP address that would be reachable only if Wireguard VPN works correctly. After 4 consecutive falures, the router will do a modem reset.
A third task to ping the internal server every hour (again checking full connectivity with the VPN). 4 failures will cause a full reboot of the router.
We are happy with the results. However, observing the behavior of the routers and how the Ping/Wget Reboot tasks are executed, we have the following concern.
It looks like those tasks are executed at the second 0 of every minute. I mean, checking the system logs, the first task scheduled every 2 minutes at 09 h 00 min 00 sec, next time at 09:02:00, next at 09:04:00, etc.
This is not a problem for one router. But if we consider the full system, where will would have potentially several hundreds or thousands of Remote Routers, and all of them synchronized via NTP, we could face the situation that all of them would try to send the Ping Test packets to the Internal Server in the Headquarter exactly at the same time.
And that would create an abrupt increase of incoming traffic in the server, and unintentionally we could be creating a Denial of Service Attack by ourselves, overloading the central server with a lot of ICMP Echo Requests (pings) at the same time.
So, it would be very useful to be able to apply a random delay to the startup of those Auto-reboot Ping tasks, so that the ping requests of all remote routers will be distributed in some interval, and would not arrive all at the same time to the Internal Server.
I noticed that after configuring the Ping/Wget Auto-reboot settings in the Web GUI, several lines are added to the file /etc/crontabs/root inside the router file system.
*/2 * * * * /usr/sbin/ping_reboot.sh cfg01c21d
*/5 * * * * /usr/sbin/ping_reboot.sh cfg02c21d
0 */1 * * * /usr/sbin/ping_reboot.sh cfg03c21d
So, I guess that on the lower level, the RUT240/RUT241 routers are using cron in the linux kernel to schedule the auto-reboot tasks.
My question is, would it be possible to edit this /etc/crontab/root file and include some modifications to add a random delay to the start of each task?
Searching on other development forums about cron and crontab configuration, I’ve read that it might be possible to delay the start of the script to be executed using the sleep command, followed by a random number of seconds that we want to delay.
For instance, for the first task to send one ping every 2 minutes, we could rewrite the entry in crontab like this, to sleep first a random number of seconds between 0 and 60.
*/2 * * * * sleep $(( RANDOM % 60 )); /usr/sbin/ping_reboot.sh cfg01c21d
Would that be supported by the cron implemented inside the firmware of RUT240/RUT241 routers? Or that would create a syntax error?
If this is not supported directly in the crontab file, maybe another alternative would be creating a modified script that will first do the random sleep, and then call the original ping_reboot.sh script.
Example in root crontab file
*/2 * * * * /usr/sbin/ping_reboot_v2.sh cfg01c21d
And the ping_reboot_v2.sh script would be manually created like this:
sleep $(( RANDOM % 60 ))
Would that work? Is that feasible in the RUT240/RUT241 devices?
Thank you in advance for your help.