Traffic to remote network 10.50.0.0/16 is sent via a single IPSEC VPN Tunnel to a Fortigate Firewall FG90 on a central Site. The rest of Internet traffic is routed a usual
Local Subnet: 10.10.92.0/24
Remote Subnet: 10.50.0.0/16
IPSEC Tunnels establishes correctly and traffic sent to that 10.50.0.0/16 works well
WAN Failover works as intended for normal Internet Access. WAN has higher priority than SIM1. Health check is performed via ping to 1.1.1.1 and 8.8.8.8 on both interfaces
The problem occurs for the IPSEC Tunnel when Wan Failover happens.
When failovers happens over a physical disconnection of the WAN (cable unplugged or if down cli command), the IPSEC Tunnel reestablishes via SIM1. As soon as I reconnect the WAN cable, IPSEC tunnel also reestablishes over the WAN connection. So far so good.
The problem occurs when I simulate a “soft” loss of internet access via WAN (I disconnect an upstream cable from the WAN internet service provider cable). In this case, the WAN interface on the RUT is still up and running, but is marked as Offline by the failover mechanism
In this case, the IPSEC persists in (apparently) sending packets through the WAN interface and no failover occurs. In the IPSEC logs you can see that packets are still being marked as sent from the WAN IP Address.
I have tried configuring DPD restarts, configured different timers and timeouts, and tried every option here (also on the FG end) Also tried checking and unchecking Flushconn track option and also marked and unmarked all the 4 possible combinations of Flush Connections in both WAN and SIM1 interfaces.
The IPSEC Tunnel is a Dial UP. The remote FG90 has Static Public IP.
The failover from SIM1 to WAN works well with IPSEC. I this case, the SIM1 interface goes from Online Status to Standby. In the problematic scenario (“soft” loss of internet access on the WAN), the WAN interface is marked as Offline by the Failover mechanism (not as Stanby, for obvious reasons). If i reestablish the WAN Internet Access, the IPSEC Tunnel starts working again.
Is what i’m trying to do really achievable? Has anyone experienced a similar problem?
Apologies for a delayed response and thank you for the detailed information and for performing the additional tests.
Based on your observations, the failover is triggered when the Ethernet link is physically lost (for example, when the cable is unplugged), but not when the WAN interface is manually disabled by setting Enable → Off.
From our understanding, this behavior is expected. The failover mechanism is designed to monitor active WAN interfaces and react to link loss and/or connectivity check failures. Manually disabling an interface removes it from operation and is not documented as a failover trigger.
Regarding the IPsec issue described in your original post, this appears to be a different scenario. In your case, the WAN interface remains active and the failover correctly switches general traffic to the LTE connection after connectivity checks fail, while only the IPsec tunnel does not re-establish over the backup WAN.
To investigate this further, could you please provide the following information?
The exact RutOS firmware version currently running on the RUT951.
Did you notice any IPsec log entries that might provide additional information?
The output of the following commands during the issue:
ip route
ip rule
ip route get <FortiGate_public_IP>
(where <FortiGate_public_IP> is the public IP address of the remote IPsec peer.)
NOTE: When posting the outputs, please remove the public IP address, since this is a private information, and we don’t want it to end up in the public forum.
This information will help determine whether the issue is related to routing, IPsec tunnel re-establishment, or another component of the failover process.
Thank you for your answer. Last week we got support fron local Teltonika engineers who got hands on the problem for a few hours. Finally they discovered what the problem was. It turned out that on a WAN Internet unavailabilty scenerio (what i call a “soft” outage), the IPSEC persisted on the WAN interface due to its better metric in the failover scheme (it is preferred). This is clearly a bug. So, they kindly provided a script that monitors the WAN interface using ping checks. When this check fails, it changes the WAN Metric to low value (5) to ensure that the SIM1 is preferred. When the WAN connections reestablishes, it sets the WAN metric again to 1.
And…it worked. The whole process lasts about 1 minute because the health check does 4 iterations in 12 seconds intervals. We can fine tune it to reduce this downtime.
In the meantime, they scalated the case to HQ to get a fix for this bug.
The script is reproduced below:
#!/bin/sh
path: /usr/local/home/root/scripts/
DEV=eth0.2
METRIC_UP=1
METRIC_DOWN=5
CHECK_INTERVAL=12 # in seconds
CYCLES=4 # 4 cycles of 12s = 48s
Check WAN current metric
CURRENT_STATE=$(uci get network.wan.metric 2>/dev/null)
Check if the physical WAN eth interface is UP
check_wan_phy_status() {
ip link show $DEV | grep -q “state UP”
return $?
}
set_metric() {
local metric=$1
logger “[wan-monitor] Changing metric of eth WAN to: $metric”
uci set network.wan.metric=“$metric”
uci commit network
reload_config
CURRENT_STATE=“$metric”
}
for i in $(seq 1 $CYCLES); do
if check_wan_phy_status; then
status=$(check_wan_status)
case "$status" in
online)
if [ "$CURRENT_STATE" != "$METRIC_UP" ]; then
set_metric "$METRIC_UP"
fi
;;
offline)
if [ "$CURRENT_STATE" != "$METRIC_DOWN" ]; then
set_metric "$METRIC_DOWN"
fi
;;
*)
logger "[wan-monitor] WAN status in transition"
;;
esac
else
logger "[wan-monitor] Physical interface is not UP"
fi
sleep "$CHECK_INTERVAL"