OTD500 (RutOS 07.24.5): GRO on qmimux0 corrupts forwarded TCP/TLS streams in passthrough mode - reproducible workaround found

I have been (trying to) use a OTD500 for serval month as my primary internet connection for Unifi LAN setup a I only a very bandwidth poor copper wire DSL connection, but I can directly see a 5G tower. (5G give me around 400Mbps, while DSL gives me a maximum of 80Mbps.)

With the last few firmware upgrades the connection became stable and very reliable for most useless but big downloads (mainly iOS or macOS updates) are never able to download and I have switch back to the DSL connection. Directly downloading the iOS updates to my phone via 5G with the same SIM provider works fine.

Unable to pinpoint the issue manually a friend suggested giving Claude access to my network and the OTD and see what it can find out.

While I can not be sure that it found the culprit, the solution as suggested and implemented works.
Stable downloads, but limited to around 90Mbps.

Maybe someone from Teltonika can take a look.
Happy to send more details/files.

Thanks!

SUMMARY

-------

Forwarded TCP traffic received from the mobile network is corrupted when GRO is enabled on the modem’s qmimux0 interface. Large HTTPS downloads from a client behind the OTD500 fail with TLS errors (“decryption failed or bad record mac”) after a few MB to a few hundred MB. Disabling GRO on qmimux0 alone (“ethtool -K qmimux0 gro off”) fixes it completely. Traffic generated on the OTD500 itself is not affected. The corruption passes the TCP checksum, so plain HTTP transfers appear to complete but are probably damaged silently.

The practical impact: large downloads (e.g. multi-GB iOS updates) over this connection never finish.

DEVICE / ENVIRONMENT

--------------------

- Device: Teltonika OTD500, serial, MAC/IMEI available on request

- Firmware: OTD5_R_00.07.24.5 (upgraded on 2026-10-02 from 07.24.3). Kernel 6.6.126 (build date 2026-09-14), SoC MediaTek MT7621.

- Earlier firmware on this unit: 07.22.3 → 07.23.7 (2026-07-31) → 07.24.3 (2026-09-14) → 07.24.5. All tests below were run on 07.24.5 only. The download problem existed long before, but I cannot say on which firmware versions.

- Modem: Quectel RG520N-EBDC, 5G-NSA, operator 26201 (Deutsche Telekom, Germany)

- Mobile config (/etc/config/network, interface mob1s1a1): proto wwan, method ‘passthrough’, pdptype ‘ip’ (IPv4 only), force_link ‘1’, auto_apn ‘1’

- Firewall defaults: flow_offloading ‘1’, flow_offloading_hw ‘1’ (the original setting)

- Topology: OTD500 LAN1 → Ethernet (1 Gbit) → UniFi Cloud Gateway Ultra, WAN2 port (DHCP client). The gateway gets a private /29 from the OTD500 (the OTD500 itself holds a /32 on br-lan). The gateway also has a second, independent WAN (DSL/PPPoE).

- Signal at the time: RSRP -87 dBm, SINR 12 dB, RSRQ -8 dB. No Ethernet errors on either side (rx_errors/FCS/CRC counters all 0).

HOW CLAUDE TESTED

------------

The tests ran on the UniFi gateway (a Linux client behind the OTD500). curl was bound to the interface connected to the OTD500:

curl -sS -o /dev/null --interface eth3 --max-time 25 https://proof.ovh.net/files/1Gb.dat

The file is 1 GiB. The only error seen was:

curl: (56) OpenSSL SSL_read: error:1408F119:SSL routines:ssl3_get_record:decryption failed or bad record mac

RESULTS

-------

A) Baseline (default offload settings on the OTD500)

  • 9 of 12 valid runs failed with the TLS error above. The other 3 ran to my time limit without error. (One connect timeout was excluded.)

  • Failures occurred at random points, between 1 MB and about 360 MB (e.g. 0.7 MB, 12.5 MB, 35 MB, 50 MB, 119 MB, 145 MB, 164 MB, 285 MB, 359 MB).

  • The failure also occurred when the client was rate-limited to 32 Mbit/s, so it is not purely a high-speed effect.

  • A plain-HTTP download of 512 MB over the same path completed without any error. That is not evidence of integrity, because plain HTTP cannot detect corruption.

B) Controls

  • The gateway’s other WAN (independent path): 296 MB of the same TLS file in 35 s, no errors.

  • TLS download run directly on the OTD500 (curl 8.19 on the device, over the mobile link): 3 runs of 25 s (about 105 MB each at about 4 MB/s), no errors. This suggests the mobile network itself is not corrupting data, and the problem lies in the forwarding path (qmimux0 → br-lan → lan1). The runs are short and slow, so this is good evidence, not proof.

C) Offload/feature bisection on the OTD500 (runtime changes only, via ethtool)

Original feature state:

 lan1:    sg on, tso on, gso on, gro on, rx/tx checksumming on

 br-lan:  sg on, tso on, gso on, gro on

 qmimux0: sg on, tso off, gso on, gro on

 wwan0:   sg off, tso off, gso off ("requested on"), gro on

| Change | Result |

|--------------------------------------------------------------|-------------------------------------|

| none (baseline) | 9 of 12 failed |

| flow_offloading_hw 1 → 0 (firewall reload) | 4 of 4 valid runs failed* |

| NIC offloads off on the client side (gateway eth3) | 3 of 3 valid runs failed* |

| tso/gso/gro/sg off on lan1, br-lan, qmimux0, wwan0 | 6 of 6 clean (about 280 MB/25 s) |

| GRO off only (on lan1, br-lan, qmimux0) | 5 of 5 clean |

| GRO off ONLY on qmimux0 | 5 of 5 clean |

| GRO off ONLY on lan1 + br-lan (qmimux0 GRO on) | 4 of 4 valid runs failed* |

| Final check: GRO off on qmimux0, flow_offloading_hw back to 1| 5 of 5 clean |

* In these series, some runs returned a 1606-byte response with exit 0 (probably a rate-limit response from the test server). I excluded them.

With the workaround, throughput is about 90 Mbit/s through the OTD500 (CPU-bound forwarding without GRO). The corrupted runs reached up to about 480 Mbit/s, but that data was damaged.

WORKAROUND

----------

ethtool -K qmimux0 gro off

Note that qmimux0 is recreated on every mobile reconnect or reboot, so the setting is lost each time and has to be re-applied, for example by a hotplug script on mobile interface ifup.

CLAUDE’S HYPOTHESIS (not verified, for your engineering team)

------------------------------------------------------

Download traffic is received on qmimux0, where GRO aggregates segments. The aggregated skbs are then forwarded through br-lan and re-segmented (GSO) by the mtk_soc_eth driver on lan1. Corruption of payload that still has a valid TCP checksum suggests a bug in GRO/GSO handling (possibly fraglist/frag handling) of skbs coming from the qmimux/QMI path on this kernel. Locally terminated traffic does not use this path, which would explain why direct tests on the device are unaffected. Disabling hardware flow offloading and the client NIC offloads did not change the result.

REQUESTS

--------

1. Please confirm whether this is a known issue and whether a fix is planned. Should GRO be disabled by default on qmimux interfaces, or is there a driver/kernel patch available?

2. A recommended, persistent configuration for the workaround that survives modem reconnects and firmware upgrades would be very helpful, ideally as a supported UCI option.

3. Please tell me if you need anything else: I can run further tests, for example a packet capture on qmimux0 and lan1 while reproducing, or test a firmware build with a fix.

Thanks for posting on the topic! I’ve been having issues with Home Assistant app downloads failing at random points with MAC errors. I’m using passthrough mode on a RUTM30 (RG520NEBDCR03A06M4G_A0.300.A0.300; RUTM_R_00.07.25.3).

I can confirm both the symptoms (also present in bridge mode) and the fix working:

➜ /tmp wget https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/16.04/release/lubuntu-16.04.6-desktop-amd64.iso--2026-10-05 19:48:26-- https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/16.04/release/lubuntu-16.04.6-desktop-amd64.isoResolving ftp.funet.fi (ftp.funet.fi)... 193.166.3.2, 2001:708:10:8::2Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:443... connected.HTTP request sent, awaiting response... 200 OKLength: 977272832 (932M) \[application/x-iso9660-image\]Saving to: 'lubuntu-16.04.6-desktop-amd64.iso'

lubuntu-16.04.6-desktop-amd64.iso 6%[====> ] 64.86M 12.3MB/s in 6.3s

2026-10-05 19:48:34 (10.3 MB/s) - Read error at byte 68007866/977272832 (error:0A000119:SSL routines::decryption failed or bad record mac; error:0A000139:SSL routines::record layer failure). Retrying.

--2026-10-05 19:48:35-- (try: 2) https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/16.04/release/lubuntu-16.04.6-desktop-amd64.iso
Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:443… connected.
HTTP request sent, awaiting response… 206 Partial Content
Length: 977272832 (932M), 909264966 (867M) remaining [application/x-iso9660-image]
Saving to: ‘lubuntu-16.04.6-desktop-amd64.iso’

lubuntu-16.04.6-desktop-amd64.iso 8%[+++++> ] 76.84M 9.27MB/s in 1.3s

2026-10-05 19:48:37 (9.27 MB/s) - Read error at byte 80575668/977272832 (error:0A000119:SSL routines::decryption failed or bad record mac; error:0A000139:SSL routines::record layer failure). Retrying.

--2026-10-05 19:48:39-- (try: 3) https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/16.04/release/lubuntu-16.04.6-desktop-amd64.iso
Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:443… connected.
HTTP request sent, awaiting response… 206 Partial Content
Length: 977272832 (932M), 896697164 (855M) remaining [application/x-iso9660-image]
Saving to: ‘lubuntu-16.04.6-desktop-amd64.iso’

lubuntu-16.04.6-desktop-amd64.iso 27%[++++++===============> ] 258.86M 31.1MB/s in 7.0s

2026-10-05 19:48:46 (25.9 MB/s) - Read error at byte 271431470/977272832 (error:0A000119:SSL routines::decryption failed or bad record mac; error:0A000139:SSL routines::record layer failure). Retrying.

--2026-10-05 19:48:49-- (try: 4) https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/16.04/release/lubuntu-16.04.6-desktop-amd64.iso
Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:443… connected.
HTTP request sent, awaiting response… 206 Partial Content
Length: 977272832 (932M), 705841362 (673M) remaining [application/x-iso9660-image]
Saving to: ‘lubuntu-16.04.6-desktop-amd64.iso’

lubuntu-16.04.6-desktop-amd64.iso 31%[++++++++++++++++++++++==> ] 292.75M 21.0MB/s ^C


root@RUTM30:~# ethtool -K qmimux0 gro off
root@RUTM30:~# exit

➜ /tmp wget https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/16.04/release/lubuntu-16.04.6-desktop-amd64.iso
–2026-10-05 19:56:22-- https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/16.04/release/lubuntu-16.04.6-desktop-amd64.iso
Resolving ftp.funet.fi (ftp.funet.fi)… 193.166.3.2, 2001:708:10:8::2
Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 977272832 (932M) [application/x-iso9660-image]
Saving to: ‘lubuntu-16.04.6-desktop-amd64.iso.1’

lubuntu-16.04.6-desktop-amd64.iso.1 100%[===============================================================================>] 932.00M 24.7MB/s in 36s

2026-10-05 19:56:58 (26.2 MB/s) - ‘lubuntu-16.04.6-desktop-amd64.iso.1’ saved [977272832/977272832]

➜ /tmp rm lubuntu-16.04.6-desktop-amd64.iso
➜ /tmp wget https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/26.10/release/snapshot-4/lubuntu-26.10-snapshot4-desktop-amd64.iso
–2026-10-05 19:57:31-- https://ftp.funet.fi/pub/unix/Linux/mirrors/lubuntu/releases/26.10/release/snapshot-4/lubuntu-26.10-snapshot4-desktop-amd64.iso
Resolving ftp.funet.fi (ftp.funet.fi)… 193.166.3.2, 2001:708:10:8::2
Connecting to ftp.funet.fi (ftp.funet.fi)|193.166.3.2|:443… connected.
HTTP request sent, awaiting response… 200 OK
Length: 4045824000 (3.8G) [application/x-iso9660-image]
Saving to: ‘lubuntu-26.10-snapshot4-desktop-amd64.iso’

lubuntu-26.10-snapshot4-desktop-amd64.i 100%[===============================================================================>] 3.77G 30.2MB/s in 2m 19s

2026-10-05 19:59:51 (27.7 MB/s) - ‘lubuntu-26.10-snapshot4-desktop-amd64.iso’ saved [4045824000/4045824000]

Update:
The 90Mbps limit was probably introduced due damaging an Ethernet cable while working on the router. After changing the cable I do get round 400Mbps after the fix, iOS downloads and fast and do not fail.

Hello @ideamechanic,

Thank you for the update. We are glad to hear that you were able to identify the cause of the issue. Please keep us informed if you experience any further issues.

@Bearnet, would you be able to reproduce the issue and generate a troubleshoot file immediately afterwards? This would allow our R&D department to analyse the behaviour in more detail.

If you are able to do so, please let me know, and I will contact you privately to collect the files.

Kind Regards,
T.

Hi @Tautvydas

thanks for the reply. I might have formulated my update unclear. To avoid any confusion:

The new ethernet cable only fixed initial reported bandwith limitation.

The gro = off fix for corruption is still needed and fixes my main issue.

Thanks

Hi @Tautvydas! Yes, I can. Even though the device is in a remote location, I have terminal access both to it and a host with a large enough disk for downloads using the RUTM30 for internet access.

At the same time, are you not able to replicate the issue in your own lab? The test is fairly trivial to duplicate.