Hi all,
I’m looking at an issue seen in combination with multiple RUT956 devices and I’m wondering whether this has been observed before and whether it might have been solved at some point.
In multiple situations on multiple devices, the ethernet switch stops working properly. This is observed with firmware version 7.04.5 and 7.07.3. When this happens, in dmesg there are continuous messages like this:
[49977.512613] rt3050-esw 10110000.esw: link changed 0x00
[49980.244647] rt3050-esw 10110000.esw: link changed 0x01
[49998.956109] rt3050-esw 10110000.esw: link changed 0x00
[50001.689003] rt3050-esw 10110000.esw: link changed 0x01
[50020.397610] rt3050-esw 10110000.esw: link changed 0x00
[50023.146422] rt3050-esw 10110000.esw: link changed 0x01
[50041.841191] rt3050-esw 10110000.esw: link changed 0x00
[50044.788665] rt3050-esw 10110000.esw: link changed 0x01
And ip neigh shows INCOMPLETE for the neighbor (arp showing a MAC address of all zeros).
I tried:
Calling soft_reset through swconfig
Regular reset through swconfig
Bringing down eth0 and then up again
None of these have any effect.
Has this been observed before? Is it likely a newer firmware would solve it? I looked at the change history of the driver and did not see anything notable:
Interestingly an upstream DSA driver recently appeared in mainline:
Many thanks in advance for any suggestion on how this situation could be prevented!
Many thanks for the fast response. I updated one of the devices to 7.24.3. Unfortunately the problem still persists.
[ 284.187948] rt3050-esw 10110000.esw: link changed 0x00
[ 284.193298] br-lan: port 1(eth0.1) entered disabled state
[ 287.307948] rt3050-esw 10110000.esw: link changed 0x01
[ 287.313275] br-lan: port 1(eth0.1) entered blocking state
[ 287.318832] br-lan: port 1(eth0.1) entered forwarding state
[ 306.027917] rt3050-esw 10110000.esw: link changed 0x00
[ 306.033268] br-lan: port 1(eth0.1) entered disabled state
[ 309.147974] rt3050-esw 10110000.esw: link changed 0x01
[ 309.153327] br-lan: port 1(eth0.1) entered blocking state
[ 309.158892] br-lan: port 1(eth0.1) entered forwarding state
[ 327.867898] rt3050-esw 10110000.esw: link changed 0x00
[ 327.875677] br-lan: port 1(eth0.1) entered disabled state
[ 330.987898] rt3050-esw 10110000.esw: link changed 0x01
[ 330.993308] br-lan: port 1(eth0.1) entered blocking state
[ 330.998875] br-lan: port 1(eth0.1) entered forwarding state
[ 349.713686] rt3050-esw 10110000.esw: link changed 0x00
[ 349.719089] br-lan: port 1(eth0.1) entered disabled state
[ 352.827871] rt3050-esw 10110000.esw: link changed 0x01
[ 352.833227] br-lan: port 1(eth0.1) entered blocking state
[ 352.838789] br-lan: port 1(eth0.1) entered forwarding state
[ 390.267866] rt3050-esw 10110000.esw: link changed 0x00
[ 390.273252] br-lan: port 1(eth0.1) entered disabled state
[ 393.387836] rt3050-esw 10110000.esw: link changed 0x01
[ 393.397259] br-lan: port 1(eth0.1) entered blocking state
[ 393.402866] br-lan: port 1(eth0.1) entered forwarding state
[ 412.107869] rt3050-esw 10110000.esw: link changed 0x00
[ 412.113209] br-lan: port 1(eth0.1) entered disabled state
[ 415.227837] rt3050-esw 10110000.esw: link changed 0x01
[ 415.235518] br-lan: port 1(eth0.1) entered blocking state
[ 415.241146] br-lan: port 1(eth0.1) entered forwarding state
[ 433.951196] rt3050-esw 10110000.esw: link changed 0x00
[ 433.956636] br-lan: port 1(eth0.1) entered disabled state
[ 437.067918] rt3050-esw 10110000.esw: link changed 0x01
[ 437.073244] br-lan: port 1(eth0.1) entered blocking state
[ 437.078805] br-lan: port 1(eth0.1) entered forwarding state
[ 455.788014] rt3050-esw 10110000.esw: link changed 0x00
[ 455.793402] br-lan: port 1(eth0.1) entered disabled state
[ 458.908149] rt3050-esw 10110000.esw: link changed 0x01
[ 458.913518] br-lan: port 1(eth0.1) entered blocking state
[ 458.919128] br-lan: port 1(eth0.1) entered forwarding state
[ 477.627850] rt3050-esw 10110000.esw: link changed 0x00
[ 477.633905] br-lan: port 1(eth0.1) entered disabled state
[ 480.748042] rt3050-esw 10110000.esw: link changed 0x01
[ 480.753446] br-lan: port 1(eth0.1) entered blocking state
[ 480.759008] br-lan: port 1(eth0.1) entered forwarding state
[ 499.467848] rt3050-esw 10110000.esw: link changed 0x00
[ 499.473193] br-lan: port 1(eth0.1) entered disabled state
[ 502.587840] rt3050-esw 10110000.esw: link changed 0x01
[ 502.593168] br-lan: port 1(eth0.1) entered blocking state
[ 502.598726] br-lan: port 1(eth0.1) entered forwarding state
[ 521.307846] rt3050-esw 10110000.esw: link changed 0x00
[ 521.313193] br-lan: port 1(eth0.1) entered disabled state
[ 524.427828] rt3050-esw 10110000.esw: link changed 0x01
[ 524.433185] br-lan: port 1(eth0.1) entered blocking state
[ 524.438738] br-lan: port 1(eth0.1) entered forwarding state
[ 561.867923] rt3050-esw 10110000.esw: link changed 0x00
[ 561.874443] br-lan: port 1(eth0.1) entered disabled state
[ 564.987848] rt3050-esw 10110000.esw: link changed 0x01
[ 564.993206] br-lan: port 1(eth0.1) entered blocking state
[ 564.998767] br-lan: port 1(eth0.1) entered forwarding state
[ 583.709215] rt3050-esw 10110000.esw: link changed 0x00
[ 583.714565] br-lan: port 1(eth0.1) entered disabled state
[ 586.827909] rt3050-esw 10110000.esw: link changed 0x01
[ 586.833318] br-lan: port 1(eth0.1) entered blocking state
[ 586.838885] br-lan: port 1(eth0.1) entered forwarding state
[ 605.547956] rt3050-esw 10110000.esw: link changed 0x00
[ 605.553302] br-lan: port 1(eth0.1) entered disabled state
[ 608.668248] rt3050-esw 10110000.esw: link changed 0x01
[ 608.674275] br-lan: port 1(eth0.1) entered blocking state
[ 608.679921] br-lan: port 1(eth0.1) entered forwarding state
[ 627.387845] rt3050-esw 10110000.esw: link changed 0x00
[ 627.393191] br-lan: port 1(eth0.1) entered disabled state
[ 630.507829] rt3050-esw 10110000.esw: link changed 0x01
[ 630.513148] br-lan: port 1(eth0.1) entered blocking state
[ 630.518707] br-lan: port 1(eth0.1) entered forwarding state
[ 649.227849] rt3050-esw 10110000.esw: link changed 0x00
[ 649.233194] br-lan: port 1(eth0.1) entered disabled state
[ 652.347833] rt3050-esw 10110000.esw: link changed 0x01
[ 652.353155] br-lan: port 1(eth0.1) entered blocking state
[ 652.358726] br-lan: port 1(eth0.1) entered forwarding state
[ 671.067869] rt3050-esw 10110000.esw: link changed 0x00
[ 671.073306] br-lan: port 1(eth0.1) entered disabled state
[ 674.187893] rt3050-esw 10110000.esw: link changed 0x01
[ 674.193275] br-lan: port 1(eth0.1) entered blocking state
[ 674.198836] br-lan: port 1(eth0.1) entered forwarding state
Any other information I could provide to diagnose this further?
Based on the logs, the continuous Ethernet link-state flapping appears to be causing the Ethernet ports to repeatedly disconnect and reconnect, or in some cases, stop forwarding traffic altogether.
This behavior could potentially be related to an auto-negotiation or duplex mismatch. Such issues can sometimes occur when connecting the device to specific industrial switches or older 10/100 Mbps devices.
For troubleshooting purposes, could you please clarify the Ethernet port settings currently configured on the RUT956?
Specifically:
Is Auto Negotiation enabled or disabled?
If Auto Negotiation is disabled, what link speed is configured for the ports, and is Duplex set to Full or Half?
If Auto Negotiation is enabled, what advertisement options are configured for each port?
These settings can be found in the device WebUI under Network → Ports → Port Settings.
Additionally, could you please confirm whether the Keep Settings option was enabled or disabled when upgrading the RUT956 firmware from 00.07.07.3 to 00.07.24.3?
There were some cases where performing a significant firmware version jump while keeping the existing configuration could potentially lead to unexpected behavior, so this information may help us narrow down the cause.
We hope this helps. Please let us know how the device and Ethernet ports are currently configured.
Auto negotiation is enabled. Link speed is 100 MBit/s. The advertisement options are not different from the usual ones. (All speeds are advertised.)
Ah this is very good to know! I tried seeing if I could change the settings using swconfig, but that did not work. Right now the problem was addressed (there was a power cycle of all equipment), but I’ll try disabling autonegotiation and fixing the speed when I see this again.
Yes, settings were kept. Otherwise I would not have been able to reach the device anymore.
I will look into fixing the speed through the UI when I see this again! Many thanks!!
This problem just got triggered again on yet another RUT956 device. I found that the option to change the network port setting was missing in the UI of the older firmware, so I upgraded this device to the latest firmware as well.
Seeing the problem still occurring with the new firmware, I then disabled auto negotiation and fixing the port settings to all possible values (starting with 100 Mbit/s full duplex, which is what is used when things work correctly).
Unfortunately no matter which set of settings I tried, the port continued to go up & down and communication was not possible.