– Rephrased from customer requests –
I have an two UbiSwitches connected as following.
The two Pis cannot ping each other? What’s the problem.
I’m going to use this as an opportunity to test and log multiple different combinations of SFP modules, as a matter of documentation. My setup looks like below.
---------------------------------------- Test 1: ----------------------------------------
SFP A = SFP-10G-SR
SFP B = SFP-10G-SR
Fiber: 1 meter OM4 Fiber
UbiSwitch A configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
UbiSwitch B configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
Ping test between Pi A and Pi B: Works
iPerf test: ~940Mbps
---------------------------------------- Test 2: ----------------------------------------
SFP A = SFP-10G-T (10GBASE-T RJ-45 SFP)
SFP B = SFP-10G-T (10GBASE-T RJ-45 SFP)
Cable: 1 foot CAT5e cable
UbiSwitch A configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
UbiSwitch B configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
Ping test between Pi A and Pi B: Works
iPerf test: ~940Mbps
---------------------------------------- Test 3: ----------------------------------------
SFP A = 10Gtek ASF-GE2-T (10/100/1000BASE-T SFP SGMII)
SFP B = 10Gtek ASF-GE2-T (10/100/1000BASE-T SFP SGMII)
Cable: 1 foot CAT5e cable
UbiSwitch A configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
UbiSwitch B configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
Ping test between Pi A and Pi B: Not able to ping
iPerf test: Failed
UbiSwitch A configuration: Port 9 configured in SGMII mode
port 9 mac mode sgmii
UbiSwitch B configuration: Port 9 configured in SGMII mode
port 9 mac mode sgmii
Ping test between Pi A and Pi B: Works
iPerf test: ~940Mbps
Remember to use port save to retain this sgmii setting on both UbiSwitches on power on. Otherwise, after a power cycle, UbiSwitch will go back into 10gbase-r modes on both SFP ports.
---------------------------------------- Test 4: ----------------------------------------
SFP A = 1000BASE-T RJ45 SFP Module (TL-SM331T)
SFP B = 1000BASE-T RJ45 SFP Module (TL-SM331T)
Cable: 1 foot CAT5e cable
UbiSwitch A configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
UbiSwitch B configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
Ping test between Pi A and Pi B: Not able to ping
iPerf test: Failed
UbiSwitch A configuration: Port 9 configured in 1000BASE-X mode
port 9 mac mode 1000basex
UbiSwitch B configuration: Port 9 configured in 1000BASE-X mode
port 9 mac mode 1000basex
Ping test between Pi A and Pi B: Works
iPerf test: ~940Mbps
---------------------------------------- Test 5: ----------------------------------------
SFP A = CWDM-SFP1G-ZX
SFP B = CWDM-SFP1G-ZX
Fiber: Fiber: 1 foot OS2 Fiber
UbiSwitch A configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
UbiSwitch B configuration: Out the box, default configuration (SFPs in 10GBASE-R mode)
Ping test between Pi A and Pi B: Not able to ping
iPerf test: Failed
UbiSwitch A configuration: Port 9 configured in 1000BASE-X mode
port 9 mac mode 1000basex
UbiSwitch B configuration: Port 9 configured in 1000BASE-X mode
port 9 mac mode 1000basex
Ping test between Pi A and Pi B: Works
iPerf test: ~940Mbps
Note: We have identified a bug when working with
SFP A = CWDM-SFP1G-ZX
SFP B = CWDM-SFP1G-ZX
Fiber: Fiber: 1 foot OS2 Fiber
Steps to reproduce:
-
Connect two UbiSwitch together in the manner described at the top of this forum post
-
Set port 9 on both UbiSwitch to 1000BASE-X
port 9 mac mode 1000basex
Note that neither of the LEDs for port 9 come on.
- Set port 9 on one UbiSwitch (Doesn’t matter which one) to SGMII
port 9 mac mode sgmii
Note how the red LEDs for port 9 illuminate on both UbiSwitch, implying that a link has been formed.
Despite this red LED, the raspberry pis cannot ping each other. Port 1 LED, Port 9 LED on the UbiSwitch configured to SGMII blinks together with the port 9 LED on the UbiSwitch configured to 1000BASE-X. The port 1 LED on UbiSwitch configured to 1000BASE-X does not blink in sync with the other LEDs.
- Now set port 9 on that same UbiSwitch (The one you set to SGMII) back to 1000BASE-X.
port 9 mac mode 1000basex
Now all LEDs blink together and the link can see each other.
Unfortunately, since 1000basex setting does not seem to work (without this weird sgmii workaround), it means there is no way to save the config the boards to 1000basex using port save.
This will need an update to solve this bug, there doesn’t appear to be any workaround that the customer can use to implement a non-volatile 1000base-x connection between two UbiSwitches.
Interestingly, this issue doesn’t occur when using 1000BASE-X to 1GBASE-T SFP modules. 1000BASE-T RJ45 SFP Module (TL-SM331T)
Talking specifically about the type of SFP that only supports 1GBASE-T (these require 1000BASE-X). These appear to work perfectly fine with UbiSwitch.