Recent SFP change?

BotBlox Hardware
1

We recently received some UbiSwitches and a large quantity of SFP-10G-T-B. We have a setup with laptop 1Gb port to SFP-10G-T-B in Port 9 and SFP-10G-T-B in Port 0 to a 10Gb port on Orin/Rogue combination. We have used this setup successfully for the past 9 months across many units without the need to configure the ports on the Ubi. During a build this week using the latest hardware, we are not able to establish a connection between the laptop and the Orin.

Troubleshooting has included checking with other Ubiswitches (including RevA), using different brand SFPs and using the oldest (lowest SN) SFP-10G-T-B we have on hand.

Set-up 1 using SPF-10G-T-B serial numbers 202509170098 and 202509170099: no connection regardless of which port a given SFP is in or what (laptop vs Orin) is plugged into a given SFP.

Set-up 1b, laptop plugged into any 1Gb port on the Ubi and SFP-10G-T-B in any of ports 0,9,10 has connectivity.

Set-up 2 10Gtek ASF-10G-T SFPs: solid connection regardless of which port a given SFP is in or what (laptop vs Orin) is plugged into a given SFP.

Set-up 3 10Gtek ASF-10G-T SFP and SPF-10G-T-B serial numbers 202509170006 solid connection regardless of which port a given SFP is in or what (laptop vs Orin) is plugged into a given SFP.

Set-up 4 10Gtek ASF-10G-T SFP and SPF-10G-T-B serial numbers 202509170011 solid connection regardless of which port a given SFP is in or what (laptop vs Orin) is plugged into a given SFP.

Set-up 5 using SPF-10G-T-B serial numbers 202509170006 and 202509170011: no connection regardless of which port a given SFP is in or what (laptop vs Orin) is plugged into a given SFP.

Set-up 6 using SPF-10G-T-B serial numbers 202509170098 or 202509170099 and 10Gtek ASF-10G-T: occasional connection without repeatability.

Are there any known changes to SFP-10G-T-B that could have occurred after an order placed Aug 2025 (we previously ordered PN SFP-10G-T, no “B”) until now that could be explain why we can’t get links with the default settings anymore? Or are there any changes to Ubi Firmware or default settings? (Ubis on hand report V1.0.0+, unknown FW on older boards)

2

Hi Keith,

The hardware inside did change, we switched to using a lower power Marvell 10G PHY inside there.

The issue here is pointing to an inability of the new 10G SFP to autonegotiate down to 1G, rather than the issue being with the switch, or with some interaction of both SFPs together on the switch.

Can you run a quick test in your setup using two 10G Orins on each SFP plugged into the UbiSwitch?

3

We will also run some tests on our end to see if this is indeed a failure of the SFP to run at 1G. If that is the case, then we’ll try changing the SFP interface on UbiSwitch from 10gbase-r to 1000base-x or sgmii. If that fails then this might be an undocumented limitation on the new 10G SFPs.

The workaround there will be to use a 1G SFP instead which is annoying! If this ends up being the case, we’ll approve an RMA and refund on the 10G SFPs you can’t use, as this was not properly documented on our side.

We’ll run some tests tomorrow.

4

I didn’t have a second ORIN available, but I have a QNAP QNA-T310G1T (10Gb capable) USB adapter. The QNAP can be set at: Auto-negotiate, 10Gb, 5GB, 2.5Gb, 1Gb, 100M. With this setup connected with the UBI and Qty( 2) SPF-10G-T-B in the middle, comms to the Orin were achieved quickly with Auto-negotiate,10Gb, 5GB, 2.5Gb settings on the QNAP. !Gb would not connect, and I didn’t attempt 100M. Also confirming that another instance of Ubi with some 5Gb devices also communicates within out setup, so it does look like an issue with SFP auto negotiating 1GB link speed.

5

You’re fast.

Can you try setting the port that the BotBlox 10G SFP is plugged into, to run in 1000BASE-X and SGMII modes.

[port <int>] mac [mode {"1000basex", "sgmii", "2500basex", "5gbaser", "10gbaser", "usxgmii"}]

So if you were using port 0, you’d type

port 0 mac mode 1000basex

Then test it. Then if no good try

port 0 mac mode sgmii

Then test it

6

1000basex worked when the QNAP was set at 1GB, but at no other speed, even wouldn’t connect with QNAP set to Auto-negotiate. sgmii would not connect at 1Gb. The 1000basex may work as a near term work-around, but we anticipate end users to potentially want link speeds higher than 1Gb for data transfer purposes. are changes to mac settings written to non-volitile memory by default, or do these need saved to persist through a power cycle?

7

Sorry just to clarify, 1000base-x worked when using the SFP-10G-T-B plugged into UbiSwitch, with another SFP-10G-T-B connected to the Orin?

You’re saying that you were able to get traffic flow between the Orin and the QNAP when the UbiSwitch SFP port connected to the QNAP was set to 1000base-x?

8

To save configurations in BloxOSLite to memory (non-volatile), you can use the conf save commands.

9

Correct, the SFP-10G-T-B set to 1000base-X going to QNAP and another SFP-10G-T-B set to default/10Gbase-R to the Orin. Correct, but only when the QNAP was set to 1Gb. No other speed setting on the QNAP would communicate with the 1000base-x.

10

Ok. Let me explain what’s happening.

First thing to understand is than an SFP has essentially two interfaces. The “media” interface (the side that connects to a fiber or ethernet cable) and the “backplane” interface (the card edge that plugs into an SFP cage.

These two interfaces are different and can even run at different speeds.

The old SFP design
When the media side was connected to a 1G device, the media side would autonegotiate down to 1G, as you’d expect it to. The interface side, however, would stay at 10G, or more specifically, 10GBASE-R.

This works because the chip in the SFP was “ferrying” data from the 1G media interface into the 10G backplane interface. It usually does this by interspersing the data; framing the 10G side with 1G packets. UbiSwitch by default sets its SFP ports to 10GBASE-R, so this all works fine.

The new SFP design
The new SFP design changes its interface port based on the media side. In this case, when a 1G port is connected to the media interface, the chip inside changes its backplane interface down to 1000BASE-X. In this case the speed of the underlying data matches the underly physical layer speed.

This doesn’t work with UbiSwitch however, because UbiSwitch expects 10GBASE-R on the SFP port. That’s why setting the UbiSwitch SFP port to 1000BASE-X solves it.

What the hell is going on?
Sadly, there isn’t really a standard every SFP agrees on, on how to handle mismatched media and backplane speeds, and there isn’t an autonegotiation mechanism. Neither approach is technically incorrect, but this causes a lot of integration hell. And sadly no one seems to document this.

Anyway it looks like you’re going to have to manually set the port to 1000BASE-X on UbiSwithc to use the new SFP modules. Does that work for your application?