UbiSwitch is dropping packets when streaming two GigE cameras into the 10G SFP port

BotBlox Hardware
1

– Summarised from customer support ticket –

  • I’m using UbiSwitch and UbiSwitch Baseboard.
  • I have two GigE cameras connected to two 1G copper ports on the baseboard.
  • I have a 10G RJ-45 SFP plugged into one of the 10G SFP+ ports on the baseboard.
  • I have an intel NUC connected to the 10G RJ-45 SFP via a 6 foot CAT6 cable.
  • I’m trying to stream data from my two GigE cameras into my 10G SFP port.
  • On one of the cameras (camera A), it works fine and I get full frame rate on the NUC.
  • On the other camera (camera B), I’m seeing a reduced frame rate (dropped packets) on the NUC.
  • When I connect camera B directly to the NUC, I see the full frame rate, so I don’t think there is a problem with the camera.
  • I tried another off-the-shelf switch in place of UbiSwitch, and I don’t see these dropped frames.
  • I did notice that the ethernet PHY inside camera B is different to the ethernet PHY inside camera A.

Is there a problem on UbiSwitch that’s causing this?

2

There could be multiple reasons for this, I’ll list them out below and you can go through and try each.

Basic Hardware Checks

  1. Check your cabling for camera B. Is it a short cable, far from any potential EMI sources?
  2. Is your power supply suitable for both the camera and UbiSwitch?
  3. Are both SFP modules compatible with each other?

Layer 1 Ethernet checks

  1. Some ethernet PHYs need auto-negotiation to be turned off, otherwise there can be dropped packets. Using the UbiSwitch CLI, try turning off auto-negotiation and fixing the speed to 1Gbps, full duplex.
  2. There is a protocol called Energy Efficient Ethernet (EEE) that can cause this kind of issue. Essentially, some older ethernet devices don’t support EEE, which causes data loss. By default, UbiSwitch turns off EEE for the 10G SFP ports, but EEE is turned on for the 1G ports by default. Using the UbiSwitch CLI, trying turning off EEE on the copper ports

Higher level problems

  1. If you still can’t solve the issue, it could be related to the software that is consuming the data on your NUC. In this case, run a wireshark and try to see what is actually happening on the wire. You’ll probably need to get in touch with our support so you can share the packet capture and we can investigate further.
3

Turning off EEE solved the issue for us, we now see full data rate on both cameras! Out of interest, how much energy does EEE actually save? We care about low power in our application.

1 Like
4

Normal 1GBASE-T constantly sends IDLE frames, even when there is no data. EEE changes this, so that IDLE frames are not constantly being sent, instead the ethernet circuitry is powered down, and periodic refresh signals are sent, rather than constant IDLE frames. As you can see, EEE only reduces power when there is no data being sent.

IEEE estimates that it can save around 0.2 Watts for each 1GBASE-T port, assuming that port is only sending 10% of the time. If a port is being used to full utilisation, then it won’t save any power. 0.2 Watts is not insignificant and in a data center type of network where there are 1000s of 1G ports, many of which are highly underutilized, these power savings really do stack up!

On mobile systems applications, the benefit is marginal because (a) your port utilisation is probably more than 10% and (b) you only really have a maximum of 8 ports, so it doesn’t really scale. The issue is that many older ethernet circuits (particularly the one in your GigE camera) don’t support EEE, and it just messes things up and causes integration hell.

We only know of this issue through debugging with other customers. Unfortunately this is the case with ethernet; in general, when it works, it’s absolutely wonderful, but when there are issues, you need to understand how the underlying ethernet standard works. In our case that’s often meant even talking to the original chip designers for some of the older ethernet PHYs, to understand some of the bugs in the silicon.

It seems like one of your cameras does support EEE so you could turn EEE back on for the port that is connected to that camera. Then again you might take the view that the power savings are marginal and it’s just not worth the hassle if you change the camera on that port. Most of our customers that encounter this issue kill EEE on all ports just to eliminate this potential problem in the future.