Puck Edge: Why do 1G devices fail to communicate through the 10G ports?

This topic was migrated from the BotBlox ticketing system after being summarized and anonymized.

Question

We’re testing a Puck Edge with a mix of 1G and 10G Ethernet devices. The dedicated 1G ports work as expected, but we’re having trouble connecting devices with 10/100/1000 network interfaces to the 10G ports.

So far, we’ve tried the following:

  • One 1G device on a 1G port and another 1G device on a 10G port: no communication.

  • Two 1G devices, both connected to 10G ports: no communication.

  • One 10G device on a 10G port and one 1G device on a 1G port: communication works.

Should 1G devices work through the 10G ports? Is there a configuration setting that could explain this behaviour?

Answer

Yes, 1G devices should be able to communicate through Puck Edge’s 10G ports. We reproduced this behaviour in the lab and traced it to an incorrect port configuration on the supplied lab test unit.

Using BloxOSLite, we checked the configuration and found that all three 10G ports were set to 10gbaser mode. This was the wrong mode for the PHYs used in Puck Edge. Changing those ports to usxgmii resolved the issue.

What caused the problem?

Puck Edge requires its three 10G ports to be configured in USXGMII mode. This configuration is normally applied during assembly, but the step had been missed on the lab test unit supplied for evaluation.

With the incorrect configuration, the dedicated 1G ports still worked, and communication was possible with a 10G-capable device connected to a 10G port. However, connecting a 1G device to one of those same 10G ports did not work.

In this case, the fix was to correct the port mode through the BloxOSLite command line and save the configuration.

How do I correct the configuration?

You will need access to Puck Edge’s UART interface and the following hardware:

  • A 6-pin Molex PicoClasp cable, available from DigiKey.

  • A 3.3 V TTL serial-to-USB converter.

Follow the UART board setup instructions to connect to the command line.

Once connected, enter these commands:

port 0 mac set usxgmii
port 9 mac set usxgmii
port 10 mac set usxgmii
conf save ports

The first three commands set the MAC interface mode for all three 10G ports to USXGMII. The final command saves the port configuration so that it persists after a reboot.

Note that 0, 9, and 10 are the command-line port identifiers used for this configuration.

What worked after the change?

After applying the configuration, we verified communication across the following combinations. Both devices in each row were connected through the same Puck Edge.

Device A Puck Edge port for A Puck Edge port for B Device B Communication verified
1G 1G 1G 1G 1G
1G 10G 1G 1G 1G
1G 10G 10G 1G 1G
10G 1G 1G 10G 1G
10G 10G 1G 10G 1G
10G 10G 10G 10G 10G
10G 1G 1G 1G 1G
10G 10G 1G 1G 1G
10G 10G 10G 1G 1G
10G 1G 10G 1G 1G

These tests confirmed that correcting the configuration restored communication with 1G devices connected to the 10G ports, while also allowing 10G communication between two 10G devices connected to 10G ports.

What if I cannot access UART?

Contact BotBlox support if you encounter these symptoms and cannot make the configuration change yourself. In this case, we also offered to exchange the evaluation unit for a correctly configured Puck Edge.