Broadcom PEX87xx RDK Series Design Guide: PCIe Gen 3.0 Switch Architecture & Evaluation

Engineer's Takeaway

• Positioning: The Broadcom PEX87xx RDK Series serves as the primary validation platform for the PEX8700 family of PCIe Gen 3.0 switches, enabling developers to prototype high-bandwidth connectivity for storage and server architectures before custom PCB fabrication.

• Key Spec Highlight: Supports 48 Lanes / 12 Ports (PEX8748 variant) with a non-blocking switch fabric operating at 8.0 GT/s per lane.

• Supply Chain Status: Special Order / Low Stock. Due to high competition and component scarcity, utilize simulation models early in the design cycle while awaiting hardware lead times.

1. Technical Architecture and Core Advantages

The Broadcom PEX87xx RDK Series provides a hardware reference design for implementing Packet Switching architectures based on the PCI Express Base Specification 3.0. These boards demonstrate the capabilities of the PEX87xx silicon, focusing on low-latency data movement between multiple hosts and endpoints.

1.1 Processing & Control (The "Brain")

Unlike general-purpose microcontrollers, the "brain" here is a dedicated PCIe Packet Switching Engine.

• Cut-Through Architecture: The PEX87xx switches utilize a cut-through architecture rather than store-and-forward, drastically reducing latency for packets passing between ports.

• Link Speed: Validated for Gen 3.0 speeds (8.0 GT/s), auto-negotiating down to Gen 2.0 (5.0 GT/s) and Gen 1.0 (2.5 GT/s).

• Management: Includes an on-chip management processor for initializing the switch, handling hot-plugs, and configuring port width/lane reversal without intervention from the host CPU.

1.2 Peripherals & Interfaces (The "Limbs")

The RDK exposes specific high-speed interconnect features critical for High-Performance Computing (HPC):

• DMA Engines: Integrated engines offload data movement tasks from the host CPU, essential for NVMe storage aggregation.

• Multi-Host / NTB: Supports Non-Transparent Bridging (NTB), allowing multiple hosts (servers) to share the switch or create failover clusters.

• Debug Interfaces: The RDK typically exposes JTAG and I2C/SMBus headers for side-band management and diagnostics.

2. Naming / Variant Map and Selection Guide

2.1 Part Number Decoding

The RDK part numbers follow a structured convention indicating lane and logic capability:

• PEX: Product Family (PCI Express Switch).

• 87: Series Generation (Gen 3.0, 8000 series).

• xx: Lane Count (e.g., 24 = 24 lanes).

• CA: Silicon revision/grade (specific to the chip mounted).

• RDK: Rapid Development Kit hardware.

2.2 Core Variant Comparison

3. Key Specifications Explained

Engineer's Note: Values below are typical. Always consult the specific datasheet for max/min limits.

3.1 Power & Operating Conditions

PCIe Gen 3.0 switching requires significant power budgets, particularly for the PHYs processing 8.0 GT/s signals.

 - Power Methodology: The RDK is typically powered via the PCIe slot (up to 75W) or an external ATX/12V connector for high-lane variants. 

- Thermal Management: The PEX87xx silicon on these RDKs usually requires active cooling (heatsink + fan) or significant airflow, as junction temperatures can rise quickly under full load.

3.2 Performance & Efficiency

The defining metric for this series is throughput and latency. 

- Throughput: 8.0 GT/s raw bit rate per lane. 

- Fabric Efficiency: Non-blocking internal interconnect ensures that traffic on one port does not artificially throttle traffic on another. 

- Latency: Low-latency forwarding (typical <150ns range [AI Added], refer to datasheet for specific packet size curves).

4. Design Notes and Common Integration Issues

The leap from Gen 2 to Gen 3 PCIe imposes strict rules on physical design.

4.1 PCB Layout Guidelines

• Trace Impedance: Must maintain strict 85Ω or 100Ω differential impedance (depending on stack-up specifications provided in the Broadcom PEX87xx RDK Series datasheet).

• Insertion Loss: At 4GHz (Nyquist for 8GT/s), FR4 materials may be insufficient for long traces. Low-loss substrates (e.g., Megtron 6) are often recommended.

• AC Coupling: Requires carefully placed AC coupling capacitors (typically 176-265nF) on TX lines.

4.2 Debugging Common Faults (Pain Points)

Integration of Broadcom PEX switches often encounters specific hurdles:

1. High Entry Cost

• Problem: RDKs for high-lane-count switches (PEX8748) are expensive, straining budgets for small prototypes.

2. Fix: Utilize simulation models (IBIS/HSPICE) or procure lower-lane-count variants (PEX8713) for initial driver/software development before committing to high-density hardware.

3. Signal Integrity Complexity

• Problem: Designing custom boards to match RDK performance at 8GT/s is difficult; links may train down to Gen 2 or Gen 1 speeds due to bit errors.

4. Fix: Strictly adhere to Broadcom's layout guidelines regarding via stubs and intra-pair skew. Use high-quality dielectric materials and perform post-layout signal integrity simulations.

5. Typical Applications

5.1 System Integration Analysis: Enterprise Storage Arrays

In All-Flash Arrays (AFA), the Broadcom PEX8748-CA-RDK acts as the central fabric connecting a CPU root complex to multiple NVMe SSDs. 

- Problem: A typical CPU has limited PCIe lanes (e.g., 64 or 128), insufficient for connecting 24+ NVMe drives directly. 

- Solution: The PEX8748 provides fan-out capabilities, aggregating bandwidth. Its internal DMA engines enable Peer-to-Peer (P2P) transfers between drives or network cards (RDMA) without burdening the host CPU, significantly increasing IOPS.

6. Competitors and Alternatives

The PCIe switch market is consolidated. 

- Vs. Microchip (Switchtec): Switchtec switches often emphasize diagnostic features and cable management for heavy storage users. Broadcom PEX series generally leads in widely established driver support and legacy integration. 

- Vs. Renesas (formerly IDT): Renesas provides strong alternatives in the timing and interconnect space. Selection often comes down to specific lane configuration availability and power consumption profiles suitable for the specific design constraint. 

- Migration: Migration between brands is generally not pin-compatible. A redesign of the PCB is required.

7. FAQ

• Q: What is the maximum data rate supported by the Broadcom PEX87xx RDK Series?    The series supports PCIe Gen 3.0, achieving data rates of 8.0 GT/s per lane.

• Q: Can the PEX8748-CA-RDK operate in Gen 1 or Gen 2 modes?    Yes, the PEX87xx switches are backward compatible and will auto-negotiate link speeds based on the connected device's capabilities.

• Q: Does the Broadcom PEX87xx RDK Series support multi-host configurations?    Yes, these switches support multi-host architectures and Non-Transparent Bridging (NTB), allowing for complex redundancy and failover designs.

• Q: Why is the price of these RDKs high compared to standard dev boards?    These acts as enterprise-grade reference designs with expensive high-speed substrate materials and complex BGA silicon, targeting specialized infrastructure development rather than hobbyist uses