This repository contains several test and proof-of-concept designs for data and time transfer over high-speed transceivers, which will enable distributed real-time I/O (DRTIO) in ARTIQ.
Communications are between two KC705 boards connected over SFP at 1.25Gbps line rate. Use 1310/1490nm SFPs with a single G.652 fiber (same as White Rabbit - http://www.ohwr.org/projects/white-rabbit/wiki/SFP).
Requires PySerial, Migen 0.4, MiSoC 0.3, and ARTIQ 2.0.
This work was supported by the Army Research Lab.
Build the designs by running demo_remote_led.py. The four DIP switches on the transmitting KC705 control LEDs 2-5 on the receiving KC705. LEDs 0-1 on both boards blink from the system and transceiver data clocks, respectively.
Build the designs by running demo_prbs.py. The transmitter outputs a pseudo-random pattern generated by a LFSR, that the receiver synchronizes to and verifies. The receiver counts errors and reports them on its RS232 link. Use demo_prbs.py --readout /dev/ttyUSBx to get the current value. The error count increases at the beginning while the receiver is synchronizing to the transmitter, but it stays constant after the receiver is locked - meaning the transmission is being received correctly.
The error counter is incremented by one when at least one error is detected in a 16-bit, 8b10b-decoded data word recovered from the fiber.
Due to the use of a self-synchronizing V.34-style PRBS checker and of 8b10b encoding, a single bit error on the fiber may cause multiple bit errors to be detected.
The receiver outputs the 62.5MHz raw recovered clock on USER_SMA_CLOCK_P, and the same clock after it has been jitter-filtered by the Si5324 on USER_SMA_CLOCK_N. The phase relationship between the transmitter and receiver clocks must be constant.
Build the transmitter design by running demo_artiq_ttl_tx.py. This design is a complete ARTIQ system which includes the system-on-chip bitstream, BIOS and runtime. It contains local RTIO TTL output channels and remote RTIO TTL output channels whose value is transmitted over the fiber to toggle TTLs on the receiving device. The file device_db.pyon that comes with this demonstration contains the RTIO channel mappings.
Run the following OpenOCD commands to flash the ARTIQ transmitter design:
ftdi_serial 123456789012 init jtagspi_init 0 bscan_spi_xc7k325t.bit jtagspi_program misoc_artiqttltx_kc705/gateware/top.bin 0x000000 jtagspi_program misoc_artiqttltx_kc705/software/bios/bios.bin 0xaf0000 jtagspi_program misoc_artiqttltx_kc705/software/runtime/runtime.fbi 0xb00000 xc7_program xc7.tap exit
The proxy bitstream bscan_spi_xc7k325t.bit can be found at https://github.com/jordens/bscan_spi_bitstreams or in any ARTIQ conda package for the KC705. See the source code of artiq_flash.py from ARTIQ for more details.
Refer to the ARTIQ documentation to configure an IP address and other settings for the transmitter device. If the board was running stock ARTIQ before, the settings will be kept.
Build the receiver design by running demo_artiq_ttl_rx.py. The bitstream is stand-alone and the design can be run by simply loading it into the FPGA's volatile memory.
Once you have the two devices running, you can perform the following:
- run
artiq_run line.pyfor an example that blinks LEDs simultaneously on both devices. - use TTL monitoring/injection in the ARTIQ dashboard.
- implement your own kernels. Due to the extremely simplified DRTIO protocol used in this demonstration, all delays must be multiples of 48ns. This restriction also applies to TTL pulse durations.
- verify clock stability. The transmitter outputs its 62.5MHz transceiver clock on USER_SMA_CLOCK_P. The receiver outputs the 62.5MHz raw recovered clock on USER_SMA_CLOCK_P, and the same clock after it has been jitter-filtered by the Si5324 on USER_SMA_CLOCK_N. The phase relationship between the transmitter and receiver clocks must be constant.
- Obtain the serial number of each board using
lsusb -v. It should be a number such as123456789012. - Run the following command to load a bitstream into the corresponding board's FPGA:
openocd -f board/kc705.cfg -c "ftdi_serial 123456789012; init; pld load 0 bitstream.bit; exit;"
If you are using the OpenOCD Conda package:
- locate the OpenOCD scripts directory with:
python3 -c "import artiq.frontend.artiq_flash as af; print(af.scripts_path)" - add
-s <scripts directory>to the OpenOCD command line.
Copyright (C) 2016 M-Labs Limited. Licensed under GNU GPL version 3, except gtx.py and gtx_init.py which are under 2-clause BSD to discourage the use of the Xilinx transceiver "wizards".