6. Development

This section provides guidelines on how to develop a new WRTD Node (or, modify an existing one), including Hardware, Gateware and Firmware, using the resources provided by the WRTD project. If you simply need to control an existing WRTD Node, please refer to Section 4 instead.

It should be noted that a new Node does not necessarily need to use the resources provided by the WRTD project itself. In fact, the only hard requirement is that the Node must be able to send and/or receive Event Messages over an Ethernet-based LAN, using multicast UDP on address 224.0.23.159, port 5044. It should also respect the Event ID naming conventions of WRTD.

Of course, the above is the absolute minimum requirement, which is already fullfilled by any LXI device supporting the relevant LXI extended functions (see also Relation to IVI and LXI).

Moving one step further, the network interface of the Node should be White Rabbit enabled, to allow for accurate timestamping and sub-ns synchronisation between the Nodes. This has implications for your Hardware.

If your Node contains an FPGA and you choose to use the WRTD-provided Gateware resources, then this also opens up the possibility of using the WRTD-provided Firmware resources as well, to develop the application(s) running inside the FPGA. A WRTD Node developed with the WRTD Gateware and Firmware resources will be accessible via the provided C Library, Python Wrapper and Tools.

6.1. Setting Up the Environment

The WRTD development environment has been tested and works best under Linux.

The source code of WRTD is maintained in a Git repository. To create a local copy of it, please use:

$ git clone "https://ohwr.org/project/wrtd.git"

And then checkout the correct commit, branch or tag that you wish to work on. Typically, the latest stable commit will be tracked by the master branch, while the latest development commit will be tracked by the proposed_master branch.

Once this is done, you need to bring in the various dependencies of WRTD (which are present as Git submodules within the WRTD repository):

$ cd wrtd
$ git submodule update --init

Important

Despite the fact that many of WRTD’s dependencies contain their own submodules, you should not bring in those dependencies recursively, as they may create conflicts (e.g. both WRTD and one of its submodules depending on another submodule but on a different version of it). WRTD’s top-level dependencies (which are brought in with the above command) contain everything that is needed for development.

Furthermore, you will need a RISC-V cross-compilation toolchain for compiling the firmware for the Nodes.

Note

Because at the time of the release of WRTD v1.0, the OHWR deployment procedures were undergoing significant changes (in particular with respect to packaging), up-to-date installation instructions for all the necessary development tools will be available through the project Wiki. Once these procedures have been finalised and tested, the contents of the wiki page will be merged here.

Alternatively, you can try to build your own toolchain like this:

$ git clone "https://ohwr.org/project/soft-cpu-toolchains.git"
$ cd soft-cpu-toolchains/riscv
$ sh ./build-riscv.sh

This will checkout the master branch of the repository and start the build process, which takes a long time to complete.

Once the process is complete, the resulting RISC-V toolchain binaries will be placed under soft-cpu-toolchains/riscv/riscv-toolchain/bin. You should either copy them to somewhere within your system’s path (typicallly /usr/local/bin), or set the environment variable CROSS_COMPILE_TARGET to point to the correct location, with the riscv32-elf- suffix appended to the path, like this:

$ export CROSS_COMPILE_TARGET="<path_to_repository>/riscv/riscv-toolchain/bin/riscv32-elf-"

You either need to configure the above command to be run automatically on a new terminal (typically by putting the command in ~/.profile or ~/.bash_profile but this depends ultimately on the shell you are using), or you should run it every time you start a new terminal and need to compile RISC-V firmware.

6.2. Hardware

Hardware-wise, WRTD does not impose any particular requirements. It is assumed that the device is capable of producing and/or consuming “events” (e.g. trigger in, trigger out), otherwise there is no point in using WRTD.

Other than that, the only requirement on the hardware is that it supports White Rabbit (WR).

If you are designing your own board and you would like to have something as a reference, here’s a page which can help you, or, more specifically, here’s a list with a few open-source boards that already support WR:

  • SPEC (Xilinx Spartan6 FPGA)
  • FASEC (Xilinx Zynq SoC FPGA)
  • VFC-HD (Intel Arria V)

All of the above examples make use of the WR PTP Core inside their FPGA.

6.3. Gateware

There is no HDL template yet for WRTD gareware development. Users who need to develop WRTD gateware for a new Node are advised to use the top-level VHDL modules of the Reference Nodes as a starting point. Their source code is available on the WRTD Git repository:

https://www.ohwr.org/project/wrtd/tree/master/hdl/top

Every Node should include at a minimum an instance of MockTurtle and the WR PTP Core.

Furthermore, the MockTurtle configuration should define at least one Host Message Queue per CPU, for communication between the WRTD library and the WRTD Application running on the CPU, as well as one Remote Message Queue per CPU for conencting the WRTD Application to the WR network.

Every Node should also include a MockTurtle Ethernet Endpoint, to interface the Remote Message Queue of the MockTurtle with the Fabric Interface of the WR PTP Core. If in doubt, have a look at how these three modules are instantiated and connected to each other in the Reference Nodes.

6.4. Firmware

WRTD provides a common firmware development framework for Applications. To use it, users must also use a compatible Gateware.

Similar to Gateware development, users are advised to use the firmware of the Reference Nodes as a starting point:

https://www.ohwr.org/project/wrtd/tree/master/software/firmware

6.4.1. Setup

To start developing firmware for WRTD, users must first include the relevant header file:

#include "wrtd-rt-common.h"

Users must also describe the various features of their application, by means of C preprocessor directives. The following needs to be copied to the application source code and adjusted as necessary:

/* Number of MockTurtle CPUs. */
#define NBR_CPUS xxx
/* Index of this CPU. */
#define CPU_IDX xxx
/* Maximum number of Rules allowed */
#define NBR_RULES xxx
/* Maximum number of Alarms allowed */
#define NBR_ALARMS xxx
/* Number of Devices. A "Device" is a unidirectional set
   of Local Channels.
   Most Applications will have one Device per direction (in/out).
   Maximum allowed number of Devices = 4. */
#define NBR_DEVICES xxx
/* Number of Local Channels per Device. */
#define DEVICES_NBR_CHS { xxx, 0, 0, 0}
/* Direction of Local Channels per Device. Direction can be either
   WRTD_CH_DIR_IN (from the environment to WRTD) or
   WRTD_CH_DIR_OUT (from WRTD to the environment). */
#define DEVICES_CHS_DIR { WRTD_CH_DIR_IN, 0, 0, 0}
/* A unique number to identify this Application. */
#define APP_ID xxx
/* Application version (major, minor). */
#define APP_VER RT_VERSION(xxx, yyy)
/* A string name for this Application. */
#define APP_NAME xxx
/* 1 if the Application can receive Events over the network,
   0 otherwise. */
#define WRTD_NET_RX x
/* 1 if the Application can send Events over the network,
   0 otherwise. */
#define WRTD_NET_TX x
/* 1 if the Application can receive Events from Local Channels,
   0 otherwise. */
#define WRTD_LOCAL_RX x
/* 1 if the Application can send Events to Local Channels,
   0 otherwise. */
#define WRTD_LOCAL_TX x

Apart from the preprocessor definitions, users must also provide implementations for a set of functions. These include:

6.4.2. Initialisation and Main Loop

WRTD firmware runs in a continuous loop. All such code should be put in the wrtd_io() function. Additionally, the wrtd_user_init() offers the possibility to execute code once, after reset, before entering the main execution loop.

static int wrtd_user_init(void)

Function to perform any apllication-specific initialisation (runs once after reset).

If the application does not need this, write a function that returns always 0.

Return
0 on success, or error code otherwise.

static void wrtd_io(void)

Function to perform the main tasks of the application. This will run in a loop.

Typically the application will check here for incoming Events and pass them to WRTD via the wrtd_route_in() function. It might also want to check if output Events previously scheduled via wrtd_local_output() have been executed or not.

If the application does not need this, write a function that returns always 0.

Return
0 on success, or error code otherwise.

6.4.4. Event I/O

Ultimately, the purpose of a WRTD firmware is to relay Events from/to the outside world.

Internally, WRTD represents an Event using the wrtd_event structure.

struct wrtd_event

WRTD Event

Public Members

struct wrtd_tstamp ts

Time of the event.

char id[WRTD_ID_LEN]

Event id.

uint32_t seq

Sequence number.

unsigned char flags

Associated flags.

6.4.4.1. Sending Events

When an incoming Event has been matched to a Rule with a Local Channel output, WRTD will call the user-provided wrtd_local_output() function. This function should perform all the application-specific actions to program the relevant Local Channel to generate the actual output.

static int wrtd_local_output(struct wrtd_event *ev, unsigned ch)

Generate an output Event on a Local Channel.

If the application does not need this, write a function that returns always 0.

Return
0 on success, or error code otherwise.
Parameters
  • ev: pointer to a struct wrtd_event, representing the Event to send.
  • ch: Local Channel number to use.

6.4.4.2. Receiving Events

Receiving an Event is also application-specific. The monitoring of Local Channel inputs is typically done periodically in the main execution loop, using the wrtd_io() function.

Once the firmware detects in incoming Event and fills in a wrtd_event structure, it should simply pass this to WRTD using the wrtd_route_in() function. The rest (rule matching, event forwarding, etc.) will be handled by WRTD.

static void wrtd_route_in(struct wrtd_event *ev)

Route an event coming from a Local Input, Network, or Alarm.

Parameters
  • ev: pointer to a struct wrtd_event, representing the received Event.

Hint

Contrary to the rest of the functions presented in Section 6.4, wrtd_route_in() is not a user-defined function. It is already provided by the WRTD firmware development framework. Users should simply create and fill in the wrtd_event ev structure with the details of the received Event and then pass it on to WRTD by calling this function.