Further optimisations of FPGA output

[JamieJQuinn]

For context, I'm part of a team at UCL, UK working on porting codes to FPGAs and we're looking at a variety of tools to do so, including dace. We've already been working in HLS targetting Xilinx and we're playing about with optimisations to memory storage, pipelining, streaming, etc. We're wondering how we can access these kinds of optimisations from inside dace.

I know there's already some optimisations applied by the FPGA transformation, and that seems to produce HLS code with includes flattened and pipelined loops, for example from the axpy_transformed.py sample:

for (int i = 0; i < N; i += 1) {
  #pragma HLS PIPELINE II=1
  #pragma HLS LOOP_FLATTEN

If we wanted to inject different pragmas or play with the settings given to existing ones, how can we go about that?

@definelicht, tagging you because you seem to be the FPGA guru!

[definelicht]

Hi Jamie! The philosophy of DaCe is to code generate the appropriate pragmas based on the parallelism allowed by the representation. There are a few different ways of going about optimizations, depending on what you want to do, and how much control you want. Ranging from high-level to low-level:

• If you are starting from an SDFG (our IR) generated from one of the high-level frontends, there are graph transformations that can apply some FPGA-related optimizations.
• If you are producing the SDFG using the graph API (adding nodes and edges yourself, etc.), you can directly implement the structure you want (choosing what is pipelined, unrolled, what goes into fast memory, etc.), similar to how you would do in HLS, but on the level of abstraction of our graphs (making it portable between Xilinx and Intel also).
• If you want full control, we also allow embedding HLS code, or even RTL code, into your kernels wherever you want. This is of course only recommended if there's some specific low level detail that you want to manipulate.

For the examples you mentioned:

• Memory storage is annotated on the data containers, and can be either global (DRAM/HBM) or local (BRAM/URAM/LUTRAM/registers).
• Pipelining is automatically done to any map that is not unrolled. Maps are the dataflow nodes in our IR that express parallelism. Unrolling a map can be done by just setting the property unroll=True on a map. Optimization an existing code could look something like first tiling a map, and then unrolling the inner map.
• Streaming is implemented using the stream primitive in DaCe. Processing elements are expressed by having multiple weakly connected components in a dataflow section, that communicate by reading and writing streams.

If you are interested in something specific, I can point you to the right place :-)

[JamieJQuinn]

Wow, thanks for the quick & detailed answer. I think that hierarchy of optimisations will suit us very well. Still scoping right now but I'll definitely be in touch if we come up with something we can't figure out, thanks!

Just a quick follow-up on streams: it looks like there's still some TODO work on streams. Does that mean FPGA streams aren't yet implemented at all or is that further stream work?

[definelicht]

FPGA streams are fully implemented. For example, there's a full systolic array implementation of matrix multiplication here.

The TODO you found is with regards to this specific transformation. What is not handled yet is having streams in the dataflow section before applying the FPGA transformation. So if you're following the high-level flow, you would first transform the program to be offloaded to the FPGA, and only then apply transformations that introduce streaming communication (for example, streaming composition, which replaces an intermediate addressed memory between two dataflow sections with a stream, so they can run concurrently in hardware).

[JamieJQuinn]

Brilliant, thanks!