Rapid deployment of configurable hardware: programming framework and accelerators for fast deployment of low latency low-power FPGA-based video processing algorithms

Description

Peripherals such as the wireless endoscopy sensor are low-power and low-latency devices with a small form factor, mostly in the form of dedicated ASICs. A limited flexibility is provided as the required components’ characteristics are driven by the image sensor. New functionality can already be supported by the peripheral. Extensions, such as detection of suspicious features or enhanced image characteristics, can be considered. Video processing architectures are still of great concern as processing power and power consumption are conflicting goals that can only be attained by using dedicated hardware accelerators supplied by chips designers at the cost of loose of flexibility. As products continuously evolve, it is important to provide a degree of freedom for future development and that includes the hardware part. Before recurring to fine grain reconfigurable, as this implies the use of expensive FPGA chips and long development times, we propose a software configurable platform supported by coarse grain hardware accelerators seen as library components. A successful preliminary version of this architecture supported a basic set of image processing.

We will extend the capabilities of that architecture towards several directions. First, to facilitate its adoption by software developers, in addition to the creation of a super-set of operators, we are going to consolidate the capability to develop, in a transparent manner, applications based on hardware accelerators. By just supporting the standard architecture, a novel IP core could be produced to support image sensor and compression characteristics of any kind. The flexibility can be extended by supporting configurable video processing capabilities such as inter-frame processing operators; more attractive algorithms can profit from that extension, which includes not just transform but also to extract information from the video stream. Two flavours can be considered: a generic architecture automatically tunable according to user and end-product requirements and a configurable one. The first alternative has the advantage of producing as result a customized IP core. By setting implementation parameters prior synthesis, requirements such as size and power consumption will be fully satisfied. The second one offers the advantage to support evolvable functionality configured after synthesis. Certainly, this option must still be competitive in term of size and consumption, compared to the previous one. Finally, the configuration process sometimes