New method and FPGA for compression of industrial data streams

Method, device, and computer program product for compressing an input data set. Data compression by means of adaptive subsampling.

• The data is compressed directly at the source
• Better compression rates than previous methods
• No expensive upgrade of the entire system hardware required, just the addition of an FPGA at the source or as a software-only solution
• Low computational effort
• No artifacts, no data loss
• Applicable to all codecs

Industrial process chains that generate large streams of data such as:

• High resolution camera systems
• Quality control through imaging techniques
• Video transmission
• Audio transmission
• Detection sensor technology

FPGA (Field Programmable Gate Arrays) are integrated circuits that can be configured for a wide variety of tasks. To perform this task in a process line, the FPGA is integrated into the hardware (e.g., a high-resolution camera) as a module and the data processing software is adapted accordingly.

In industrial data processing, ever greater data transfer rates are required; the common standard is Gigabit Ethernet. This standard supports maximum data transmission rates of 1,000 megabit per second, but the raw data of modern imaging procedures often exceeds this rate. Connections with higher transmission rates mean far greater investment, since the entire hardware must be adapted in addition to the transmission protocol.

The Institute for Parallel and Distributed Systems (IPVS) at the University of Stuttgart has many years of experience in this field and has developed a combined software and hardware solution to solve this problem.

The new method enables virtually lossless compression of any industrial data stream. Specially developed hardware circuits (FPGA), precisely tailored to the required application purpose, are deployed before transferring the data. The FPGA is able to compress the data directly at the source. This allows large data streams to be sent and processed in the system without limiting data quality.

The process is being implemented in several steps:

1. Decoding of the data in the lossy encoder
2. Subtraction from the original data
3. Location-selective encoding of the difference (optionally lossless or lossy)

Z. Wang, S. Simon, Y. Baroud and S. M. Najmabadi, "Visually lossless image compression extension for JPEG based on just-noticeable distortion evaluation," 2015 International Conference on Systems, Signals and Image Processing (IWSSIP), 2015, pp. 237-240, doi: 10.1109/IWSSIP.2015.7314220. (https://ieeexplore.ieee.org/document/7314220)