The NVIDA Tegra™ Processors TX1/TX2 with their powerful GPUs are ideal for use in professional audio mixing consoles or audio video equipment. However, if multiple audio channels are required, the TX1/TX2 is limited to I2S audio inputs. Utilizing the MIPI® Camera Serial Interface (CSI-2) and the InES I2S to CSI-2 converter IP, enables streaming of up to 256 digital audio channels into the TX1/TX2.
Institute of embeddeded Systems (InES) developed an FPGA-IP which converts the I2S audio to up to four CSI-lanes for feeding audio into mobile processors like the NVIDA Tegra™ TX1/TX2.
A Linux driver, which links the received CSI signals to the Tegra™ TX1/TX2 processor buses, is also available. Hence, audio can be processed on the TX1/TX2 GPU or the internal audio blocks.
I2S sources could be audio codecs, SDI or HDMI chips. The CSI-2 protocoll engine can be configured to generate CSI-2 data packets for one or four CSI lanes, depending on the required bandwidth. The CSI clock and data physical interfaces support differential (high speed) and low power CSI-2 signals.
The IP is written in VHDL and tested with Intel Cyclone-IV FPGAs. It is also possible to be synthesized into Xilinx or Lattice FPGAs.
For more information contact Hans-Joachim Gelke (firstname.lastname@example.org)
Block Diagramm of I2S to CSI IP
The InES HPMM research group presents a concept for a mobile and redundant 4K video streaming over LTE networks. It combines powerful 4K video capturing and processing capabilities of dedicated accelerators with the modularity and flexibility of an embedded high performance SoC. The Nvidia TX2 Module is the ideal platform for this purpose.
Since the TX2 supports efficient HEVC encoding, one stream in 4k quality 1), or several streams in HD-quality 2) can be transmitted over one LTE connection 3). Several LTE channels can be combined together for redundant transmission via different LTE networks.
A video input mixer on the NVIDIA-TX2 GPU allows scaling, overlay and side by side mixing of video sources.
HDMI is fed directly into the TX2 video path via a HDMI to CSI converter.
1) Main profile, up to 1 x 2160p60
2) 4x 1080p60 or 8x 1080P30
3) min. 5 Mbps are required for 2160p30
Institute of Embedded Systems, a research institute of Zurich University of Applied Sciences generated a reference design for a low latency, highly reliable wireless video transmission from a battery operated camera to an iPad or iPhone. The design is suitable for everything that requires a robust low latency video link such as vehicle remote control, industrial applications, automotive applications and others. Since the transmission is Wi-Fi based, no extra hardware to receive the video stream on an iPad or iPhone is required.
The camera module consists of an Intel SoC-FPGA with integrated single core ARM-A9 with flexible interface to various types of cameras and SDIO interface to the Wi-Fi module. Optional LCD interfaces or an SD-card slot allow monitoring and recording of the video at the camera module.
The low latency video compression algorithm is nearly lossless and always transmits full frames. While the compression is implemented in the FPGA fabric, control is accomplished by a Linux operating system in the ARM-A9.
Error correction avoids pixel and frame drops even if Wi-Fi transmission is problematic, like in busy areas or in difficult topography. The Wi-Fi standard includes automatic retransmission of lost packets. However, there still remains a chance that packets are lost. To increase reliability even further, we add redundant packets. This slightly increases the bandwidth however does not add significant latency.
To receive the video stream, it is enough to install a viewer app, no extra hardware is required. Video decompression and error correction are solely handled in the GPU and the CPU of the iPad.
The FPGA IP requires only 2.9k logic cells, which is 18% of a 15k logic cell Intel Cyclone-V SoC.
The transmitter IP controls an 802.11n Wi-Fi module like the Texas Instruments WL1835MOD, however other TI modules are supported as well.
The measured glass to glass latency can be as low as 65 ms (2 video frames at 30 fps). However, dependent on the selected compression rate and the Wi-Fi channel quality, the latency might be higher.
For more information, contact Tobias.Welti@zhaw.ch