WO2018182447A1

WO2018182447A1 - Video signal processing method

Info

Publication number: WO2018182447A1
Application number: PCT/RU2017/000201
Authority: WO
Inventors: Петр Вячеславович СЕВОСТЬЯНОВ
Original assignee: Общество С Ограниченной Ответственностью "Заботливый Город"
Priority date: 2017-03-31
Filing date: 2017-04-04
Publication date: 2018-10-04
Also published as: RU2647664C1

Abstract

A video signal processing method relates to the field of computer technology, and in particular to video signal processing methods, and can be used in augmented reality and virtual reality (AR/VR) systems for reducing display system delay. The technical result provided by the technical solution claimed consists in reducing delay when a video signal is output to a screen. The technical result is achieved by using FIFO algorithms programmed in PLICs and chips that receive a signal from an HDMI connector and perform buffering of less than one line of pixels, making it possible to reduce delay.

Description

VIDEO PROCESSING METHOD DESCRIPTION

FIELD OF TECHNOLOGY

[0001] The technical solution relates to the field of computer technology, in particular 5 to methods for processing a video signal and can be used in virtual and augmented reality (AR / VR) systems to reduce the delay of the visualization system.

BACKGROUND

[0002] The problem of video display delay is an urgent problem in the implementation of most AR / VR technologies based on tracking systems. In tracking AR / VR systems, the virtual environment changes in response to user movement. The amount of delay is the total time between the movement of the user and the corresponding change in the image on the display.

15 [0003] Features of the visual perception of three-dimensional objects lead to the fact that the delay in the response of the system to a change in the user's position is perceived not as a temporary phenomenon, but as a spatial phenomenon, that is, the brain does not perceive the delay of the system, but sees how the object of cubic shape changes its proportions in observer movement time, and

20, realistic perception is achieved only when the user is motionless for a time equal to or greater than the delay time of the system.

[0004] Due to the apparent discrepancy between visual information and user position information coming from the vestibular apparatus, delay is the most common cause of symptoms in users

25 seasickness: dizziness, nausea, weakness.

[0005] The choice of a display device directly affects the amount of total system display delay, since most modern displays, unlike older CRT displays, perform buffering and additional image processing to improve its quality, and this leads to a delay in output, which is often very significant .

[0006] Thus, it turns out that for serial television LCD displays, the actual display delay is significantly higher than the minimum time to display the image due to physical screen characteristics. So, for LCD displays with a response time of 2 ms, the actual display delay is 20-140 ms due to additional image processing on the control panel of the screen.

[0007] Additional processing for television panels is designed to improve the quality of the input signal, and is relevant in a situation where the quality of the input television signal is unknown. However, for an AR / VR system, in which the rendering system on the graphics server initially prepares a high-quality image, the additional processing on the screen control board is redundant,

[0008] In addition to the large display delay, a serious limitation of large-format serial LCDs (32 "and higher) is the low resolution in 3D (120 Hz mode).

[0009] To date, the HDMI protocol allows the transmission of 1280x720 (HD) image at 120 frames per second (60 frames per eye). The 48 Hz mode (24 frames per eye) is supported for 1920x1080 (FullHD) image output: a similar mode is acceptable when watching movies, but for dynamic scenes when moving the user relative to the screen it is uncomfortable. It is worth noting that the HDMI specification also describes the transmission format 1920x 080 @ 20fps, but this format is optional, and its support is not found among serial television LCD panels.

[0010] The inventors developed a video signal processing board for an LCD panel with a near-zero delay, with support for 1920x1080 resolution (FullHD) in 120 Hz mode. The near-zero processing delay is achieved through the use of high-speed parallel signal processing algorithms using FPGAs.

[0011] A method and apparatus for rendering images using parallel processing is known from US5546530A. This solution offers rendering and processing by dividing the whole image. The image is rendered on this device by parallelizing the processing of information by processors, after which the pixels from each processor are sent to a separate buffer for the further formation of a single unit and output. [0012] A portable device integrable with an external video signal function is known from US20080018789. This device includes a video capture interface, a display, a display chip, and a controller. The video capture interface receives an external video signal, and display chip 5 provides an internal video. The controller includes a first input interface, a second input interface and an output interface. This device includes additional pixel processing and buffering, which leads to an increase in delay.

[0013] The above examples use post-processing of the image, a large amount of buffering, which, despite the use of parallelization in some of them, leads to an increase in delay.

ESSENCE OF TECHNICAL SOLUTION

[0014] The task to which the claimed technical solution is directed is to process a video signal with a minimum delay.

15 [0015] The technical result provided by the claimed technical solution is to reduce the amount of delay when outputting the video signal to the screen.

[0016] In general terms, the task to which the claimed technical solution is directed is achieved through the use of FIFO 20 algorithms programmed on the FPGA and chips receiving a signal from the HD I connector and buffering less than one row of pixels, which reduces the delay.

[0017] The FPGA based on the FIFO1 and FIFO2 systems outputs the video signal through the LVDS transmitter to the screen portionwise for each half of the screen, without combining, 25 which also allows reducing the delay value.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The signs and advantages of the claimed technical solution will become apparent from the following detailed description and the accompanying drawings, in which:

zo [0019] FIG. 1. shows a diagram of the video signal processing cycle.

[0020] In FIG. 2 shows a diagram of the FPGA operation algorithm.

[0021] In FIG. 3 shows a diagram of a video processing board.

[0022] In FIG. 4 shows a block diagram of a video signal processing algorithm. DETAILED DESCRIPTION OF THE TECHNICAL SOLUTION

[0023] Hereinafter, the terms used in the application will be described.

Programmable logic integrated circuit (FPGA) is an electronic component, 5 used to create digital integrated circuits. Unlike conventional digital microcircuits, the FPGA logic is not determined during manufacturing, but is set through programming.

[0024] High Definition Multimedia Interface (HDMI) is an interface for high definition multimedia that allows you to transmit high-resolution digital video data and copy-protected multi-channel digital audio signals.

[0025] HD is a resolution of 1280 * 720 pixels (pixels).

[0026] Full HD - resolution 1920 * 1080 pixels (pixels). This marketing name was first introduced by Sony in 2007 for a number of products. 15 Used in broadcasts of high-definition television (HDTV), in televisions, computer displays, in smartphone cameras, in video projectors, etc.

[0027] FIFO (the acronym First In, First Out - “first come, first go”) is a way of organizing and manipulating data relative to time and

20 priorities. This expression describes the principle of technical processing of a queue or servicing conflicting claims by streamlining the process according to the principle: “first come, first served”. The one who comes first is served first, the next one waits until the first service is completed, and so on.

25 [0028] Augmented reality (English augmented reality, AR - “augmented reality”) is the result of introducing any sensory data into the perception field in order to supplement information about the environment and improve the perception of information.

[0029] Virtual reality (BP, Engl. Virtual reality, VR, artificial reality) is a world created by technical means (objects and subjects), transmitted to a person through his sensations: sight, hearing, smell, touch, and others. Virtual reality simulates both exposure and response to exposure. To create a convincing complex of sensations of reality, a computer synthesis of the properties and reactions of virtual reality is performed in real time. [0030] Low-voltage differential signaling (LVDS) is a method of transmitting electrical signals that allows the transmission of information at high frequencies using cheap connections based on copper twisted pair.

5 [0031] Integrated circuit, microcircuit, chip (eng. Chip - thin plate - originally the term refers to the chip plate of a microcircuit) - microelectronic device - an electronic circuit of arbitrary complexity, made on a semiconductor substrate and placed in a non-separable case, in the case of occurrences in the microassembly.

Yu [0032] The claimed technical solution works as follows.

The frame is formed by a sp-engine with subsequent output to two video outputs. A circuit with two video outputs of one video card or with two video outputs of two different video cards is possible. From each video output, a video signal with a resolution of 960x1080 pixels is transmitted to the board through the corresponding

15 HDMI ports. Using the video synchronization check system, the presence of the video signal at the inputs of the board is checked. If there is a video signal, it is transmitted to the FPGA, which synchronizes and combines the signals from two video outputs into a single video signal with a resolution of 1920x1080 pixels, and transmits it through an LVDS transmitter to the screen.

20 [0033] To minimize the delay FPGA is used, working with two FIFO systems, the cycle of which is performed as follows.

[0034] There is a accumulation of pixels in the first FIFO system, with a synchronous accumulation of pixels in the second FIFO system, until the threshold in the first FIFO system is sufficient to output one row of pixels to

25 first half of the screen.

[0035] A row of pixels is output to the first half of the screen, with a synchronous accumulation of pixels in the second FIFO system.

[0036] After the end of the output of the pixel row in the first half of the screen, the output of the pixel row in the second half of the screen begins, with synchronous accumulation of pixels of the next row in the first and second FIFO systems.

[0037] The loop is repeated until all the pixels in the frame are displayed. After the output of all the pixels of the frame, the cycle starts again for the next frame. [0038] Ultimately, the maximum delay of the above video signal processing method will be less than one line of the frame, which corresponds to less than 10 microseconds.

[0039] At this stage of technology development in the consumer electronics market, there are no analogues of the claimed solution for drastically reducing the delay, and the claimed technical solution is unique in its kind. The claimed technical solution can also serve not only for use in conjunction with MotionParallax3D technology and the corresponding LCD displays, but also for general purposes on standard LCD displays. It is also important to note that such an implementation of the solution supports FullHD resolution in 3D mode at 120Hz.

[0040] The claimed technical solution is industrially applicable, since industrialized and industrially applicable resources are used for its implementation.

[0041] Although the claimed technical solution is described by a specific example of its implementation, this description is not limiting, but is provided only to illustrate and better understand the essence of the technical solution, the volume of which is determined by the attached formula.

Claims

VIDEO PROCESSING METHOD

FORMULA

[0042] 1. A method for processing a video signal, comprising the steps of:

using at least one HDMI port receive a video signal from at least one video card;

using at least one chip, divide the received video signal and transmit the divided signal to the FPGA;

process the video signal using FPGA as follows:

accumulate pixels in the first FIFO system, with synchronous accumulation of pixels in the second FIFO system, until the threshold value is sufficient in the first FIFO system to display one row of pixels in the first half of the screen;

using an LVDS transmitter, a pixel string is output to the first half of the screen, with a synchronous accumulation of pixels in the second FIFO system;

after the end of the output of the pixel line to the first half of the screen, the output of the pixel line to the second half of the screen is started using the LVDS transmitter, with the synchronous accumulation of pixels of the next line in the first and second FIFO systems;

repeat the cycle until all the pixels in the frame are displayed.