WO2017156742A1

WO2017156742A1 - Virtual reality-based image displaying method and related device

Info

Publication number: WO2017156742A1
Application number: PCT/CN2016/076584
Authority: WO
Inventors: 李刚; 龙寿伦
Original assignee: 深圳多哚新技术有限责任公司
Priority date: 2016-03-17
Filing date: 2016-03-17
Publication date: 2017-09-21

Abstract

A virtual reality-based image displaying method and a related device. Head posture display data Q1 displayed in a previous image is acquired (S201), head posture current data Q2' pre-displayed in a current image is acquired (S202), a first difference Δ between the head posture display data Q1 and the head posture current data Q2' is determined (S203), a determination is made on whether the difference Δ exceeds a preset threshold; and if yes, then the head posture current data Q2' is smoothed to produce head posture pre-display data Q2 (S205), thus reducing visually observed image jittering and drifting, and increasing user experience.

Description

Virtual reality based screen display method and related equipment

Technical field

The present invention relates to the field of image processing, and in particular, to a virtual reality based screen display method and related device.

Background technique

Virtual reality technology is a computer simulation system that can create and experience a virtual world. It uses a computer to generate a simulation environment. It is a multi-source information fusion interactive system simulation of three-dimensional dynamic vision and physical behavior to immerse users in the system. Environment.

In the virtual reality (VR) smart device work process, the user's head position needs to be tracked to display the image that the user can see in real time, specifically through the sensor integrated on the VR glasses. The gyroscope collects the offset angle θ data of the user's head, and then the MCU collects the collected data to generate the user's head angle data, and adds the predicted point deviation angle to the CPU and the GPU for processing and rendering, and finally reaches the real-time display. The image seen by the user's head.

As shown in FIG. 1 , the existing image data smoothing processing method generally adopts a mean value processing, that is, a position of several adjacent calculation points, for example, a calculation point position X1, X2 of a certain two adjacent time frames, and display of the latter frame. The point adopts the mean value, that is, (X1+X2)/2, to achieve smooth processing of the data, and reduce the user's picture jitter feeling. However, when the calculation point of a certain frame is greatly offset from the display point of the previous frame, The mean data smoothing method causes the display point X3 of the frame to be offset from the display point position of the previous frame, that is, the position difference between the two frames is large, which is easy to bring obvious image jitter to the user, and reduces the user experience. .

Summary of the invention

In view of this, an embodiment of the present invention provides a virtual reality based screen display method and related device.

A virtual reality based screen display method, the method comprising:

Obtaining the head posture display data Q1 of the last screen display;

Obtaining the head data current data Q2' pre-displayed on the current picture;

Determining between the head posture display data Q1 and the head posture current data Q2' a difference Δ;

Determining whether the difference Δ exceeds a preset threshold, if so, smoothing the head posture current data Q2' to obtain head posture pre-display data Q2;

The current screen is displayed based on the head posture pre-display data Q2.

Optionally, the method further includes:

If the difference Δ does not exceed the preset threshold, the current picture is displayed according to the head posture current data Q2'.

The smoothing processing of the head posture current data Q2' to obtain the head posture pre-display data Q2 includes:

The difference between the head posture pre-display data Q2 after smoothing and the head posture display data Q1 is made smaller than the first difference Δ.

Optionally, the difference between the smoothed processed head posture pre-display data Q2 and the head posture display data Q1 is smaller than the first difference Δ.

Acquiring an average value of the head posture display data Q1 and the head posture current data Q2', and smoothing the head posture pre-display data Q2 into the average value;

or

Setting a smoothing adjustment value M, and smoothing the head posture pre-display data Q2 to a value at an adjustment smoothing value M in the same direction as the head posture display data Q1;

or

Obtaining a second difference Δ′ between the previous data Q1′ of the head posture of the previous screen display and the head posture current data Q2′, setting a smoothing adjustment value M, and smoothing the head posture pre-display data Q2 The processing is at a sum of the smoothing adjustment value M in the same direction as the head posture display data Q1 and the second difference Δ'. A virtual reality based screen display device, the device comprising:

An obtaining unit, configured to acquire head posture display data Q1 of the last screen display;

The obtaining unit is further configured to acquire a head posture current data Q2' pre-displayed by the current picture;

a determining unit, configured to determine a first difference Δ between the head posture display data Q1 and the head posture current data Q2';

a processing unit, configured to determine whether the difference Δ exceeds a preset threshold, and if so, smoothing the head posture current data Q2' to obtain head posture pre-display data Q2;

a display unit, configured to display the current picture according to the head posture pre-display data Q2 Show.

Optionally, the processing unit is further configured to:

If the first difference Δ does not exceed the preset threshold, the current picture is displayed according to the head posture current data Q2'.

Optionally, the processing unit is further configured to

The difference between the head posture pre-display data Q2 and the head posture display data Q1 after smoothing is smaller than the first difference Δ. Optionally, the image processing device is a virtual reality device.

A device for displaying a virtual reality based screen, comprising: a processor and a memory, wherein

a computer readable program is stored in the memory;

The processor is configured to execute a method based on virtual reality based screen display by running a program in the memory.

The virtual reality-based screen display method and related device in the embodiment of the present invention acquires the head posture display data Q1 displayed on the previous screen, acquires the head posture current data Q2′ displayed on the current screen, and determines the head posture. And displaying a difference Δ between the data Q1 and the head posture current data Q2′, determining whether the first difference Δ exceeds a preset threshold, and if so, smoothing the head posture current data Q2 ′ The head posture pre-display data Q2 is obtained, and the current screen is displayed according to the head posture pre-display data Q2, which can reduce the jitter and drift of the screen seen by the human eye, and improve the user experience.

DRAWINGS

1 is a schematic diagram of a method for processing a mean value in the prior art;

2 is a flowchart of an embodiment of a virtual reality based screen display method according to an embodiment of the present invention;

3 is a structural diagram of an embodiment of a virtual reality based screen display device in an embodiment of the present invention;

4 is a structural diagram of an embodiment of a virtual reality based screen display device in an embodiment of the present invention.

detailed description

In order to provide a better understanding of the present invention by those skilled in the art, the present invention will be described below. The embodiments of the present invention are clearly and completely described in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of the present invention.

The terms "first", "second", "third", "fourth" and the like in the specification and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order. Or prioritization. It is to be understood that the data so used may be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than what is illustrated or described herein. In addition, the terms "comprises" and "comprises" and "the" and "the" are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to Those steps or units may include other steps or units not explicitly listed or inherent to such processes, methods, products or devices.

As shown in FIG. 2, an embodiment of an image processing method provided in an embodiment of the present invention includes:

S201. Acquire head posture display data Q1 displayed on the previous screen.

The head posture display data Q1 refers to data used for displaying a picture in the previous frame.

S202. Acquire a head posture current data Q2' that is pre-displayed on the current screen.

The pre-displayed head pose current data Q2' refers to the display data that is to be used as the current frame screen display. It can be understood that it needs to be tested before it can be used. If it fails to pass the test, it can be used after the subsequent smoothing process. There is no limit here.

S203. Determine a first difference Δ between the head posture display data Q1 and the head posture current data Q2'.

The head posture display data Q1 used by the picture of the previous frame is used to check the pre-displayed head posture current data Q2', because studies have shown that when the previous frame picture and the current frame picture are greatly offset, The current picture is drifted, which may cause discomfort to the user. The limit value of the offset is used as a preset threshold, which is not limited herein.

S204. Determine whether the first difference Δ exceeds a preset threshold. If yes, perform smoothing on the head posture current data Q2' to obtain head posture pre-display data Q2, and execute S205.

When the preset threshold is exceeded, it indicates that there is a problem with the current posture Q2' of the head posture used for the pre-display. If it is not directly used, the current data Q2' of the head posture can be processed by the smoothing processing, and the specific operation for the smoothing processing is It should be understood by ordinary technicians that the details are not described.

It should be noted that the head posture current data Q2' is smoothed to obtain the head posture pre-display data Q2, which may include the following methods:

1. The difference between the head posture pre-display data Q2 after the smoothing process and the head posture display data Q1 is made smaller than the first difference value. The difference here refers to the absolute value of the difference between the two head pose data.

2. The smoothing processing of the head posture current data Q2 ′ to obtain the head posture pre-display data Q2 includes: acquiring an average value of the head posture display data Q1 and the head posture current data Q2 ′, The head posture pre-display data Q2 is smoothed to the average value, for example, the head posture pre-display data Q2=(Q1+Q2')/2, that is, the average value calculation.

3. The smoothing adjustment value M is set, and the head posture pre-display data Q2 is smoothed to a value at the adjustment smoothing value M in the same direction as the head posture display data Q1; for example, the head posture pre-display The data Q2 = Q1 ± M, ± indicates a range of positive and negative, where M is a constant, which is a preferred value for statistical analysis based on a large amount of experimental data. The same direction here means that if the latter picture is above the previous picture, it is +M. If the latter picture is below the previous picture, it is -M, that is, the vector of the pre-display data Q2 and the head posture display data Q1. The direction and the vector direction of the head posture current data Q2' and the head posture display data Q1 are on the same side in the horizontal direction. 4. Obtaining a second difference Δ′ between the previous data Q1′ of the head posture displayed on the previous screen and the head posture current data Q2′, setting a smooth adjustment value M, and pre-displaying the head posture data. The Q2 smoothing process is at the sum of the smoothing adjustment value M in the same direction as the head posture display data Q1 and the second difference Δ'. For example, the head posture pre-display data Q2=Q1±M+Δ′, Q1′ is the last data of the head posture displayed on the previous screen, where M is a constant, which is a preference based on statistical analysis of a large amount of experimental data. value. Further, when the head posture pre-display data Q2=Q1±N+(Q2′−Q1′), it is determined whether the second difference Δ′ of (Q2′−Q1′) is greater than the first difference Δ . Here, the previous data Q1' refers to the obtained posture data for displaying the previous screen without performing smoothing processing.

S205. Display the current screen according to the head posture pre-display data Q2.

The head posture pre-display data Q2 is smoothed to display the current picture, and the current intelligent reality picture is optimized in advance to avoid user discomfort caused by the drift of the picture.

S206. Determine whether the first difference Δ exceeds a preset threshold. If not, display the current picture according to the head posture current data Q2'.

After checking that the head pose current data Q2' does not appear to have a large offset, the head pose current data Q2' can be directly used to display the current screen.

An embodiment of the method for performing head posture adjustment by using a first calculation point as a calculation reference, the method includes:

S1: Acquire a detection coordinate of a position where the first calculation point in the current frame is located.

The first calculation point here refers to a reference point selected in order to calculate the relative position of the current frame picture and the next frame picture when performing image display, and the position of the first calculation point can be flexibly selected, and is not limited, specifically selected. After the first calculation point is determined, the detection coordinates in the current frame, that is, the coordinate position of the current first calculation point, are acquired.

S2: Obtain pre-display coordinates of a position where the first calculation point is located in the next frame.

Since the image needs to be adjusted in real time during the movement, the parameters such as the moving direction and speed of the device can be determined according to the collected sensor parameters, and the pre-display coordinates of the first calculated point in the next frame image can be obtained according to the moving direction and the speed parameters. The pre-display coordinate is the parameter actually detected by the sensor. If the parameter displayed by the parameter with excessive deviation is jittery, it is necessary to judge the obtained pre-display coordinates in the subsequent steps to determine whether it can be used.

S3. Calculate a displacement value between the detection coordinate and the pre-display coordinate.

The deviation of the distance is used to determine whether a large offset occurs. First, the detection coordinates of the first calculation point of the current frame are calculated and the pre-display coordinates are calculated to calculate the displacement value between the two.

S4. Determine that the displacement value exceeds a preset distance threshold, and use any point on the line between the detection coordinate and the pre-display coordinate as the actual display coordinate of the first calculation point in the next frame.

The distance threshold is preset. The specific value can be obtained according to the test. It is not limited here. When the calculated displacement value exceeds the preset distance threshold, the pre-display coordinate has a large displacement. Poor, if the next frame image is displayed according to the pre-display coordinates, screen jitter may occur, so a point on the line between the detected coordinates and the pre-display coordinates may be used as the actual display coordinate of the first calculation point in the next frame image. The actual display coordinate is in the direction of the pre-display coordinate. Optionally, the distance between the actual display coordinate and the detected coordinate is within a preset fixed value, and the actual display coordinate is when the pre-display coordinate is located at the lower left of the detected coordinate. Then the actual display coordinates should also be in the direction of the lower left of the detected coordinates. It should be noted that the displacement value between the actual display coordinate position and the detected coordinates in the current frame should not exceed the preset distance threshold to avoid a large occurrence. Deviations will be understood by those of ordinary skill in the art and will not be described.

By adopting the above scheme, the jitter of the picture seen by the human eye can be reduced, and the user's experience is enhanced.

As shown in FIG. 3, in the embodiment of the present invention, a virtual reality-based screen display device is provided, wherein the device includes:

The acquiring unit 301 is configured to acquire the head posture display data Q1 displayed on the previous screen;

The obtaining unit 301 is further configured to acquire a head posture current data Q2' pre-displayed by the current picture;

a determining unit 302, configured to determine a first difference Δ between the head posture display data Q1 and the head posture current data Q2';

The processing unit 303 is configured to determine whether the difference Δ exceeds a preset threshold, and if so, smoothing the head posture current data Q2' to obtain head posture pre-display data Q2;

The display unit 304 is configured to display the current screen according to the head posture pre-display data Q2.

Optionally, the processing unit 303 is further configured to:

Optionally, the processing unit 303 is further configured to make the difference between the smoothed processed head posture pre-display data Q2 and the head posture display data Q1 be smaller than the first difference value Δ.

Optionally, the virtual reality based screen display device is a virtual reality device.

An embodiment of the present invention further provides an apparatus for displaying a virtual reality based screen, including: a processor and a memory, where

a computer readable program is stored in the memory;

The processor is configured to execute virtual reality based by running a program in the memory The method of displaying the screen.

As shown in FIG. 4, in order to implement the foregoing method, an apparatus for displaying a virtual reality based screen is provided in the embodiment of the present invention, including: a processor 401 and a memory 403, where

The computer 401 stores a computer readable program;

The processor 403 is configured to: run the program in the memory, specifically:

Obtaining the head posture display data Q1 of the last screen display;

Obtaining the head data current data Q2' pre-displayed on the current picture;

Determining a first difference Δ between the head posture display data Q1 and the head posture current data Q2';

Determining whether the first difference Δ exceeds a preset threshold, and if so, smoothing the head posture current data Q2' to obtain head posture pre-display data Q2;

The current screen is displayed based on the head posture pre-display data Q2.

FIG. 4 is a schematic diagram of a virtual reality based screen display device according to an embodiment of the present invention. The virtual reality based picture display device 400 includes at least one processor 401, a communication bus 402, a memory 403, and at least one communication interface 404.

The processor 401 can be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present invention.

Communication bus 402 can include a path for communicating information between the components described above. The communication interface 404 uses devices such as any transceiver for communicating with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), and the like.

The memory 403 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type that can store information and instructions. The dynamic storage device can also be an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, and a disc storage device. (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), disk storage media or other magnetic storage devices, or can be used to carry or store desired program code in the form of instructions or data structures and Any other medium that is accessible by the computer, but is not limited to this. The memory can exist independently and be connected to the processor via a bus. The memory can also be integrated with the processor.

The memory 403 is used to store program code for executing the solution of the present invention, and is controlled by the processor 401 for execution. The processor 401 is configured to execute program code stored in the memory 403.

In a specific implementation, as an embodiment, the processor 401 may include one or more CPUs, such as CPU0 and CPU1 in FIG.

In a specific implementation, as an embodiment, the virtual reality based picture display device 400 may include multiple processors, such as the processor 401 and the processor 408 in FIG. Each of these processors can be a single-CPU processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data, such as computer program instructions.

In a specific implementation, as an embodiment, the virtual reality based screen display device 400 may further include an output device 405 and an input device 406. Output device 405 is in communication with processor 401 and can display information in a variety of ways. For example, the output device 405 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector. Wait. Input device 406 is in communication with processor 401 and can accept user input in a variety of ways. For example, input device 406 can be a mouse, keyboard, touch screen device, or sensing device, and the like.

The above-described virtual reality-based screen display device 400 can be a general purpose computer device or a dedicated computer device. In a specific implementation, the virtual reality based screen display device 400 may be a virtual reality device, a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, and a communication device. , embedded devices or devices with similar structures in Figure 4-b. The embodiment of the present invention does not limit the type of the virtual reality based screen display device 400.

A person skilled in the art can clearly understand that for the convenience and brevity of the description, the specific working process of the system, the device and the unit described above can refer to the corresponding process in the foregoing method embodiment, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are only For example, the division of the unit is only a logical function division, and the actual implementation may have another division manner, for example, multiple units or components may be combined or may be integrated into another system, or some features. Can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or unit, and may be in an electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

A person skilled in the art may understand that all or part of the various steps of the foregoing embodiments may be performed by a program to instruct related hardware. The program may be stored in a computer readable storage medium, and the storage medium may include: Read Only Memory (ROM), Random Access Memory (RAM), disk or optical disk.

It will be understood by those skilled in the art that all or part of the steps of implementing the above embodiments may be performed by a program to instruct related hardware, and the program may be stored in a computer readable storage medium, the above mentioned storage. The medium can be a read only memory, a magnetic disk or an optical disk or the like.

The VR-based image display method and related device provided by the present invention are described in detail above. For those skilled in the art, according to the idea of the embodiment of the present invention, there will be a specific implementation manner and application range. The details of the description are not to be construed as limiting the invention.

Claims

A virtual reality based screen display method, characterized in that the method comprises:

Obtaining the head posture display data Q1 of the last screen display;

Obtaining the head data current data Q2' pre-displayed on the current picture;

Determining a first difference Δ between the head posture display data Q1 and the head posture current data Q2';

Determining whether the first difference Δ exceeds a preset threshold, and if so, smoothing the head posture current data Q2' to obtain head posture pre-display data Q2;

The current screen is displayed based on the head posture pre-display data Q2.
The method of claim 1 further comprising:

If the first difference Δ does not exceed the preset threshold, the current picture is displayed according to the head posture current data Q2'.
The method according to claim 1 or 2, wherein the smoothing the head pose current data Q2' to obtain the head pose pre-display data Q2 comprises:

The difference between the head posture pre-display data Q2 after smoothing and the head posture display data Q1 is made smaller than the first difference Δ.
The method according to claim 3, wherein the difference between the smoothed processed head posture pre-display data Q2 and the head posture display data Q1 is smaller than the first difference Δ :

Acquiring an average value of the head posture display data Q1 and the head posture current data Q2', and smoothing the head posture pre-display data Q2 into the average value;

or

Setting a smoothing adjustment value M, and smoothing the head posture pre-display data Q2 to a value at an adjustment smoothing value M in the same direction as the head posture display data Q1;

or

Obtaining a second difference Δ′ between the previous data Q1′ of the head posture of the previous screen display and the head posture current data Q2′, setting a smoothing adjustment value M, and smoothing the head posture pre-display data Q2 The processing is at a sum of the smoothing adjustment value M in the same direction as the head posture display data Q1 and the second difference Δ'.
The method according to claim 4, wherein when the head posture pre-display data Q2 is smoothed into a smooth adjustment value M and the second difference in the same direction as the head posture display data Q1 At the sum of Δ', the head posture pre-display data Q2 is smoothed to a sum of the smoothing adjustment value M and the second difference Δ' that are in the same direction as the head posture display data Q1. Before, the method further comprises determining whether the second difference Δ' value is greater than the first difference Δ, and if it is greater, performing subsequent processing.
A virtual reality based picture display device, characterized in that the device comprises:

An obtaining unit, configured to acquire head posture display data Q1 of the last screen display;

The obtaining unit is further configured to acquire a head posture current data Q2' pre-displayed by the current picture;

a determining unit, configured to determine a first difference Δ between the head posture display data Q1 and the head posture current data Q2';

a processing unit, configured to determine whether the difference Δ exceeds a preset threshold, and if so, smoothing the head posture current data Q2' to obtain head posture pre-display data Q2;

And a display unit, configured to display the current screen according to the head posture pre-display data Q2.
The device according to claim 6, wherein the processing unit is further configured to:

If the first difference Δ does not exceed the preset threshold, the current picture is displayed according to the head posture current data Q2'.
The apparatus according to claim 6 or 7, wherein the processing unit is further configured to make the difference between the smoothed processed head posture pre-display data Q2 and the head posture display data Q1 smaller than The first difference Δ is described.
The apparatus of claim 6 wherein said image processing device is a virtual reality device.
An apparatus for displaying a virtual reality based screen, comprising: a processor and a memory, wherein

a computer readable program is stored in the memory;

The processor is operative to execute the method of the above claims 1 to 5 by running a program in the memory.