WO2017092432A1

WO2017092432A1 - Method, device, and system for virtual reality interaction

Info

Publication number: WO2017092432A1
Application number: PCT/CN2016/096983
Authority: WO
Inventors: 张超
Original assignee: 乐视控股（北京）有限公司; 乐视致新电子科技（天津）有限公司
Priority date: 2015-12-01
Filing date: 2016-08-26
Publication date: 2017-06-08
Also published as: CN105892638A

Abstract

A method, device, and system for virtual reality interaction. The method comprises: capturing at least two first infrared images of a calibration object via a first infrared camera, capturing at least two second infrared images of the calibration object via a second infrared camera (S11), extracting corresponding feature information in the first infrared images and corresponding feature information in the second infrared images of calibration points (S12), utilizing the corresponding feature information in the first infrared images and the corresponding feature information in the second infrared images of the calibration points to determine three-dimensional motion trajectory information of the calibration points, and performing virtual reality interaction via the three-dimensional motion trajectory information of the calibration points (S13). Virtual reality interaction is performed by simultaneously capturing the infrared images of the calibration object via the two infrared cameras, solved is the technical problem that an alternate virtual reality technique being urgently required as implementation is difficult in some scenarios when images are captured by a three-dimensional depth camera for virtual reality interaction.

Description

Virtual reality interaction method, device and system

The present application claims priority to Chinese Patent Application No. 201510870209.8, filed on Dec. 1, 2015, the entire disclosure of which is incorporated herein by reference. In this application.

Technical field

The present application relates to the field of virtual reality technologies, and in particular, to a virtual reality interaction method, apparatus, and system.

Background technique

With the development of society, the progress of various industries has made outstanding contributions to the improvement of quality of life. Among them, the emergence of Virtual Reality (VR) technology has greatly enriched people's lives. Virtual reality technology generates a simulated environment through computer and combines the acquired image information to realize interactive 3D vision and behavior, thus enabling Users are immersed in the simulation environment to realize the interaction between people and the virtual reality environment. In virtual reality technology, an important aspect affecting the interaction effect involves the technology of image information collection.

The prior art generally collects image information through a three-dimensional depth camera, and calculates the distance from a target object such as a collected object or a person by using the basic principle of stereo vision ranging to realize an interactive three-dimensional view, stereoscopic vision ranging. The basic principle is to observe the same object from different viewpoints to obtain the perceptual images at different viewing angles, and calculate the distance information of the target objects by calculating the pixel deviation between the pixels of the image by the principle of triangulation.

Although the prior art can simulate a three-dimensional view and behavior through image information collected by a three-dimensional depth camera, thereby realizing virtual reality interaction. However, the three-dimensional depth camera adopted by the prior art has difficulty in realizing virtual reality technology based on the camera in some scenarios due to its own price, technical maturity, ease of use, and the like, and an urgent need for other virtual reality technologies.

Summary of the invention

The embodiment of the present application provides a virtual reality interaction method, device, and system, which are used to solve the problem of acquiring an image through a three-dimensional depth camera in the prior art, and the virtual reality technology based on the camera is in some scenarios. Difficult to achieve, there is an urgent need for a problem with other virtual reality technologies.

An embodiment of the present application provides a virtual reality interaction method, where the method includes:

Acquiring at least two first infrared images of the calibration by the first infrared camera, and acquiring at least two second infrared images of the calibration by the second infrared camera, the calibration comprising at least one calibration point, the calibration point For providing infrared light;

Extracting corresponding feature information of each of the calibration points in each of the first infrared images and corresponding feature information in each of the second infrared images, wherein the feature information is used to display each of the calibration points in the first a position in an infrared image or a second infrared image;

Determining three-dimensional motion trajectory information of each of the calibration points by using corresponding feature information of each of the calibration points in the first infrared image and corresponding feature information in the second infrared image, and passing each of the labels The fixed three-dimensional motion trajectory information performs virtual reality interaction.

The embodiment of the present application further provides a virtual reality interaction device, where the device includes: a first infrared camera unit, a second infrared camera unit, an extraction unit, a determination unit, and an interaction unit, wherein:

a first infrared imaging unit configured to acquire at least two first infrared images of the calibration object by the first infrared camera, the calibration object comprising at least one calibration point, wherein the calibration point is used to provide infrared light;

a second infrared imaging unit, configured to acquire at least two second infrared images of the calibration object by using a second infrared camera;

An extracting unit, configured to extract corresponding feature information of each of the calibration points in each of the first infrared images and corresponding feature information in each of the second infrared images, where the feature information is used to display each of the targets Positioning in the first infrared image or the second infrared image;

a determining unit, configured to determine three-dimensional motion trajectory information of each of the calibration points by using corresponding feature information of each of the calibration points in the first infrared image and corresponding feature information in the second infrared image;

An interaction unit, configured to perform virtual reality interaction by using three-dimensional motion trajectory information of each of the calibration points.

The embodiment of the present application further provides a virtual reality interaction system, where the system includes: a virtual reality interaction device and a calibration object, wherein:

The virtual reality interaction device includes a first infrared camera unit, a second infrared camera unit, an extraction unit, a determination unit, and an interaction unit, wherein: the first infrared camera unit is configured to pass the first infrared camera Collecting at least two first infrared images of the calibration object; a second infrared imaging unit configured to acquire at least two second infrared images of the calibration object by the second infrared camera; and an extracting unit configured to extract each of the calibration points Corresponding feature information in each of the first infrared images and corresponding feature information in each of the second infrared images, the feature information is used to display each of the calibration points in the first infrared image or the second infrared a position determining unit, configured to determine a three-dimensional shape of each of the calibration points by using corresponding feature information of each of the calibration points in the first infrared image and corresponding feature information in the second infrared image Motion trajectory information; an interaction unit configured to perform virtual reality interaction through three-dimensional motion trajectory information of each of the calibration points.

The calibration object includes at least one calibration point for reflecting infrared light.

The embodiment of the present application provides an electronic device, including the virtual reality interaction device described in any of the foregoing embodiments.

The embodiment of the present application provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium can store a computer instruction, and the virtual reality interaction method provided by the embodiment of the present application can be implemented when the computer instruction is executed Some or all of the steps in each implementation.

An embodiment of the present application provides an electronic device, including: one or more processors; and a memory; wherein the memory stores instructions executable by the one or more processors, the instructions being set to It is used to perform any of the above virtual reality interaction methods of the present application.

An embodiment of the present application provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, The computer is caused to execute any of the above virtual reality interaction methods in the embodiments of the present application.

The virtual reality interaction method, device and system provided by the embodiment of the present application collect the infrared image of the calibration object through the first infrared camera and the second infrared camera, and perform feature information extraction and feature information on the collected infrared image. Analysis, determining the three-dimensional motion trajectory information of each calibration point of the calibration object, thereby performing virtual reality interaction, thereby solving the prior art virtual reality interaction by acquiring images through the three-dimensional depth camera, due to the price and technical maturity of the three-dimensional depth camera itself, The influence of the convenience and the like causes a problem that the virtual reality technology based on the camera is difficult to implement in some scenarios, and provides a new virtual reality technology.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief description of the drawings used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

1 is a flowchart of a virtual reality interaction method according to Embodiment 1 of the present application;

2 is a schematic diagram of a calibration glove in a practical application in Embodiment 1 of the present application;

3 is a flowchart of a virtual reality interaction method according to Embodiment 2 of the present application;

4 is a schematic diagram of a virtual reality interaction device in a practical application in Embodiment 2 of the present application;

FIG. 5 is a schematic structural diagram of a virtual reality interaction apparatus according to Embodiment 3 of the present application;

6 is a schematic structural diagram of a virtual reality interaction system in Embodiment 4 of the present application;

FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present application. Some embodiments are applied, not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.

The technical solutions provided by the embodiments of the present application are described in detail below with reference to the accompanying drawings.

Example 1

Embodiment 1 provides a virtual reality interaction method for solving the problem that the prior art acquires an image through a three-dimensional depth camera, and the virtual reality technology based on the camera is difficult to implement in a specific scene due to the price of the three-dimensional depth camera. The specific flow chart of the method is shown in FIG. 1 and includes the following steps:

Step S11: acquiring at least two first infrared images of the calibration object by the first infrared camera, and acquiring the same number of second infrared images of the calibration object by the second infrared camera.

The first infrared camera and the second infrared camera refer to a camera capable of imaging infrared light, Since the price of the infrared camera in the market is very low compared to the three-dimensional depth camera, the interaction method based on the infrared camera has a relatively low cost, and the infrared light has a large wavelength and a low frequency, so the infrared light is transmitted in the air. When the energy loss is small, it is not easily distorted by infrared light imaging. It should be noted that, in practical applications, the infrared camera may be a camera that adds an ordinary light of an infrared filter between the photosensitive device of the camera and the lens, thereby further reducing the implementation cost of the virtual reality interaction method, especially for To increase the shooting effect of the infrared camera, the infrared filter used can be an 850 nm infrared band pass filter.

In practical applications, the first infrared camera and the second infrared camera are usually installed on the same device, and the device may be a server, a mobile terminal such as a mobile phone, an iPad or a smart helmet, or a terminal such as a smart TV or a computer. The virtual reality interaction method can transmit the collected infrared image to the server, and then perform calculation and simulation of the real environment, and can also perform calculation and simulation of the real environment by terminals such as mobile phones, iPads, smart helmets, smart televisions or computers. The application examples do not limit this.

The calibration object includes at least one calibration point for providing infrared light. The calibration object refers to an object that is simultaneously photographed by the first infrared camera and the second infrared camera. In reality, the object may be a person or an object, and the outer surface of the object has at least a partial area for providing infrared light, which provides infrared light. Part of the area is called the calibration point, and there must be at least one calibration point in the calibration. Especially in practical applications, there are many ways to provide infrared light at the calibration point, including reflecting infrared light and the calibration point itself emitting infrared light. The way to provide infrared light is to install a reflective material on the outer surface of each calibration point of the calibration object to reflect the infrared light emitted by other devices to the calibration object.

The first infrared camera captures at least two first infrared images of the calibration object, and the second infrared camera captures at least two second infrared images of the calibration object, that is, the first infrared camera and the second infrared camera respectively acquire the same N and M infrared images are calibrated, and N and M are both greater than or equal to two. In practical applications, usually the first infrared camera and the second infrared camera collect the same number of infrared images, that is, N and M are equal. When calculating the three-dimensional motion trajectory of the calibration object, at least two infrared images are needed, and the more infrared images are acquired over a period of time, the more accurately the trajectory of the calibration object during this time can be accurately described, but the infrared image is acquired. The more the calculation, the more computationally intensive, so it is usually possible to better describe the trajectory of the calibration during this time by collecting three infrared images of the calibration over a period of time.

Step S12: extract corresponding feature information of each of the calibration points in each of the first infrared images, and And extracting corresponding feature information of each of the calibration points in each of the second infrared images.

The feature information is used to display the position of each of the calibration points in each of the first infrared images or each of the second infrared images.

Since the first infrared camera captures at least two infrared images of the calibration object, and there may be multiple calibration points in the calibration, the manner of extracting the corresponding characteristic information of each of the calibration points in each of the first infrared images is The following operations may be performed for each calibration point: first determining a corresponding area of the calibration point in each of the first infrared images, and then using a clustering algorithm to extract features corresponding to the calibration point in the corresponding area Information; for each of the first infrared images, first calculating feature information of each calibration point in each of the first infrared images, and then determining corresponding features of each calibration point in each of the first infrared images information.

In practical applications, the trajectory prediction algorithm such as Kalman prediction can be used to determine the corresponding region of the same calibration point in each of the first infrared images, and then k-means or k is used in all corresponding regions corresponding to the calibration point. The clustering algorithm such as -mediods extracts the feature information corresponding to the calibration point, and may first calculate each of the first infrared images by using a clustering algorithm such as k-means or k-mediods for each of the first infrared images. The feature information of the fixed point is then determined by a trajectory prediction algorithm such as a Kalman prediction to determine corresponding feature information of the calibration point in each of the first infrared images.

And a method for extracting corresponding feature information of each of the calibration points in each of the second infrared images, and adopting a method of extracting the same feature information corresponding to each of the calibration points in each of the first infrared images.

Step S13: determining the three-dimensionality of each of the calibration points by using corresponding feature information in each of the first infrared images and corresponding feature information of each of the calibration points in each of the second infrared images by using the calibration points. Motion track information, and virtual reality interaction through the three-dimensional motion track information of each of the calibration points.

The three-dimensional motion trajectory information of each of the calibration points refers to information of a motion trajectory of each of the calibration points on the calibration object in a three-dimensional space when the calibration object moves in a three-dimensional space. For example, in a practical application, a calibration glove, as shown in FIG. 2, the calibration points on the calibration glove are respectively 5 fingers, and when the calibration glove is moved in a three-dimensional space, the three-dimensional motion track information of each calibration point is Refers to the information of the trajectory of the calibration point on the five fingers of the calibration glove in three-dimensional space.

Determining three-dimensional motion track information of each of the calibration points by using corresponding feature information of each of the calibration points in each of the first infrared images and corresponding feature information of each of the calibration points in each of the second infrared images . Taking the above-mentioned calibration gloves as an example, each of the calibration points corresponds to each of the first infrared images. The feature information, and the corresponding feature information of the calibration point in each second infrared image, determine the three-dimensional motion track information of the calibration point.

The virtual reality interaction may be performed by using the three-dimensional motion trajectory information of each of the calibration points, and the three-dimensional motion trajectory information of the calibration object may be determined by using the three-dimensional motion trajectory information of each of the calibration points, and the calibration object is determined. Comparing the three-dimensional motion trajectory information with the information in the database, acquiring an interaction instruction corresponding to the three-dimensional motion trajectory information of the calibration object in the database, performing virtual reality interaction through the interaction instruction; The three-dimensional motion trajectory information of the calibration point is compared with the information in the database, and the interaction instruction corresponding to the three-dimensional motion trajectory information of each of the calibration points in the database is acquired, and the virtual reality interaction is performed by the interaction instruction.

Using the virtual reality interaction method provided in Embodiment 1, the infrared image of the calibration object is collected by the first infrared camera and the second infrared camera, and the feature information is extracted and the characteristic information is analyzed by the acquired infrared image to determine the The three-dimensional motion trajectory information of each calibration point of the calibration object is used for virtual reality interaction, thereby solving the prior art virtual reality interaction by acquiring images through the three-dimensional depth camera, due to the price, technical maturity, ease of use, etc. of the three-dimensional depth camera itself The influence of the cause causes the virtual reality technology based on the camera to be difficult to implement in some scenarios, and there is an urgent need for other virtual reality technologies.

Example 2

It is mentioned in step S13 of the first embodiment that the corresponding feature information in each of the first infrared images and the corresponding feature information of each of the calibration points in each of the second infrared images are determined by using the calibration points. The three-dimensional motion trajectory information of each of the calibration points is determined by determining corresponding three-dimensional motion trajectory information of each calibration point by corresponding feature information of each calibration point in each first infrared image and corresponding feature information in each second infrared image. There are many ways. When the lenses of the first infrared camera and the second infrared camera are in the same plane, the feature information corresponding to each of the first infrared images by using the calibration points and each of the calibration points are Determining, in the second infrared image, the three-dimensional motion trajectory information of the calibration object, that is, first determining, when the first infrared image is collected, each calibration point in the infrared image to the first The information about the distance between the center of the lens of the infrared camera and the second infrared camera is determined by the information of at least two of the distances, and the three-dimensional motion track information of each of the calibration points is determined, thus forming Embodiment 2 of the present application, such as Figure 3 is described.

Step S21: collecting at least two first infrared images of the calibration object by the first infrared camera, and acquiring the same number of second infrared images of the calibration object by the second infrared camera, the first infrared camera The lens of the head and the second infrared camera are in the same plane.

Generally, in describing the motion trajectory of the calibration object, in order to make full use of the acquired infrared image, the first infrared camera may be configured to acquire at least two first infrared images of the calibration object, while the second infrared camera acquires the same calibration object. The second infrared image, that is, the first infrared camera and the second infrared camera, simultaneously acquire the same calibration R image, and R is greater than or equal to 2.

As shown in FIG. 4, in a practical application, the first infrared camera and the second infrared camera are usually fixed on the same device, and the lenses of the first infrared camera and the second infrared camera are in the same plane, by adjusting the device. The infrared image of the calibration object is acquired in the direction, and an infrared light emitting device is usually also mounted on the device, and the infrared light emitting device emits infrared light to the calibration object, and the infrared light is reflected by the calibration point in the calibration object.

Step S22: Extract corresponding feature information of each of the calibration points in each of the first infrared images, and extract feature information corresponding to each of the calibration points in each of the second infrared images.

Step S22 is the same as step S12 in Embodiment 1, and will not be described here.

Step S23: Perform the following operations for each calibration point:

Step S231: determining a second infrared image acquired simultaneously with each of the first infrared images;

Step S232: determining, by using the feature information corresponding to the calibration point in the first infrared image, and the corresponding feature information of the calibration point in the second infrared image, when acquiring the first infrared image, Vertical distance information of the line connecting the calibration point to the lens center of the first infrared camera and the second infrared camera;

The vertical distance information of the line connecting the calibration point to the lens center of the first infrared camera and the second infrared camera refers to: the calibration point to the lens center of the first infrared camera and the lens center of the second infrared camera The distance information of the vertical line segment.

In practical applications, generally, the distance between the first infrared camera and the second infrared camera is known, and when the focal lengths of the two cameras are the same and are known, the corresponding feature information in the first infrared image is utilized by the calibration point, and Corresponding feature information in the two infrared images may be obtained by using a similar triangle to obtain vertical distance information of the connection point of the calibration point to the lens center of the first infrared camera and the second infrared camera; Corresponding feature information in the infrared image, and corresponding feature information in the second infrared image and information on the focal lengths of the two cameras, determining the parallax of the two cameras to the calibration point, through the parallax and the first infrared camera and the second infrared camera The distance between the calibration points to the first Vertical distance information of a line connecting the center of the lens of the infrared camera and the second infrared camera.

Step S233: determining, by using at least two pieces of the vertical distance information corresponding to the calibration point, the calibration point three-dimensional motion track information.

Step S234: Perform virtual reality interaction through the three-dimensional motion trajectory information of each of the calibration points.

According to a virtual reality interaction method provided in Embodiment 2, the first infrared camera and the second infrared camera simultaneously acquire the same number of infrared images of the calibration object, and are disposed in the same lens by setting the first infrared camera and the second infrared camera. a plane, so that the vertical distance information of each calibration point of the calibration point to the lens center of the first infrared camera and the second infrared camera can be used to determine the three-dimensional motion trajectory information of each calibration point of the calibration object, thereby making the virtual reality method more Easy to implement.

Finally, it should be understood that those skilled in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a non-transitory computer. In a readable storage medium, the program, when executed, may include the flow of an embodiment of the methods as described above. The non-transitory computer readable storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Example 3

Embodiment 3 provides a virtual reality interaction device for solving the problem that the prior art captures an image through a three-dimensional depth camera, and the virtual reality technology based on the camera is difficult to implement in a specific scene due to the price of the three-dimensional depth camera. A schematic diagram of a specific structure of the apparatus 500 is shown in FIG. 5, and includes the following units: a first infrared camera unit 501, a second infrared camera unit 502, an extraction unit 503, a determination unit 504, and an interaction unit 505, wherein:

The first infrared camera unit 501 is configured to collect at least two first infrared images of the calibration object by using the first infrared camera, the calibration object includes at least one calibration point, and the calibration point is used to provide infrared light;

a second infrared imaging unit 502, configured to acquire at least two second infrared images of the calibration object by using a second infrared camera;

The extracting unit 503 is configured to extract feature information corresponding to each of the first infrared images of the calibration point and corresponding feature information in each of the second infrared images, where the feature information is used to display each of the a position of the calibration point in the first infrared image or the second infrared image;

a determining unit 504, configured to use a corresponding feature letter in each of the first infrared images by using each of the calibration points And corresponding feature information in the second infrared image, determining three-dimensional motion trajectory information of each of the calibration points;

The interaction unit 505 is configured to perform virtual reality interaction by using the three-dimensional motion trajectory information of each of the calibration points.

In an actual application, the extracting unit 503 may further include a first extracting subunit 5031 and a second extracting subunit 5032, wherein:

The first extraction subunit 5031 is configured to determine, for each calibration point, a corresponding area of the calibration point in each of the first infrared images or each of the second infrared images;

The second extraction sub-unit 5032 is configured to use the clustering algorithm to extract feature information corresponding to the calibration point in the corresponding area.

Specifically, the interaction unit 505 may further include a first interaction unit 5051, a second interaction unit 5052, and a third interaction unit 5053, where:

a first interaction unit 5051, configured to determine three-dimensional motion trajectory information of the calibration object by using three-dimensional motion trajectory information of each of the calibration points;

The second interaction unit 5052 is configured to compare the three-dimensional motion trajectory information of the calibration object with the information in the database, and acquire an interaction instruction corresponding to the three-dimensional motion trajectory information of the calibration object in the database;

The third interaction unit 5053 is configured to perform virtual reality interaction by using the interaction instruction.

Using the virtual reality interaction device provided in Embodiment 3, the first infrared camera unit and the second infrared camera unit collect at least two infrared images of the same calibration object through the infrared camera, and then extract the unit to extract each calibration point in each infrared image. Corresponding feature information, the determining unit determines the three-dimensional motion trajectory information of each calibration point by using the feature information corresponding to each calibration point, and the interaction unit performs virtual reality interaction based on the three-dimensional motion trajectory information of each calibration point. Therefore, the prior art solves the virtual reality interaction by acquiring images through the three-dimensional depth camera. Due to the influence of the price of the three-dimensional depth camera and the like, the virtual reality technology based on the camera is difficult to realize in a specific scene.

In addition, it should be noted that the foregoing related functional modules may be implemented by a hardware processor in the embodiment of the present application.

Example 4

Embodiment 4 provides a virtual reality interaction system for solving the problem that the prior art acquires an image through a three-dimensional depth camera, and the virtual reality technology based on the camera is difficult to implement in a specific scene due to the price of the three-dimensional depth camera itself. FIG. 6 is a schematic diagram of a specific structure of the virtual reality interaction system 600, including: a virtual reality interaction device 601 and a calibration object 602, wherein:

The virtual reality interaction device 601 includes: a first infrared imaging unit, a second infrared imaging unit, an extraction unit, a determining unit, and an interaction unit, wherein: the first infrared imaging unit is configured to collect at least the calibration object by the first infrared camera. Two first infrared images, the calibration object includes at least one calibration point, the calibration point is used to provide infrared light, and the second infrared imaging unit is configured to collect at least two second objects of the calibration object by the second infrared camera An infrared image; an extracting unit, configured to extract corresponding feature information of each of the calibration points in each of the first infrared images and corresponding feature information in each of the second infrared images, wherein the feature information is used to display each a position of the calibration point in the first infrared image or the second infrared image; a determining unit, configured to utilize corresponding feature information of each of the calibration points in the first infrared image and in the second infrared Corresponding feature information in the image, determining three-dimensional motion track information of each of the calibration points; and an interaction unit for performing three-dimensional transport through each of the calibration points Track information virtual reality interaction.

The calibration 602 includes at least one calibration point for reflecting infrared light.

A virtual reality interactive system in practical applications, including a virtual reality interactive helmet and calibration gloves. The virtual reality interactive helmet has a dual infrared camera for collecting infrared images of the calibration gloves, and can usually be installed on the virtual reality interactive helmet. Infrared light emitting device, the five fingers of the calibration glove have materials capable of reflecting infrared light, and the dual infrared camera on the virtual reality interactive helmet can transmit the collected infrared image to the remote server for processing after collecting the infrared image. A processing device can also be installed on the virtual reality interactive helmet for processing.

The virtual reality interaction system provided in Embodiment 4 is configured, wherein the virtual reality interaction device collects an infrared image of the calibration object through the infrared camera in the first infrared imaging unit and the second infrared imaging unit, and collects the collected infrared The image performs a series of processing for virtual reality interaction. Therefore, the prior art solves the virtual reality interaction by acquiring images through the three-dimensional depth camera. Due to the influence of the price of the three-dimensional depth camera and the like, the virtual reality technology based on the camera is difficult to realize in a specific scene.

In another embodiment of the present application, an electronic device is provided, including the foregoing any one of the embodiments. Virtual reality interactive device.

In another embodiment of the present application, a non-transitory computer readable storage medium is also provided, the non-transitory computer readable storage medium storing computer executable instructions executable by any of the above methods The virtual reality interaction method in the example.

FIG. 7 is a schematic diagram of a hardware structure of an electronic device for performing a virtual reality interaction method according to an embodiment of the present disclosure. As shown in FIG. 7, the device includes:

One or more processors 710 and memory 720, one processor 710 is taken as an example in FIG.

The apparatus that performs the virtual reality interaction method may further include: an input device 730 and an output device 740.

The processor 710, the memory 720, the input device 730, and the output device 740 may be connected by a bus or other means, as exemplified by a bus connection in FIG.

The memory 720 is used as a non-transitory computer readable storage medium, and can be used for storing a non-volatile software program, a non-volatile computer executable program, and a module, such as a program instruction corresponding to the virtual reality interaction method in the embodiment of the present application. / Module (for example, the first infrared camera unit 501, the second infrared camera unit 502, the extraction unit 503, the determination unit 504, and the interaction unit 505 shown in FIG. 5). The processor 710 executes various functional applications and data processing of the electronic device by executing non-volatile software programs, instructions, and modules stored in the memory 720, that is, implementing the virtual reality interaction method of the above method embodiment.

The memory 720 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to use of the virtual reality interactive device, and the like. Moreover, memory 720 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 720 can optionally include memory remotely located relative to processor 710, which can be connected to the virtual reality interactive device over a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Input device 730 can receive input numeric or character information and generate key signal inputs related to user settings and function control of the virtual reality interaction device. The output device 740 can include a display device such as a display screen.

The one or more modules are stored in the memory 720, and when executed by the one or more processors 710, perform a virtual reality interaction method in any of the above method embodiments.

The above product can execute the method provided by the embodiment of the present application, and has a corresponding functional module for executing the method. And beneficial effects. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiments of the present application.

The electronic device of the embodiment of the present application exists in various forms, including but not limited to:

(1) Mobile communication devices: These devices are characterized by mobile communication functions and are mainly aimed at providing voice and data communication. Such terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

(2) Ultra-mobile personal computer equipment: This type of equipment belongs to the category of personal computers, has computing and processing functions, and generally has mobile Internet access. Such terminals include: PDAs, MIDs, and UMPC devices, such as the iPad.

(3) Portable entertainment devices: These devices can display and play multimedia content. Such devices include: audio, video players (such as iPod), handheld game consoles, e-books, and smart toys and portable car navigation devices.

(4) Server: A device that provides computing services. The server consists of a processor, a hard disk, a memory, a system bus, etc. The server is similar to a general-purpose computer architecture, but because of the need to provide highly reliable services, processing power and stability High reliability in terms of reliability, security, scalability, and manageability.

(5) Other electronic devices with data interaction functions.

The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. Those of ordinary skill in the art can understand and implement without deliberate labor.

Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on such understanding, the above-described technical solutions may be embodied in the form of software products in essence or in the form of software products, which may be stored in a computer readable storage medium such as ROM/RAM, magnetic Discs, optical discs, and the like, including a number of signals, are used to cause a computer device (which may be a personal computer, server, or network device, etc.) to perform the methods described in various embodiments or portions of the embodiments.

Finally, it should be noted that the above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still The technical solutions described in the foregoing embodiments are modified, or the equivalents of the technical features are replaced by the equivalents. The modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

A virtual reality interaction method, which is characterized in that it is applied to an electronic device, including:

Acquiring at least two first infrared images of the calibration by the first infrared camera, and acquiring at least two second infrared images of the calibration by the second infrared camera, the calibration comprising at least one calibration point, the calibration point For providing infrared light;

Extracting corresponding feature information of each of the calibration points in each of the first infrared images and corresponding feature information in each of the second infrared images, wherein the feature information is used to display each of the calibration points in the first a position in an infrared image or a second infrared image;

Determining three-dimensional motion trajectory information of each of the calibration points by using corresponding feature information of each of the calibration points in the first infrared image and corresponding feature information in the second infrared image, and passing each of the labels The fixed three-dimensional motion trajectory information performs virtual reality interaction.
The method according to claim 1, wherein the extracting the corresponding feature information of each of the calibration points in each of the first infrared images and the corresponding feature information in each of the second infrared images is specifically :

Do the following for each calibration point:

Determining a corresponding area of the calibration point in each of the first infrared image or each of the second infrared images;

A clustering algorithm is used in the corresponding area to extract feature information corresponding to the calibration point.
The method according to claim 1, wherein at least two first infrared images of the calibration object are acquired by the first infrared camera, and at least two second infrared images are collected by the second infrared camera. : acquiring at least two first infrared images of the calibration object by the first infrared camera while acquiring the same number of second infrared images of the calibration object by the second infrared camera, and the first infrared camera and the second infrared image The lens of the camera is in the same plane;

And determining, by using the feature information corresponding to each of the first infrared images in each of the first infrared images and the corresponding feature information of each of the calibration points in each of the second infrared images, determining the three-dimensionality of the calibration object The motion track information is specifically:

Do the following for each calibration point:

Determining a second infrared image acquired simultaneously with each of the first infrared images;

Determining, by using the corresponding feature information of the calibration point in the first infrared image, and corresponding feature information of the calibration point in the second infrared image, when the first infrared image is acquired, the target Vertical distance information that is fixed to a line connecting the lens centers of the first infrared camera and the second infrared camera;

The calibration point three-dimensional motion trajectory information is determined by at least two of the vertical distance information corresponding to the calibration point.
The method according to claim 1, wherein the virtual reality interaction by the three-dimensional motion trajectory information of each of the calibration points is specifically:

Determining three-dimensional motion trajectory information of the calibration object by three-dimensional motion trajectory information of each of the calibration points;

Comparing the three-dimensional motion trajectory information of the calibration object with the information in the database, and acquiring an interaction instruction corresponding to the three-dimensional motion trajectory information of the calibration object in the database;

The virtual reality interaction is performed by the interactive instruction.
The method of claim 1 wherein said first infrared camera and said second infrared camera are specifically cameras having an infrared filter between the photosensitive device and the lens.
The method of claim 1 further comprising: emitting infrared light to said calibrator, said calibrator reflecting infrared light through a calibrated point on the calibrator.
A virtual reality interaction device, the device comprising:

a first infrared camera unit, a second infrared camera unit, an extraction unit, a determination unit, and an interaction unit, wherein:

a first infrared imaging unit configured to acquire at least two first infrared images of the calibration object by the first infrared camera, the calibration object comprising at least one calibration point, wherein the calibration point is used to provide infrared light;

a second infrared imaging unit, configured to acquire at least two second infrared images of the calibration object by using a second infrared camera;

An extracting unit, configured to extract corresponding feature information of each of the calibration points in each of the first infrared images and corresponding feature information in each of the second infrared images, where the feature information is used for display a position of each of the calibration points in the first infrared image or the second infrared image;

a determining unit, configured to determine three-dimensional motion trajectory information of each of the calibration points by using corresponding feature information of each of the calibration points in the first infrared image and corresponding feature information in the second infrared image;

An interaction unit, configured to perform virtual reality interaction by using three-dimensional motion trajectory information of each of the calibration points.
The apparatus according to claim 7, wherein said extracting unit comprises a first extracting subunit and a second extracting subunit, wherein:

The first extraction subunit is configured to determine, for each calibration point, a corresponding area of the calibration point in each of the first infrared images or each of the second infrared images;

The second extraction subunit is configured to extract, by using a clustering algorithm, feature information corresponding to the calibration point in the corresponding area.
The apparatus according to claim 7, wherein the interaction unit comprises a first interaction unit, a second interaction unit, and a third interaction unit, wherein:

a first interaction unit, configured to determine three-dimensional motion trajectory information of the calibration object by using three-dimensional motion trajectory information of each of the calibration points;

a second interaction unit, configured to compare the three-dimensional motion track information of the calibration object with the information in the database, and acquire an interaction instruction corresponding to the three-dimensional motion track information of the calibration object in the database;

The third interaction unit is configured to perform virtual reality interaction by using the interaction instruction.
A virtual reality interaction system, characterized in that the system comprises: a virtual reality interaction device and a calibration object, wherein:

The virtual reality interaction device includes a first infrared imaging unit, a second infrared imaging unit, an extraction unit, a determining unit, and an interaction unit, wherein: the first infrared imaging unit is configured to collect at least two of the calibration objects by using the first infrared camera. a first infrared image; a second infrared camera unit, configured to acquire at least two second infrared images of the calibration object by using a second infrared camera; and an extracting unit, configured to extract each of the calibration points in each of the first infrared images Corresponding feature information and corresponding feature information in each of the second infrared images, the feature information is used to display a position of each of the calibration points in the first infrared image or the second infrared image; For the use of each of the calibrations Determining three-dimensional motion trajectory information of each of the calibration points by corresponding feature information in the first infrared image and corresponding feature information in the second infrared image; and an interaction unit for passing each of the calibration points 3D motion trajectory information for virtual reality interaction;

The calibration object includes at least one calibration point for reflecting infrared light.
A non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer instructions for causing the computer to perform the method of any of claims 1-6 .
An electronic device, comprising:

One or more processors; and,

a memory communicatively coupled to the one or more processors; wherein

The memory stores instructions executable by the one or more processors, the instructions being executed by the one or more processors to enable the one or more processors to:

Acquiring at least two first infrared images of the calibration by the first infrared camera, and acquiring at least two second infrared images of the calibration by the second infrared camera, the calibration comprising at least one calibration point, the calibration point For providing infrared light;

Extracting corresponding feature information of each of the calibration points in each of the first infrared images and corresponding feature information in each of the second infrared images, wherein the feature information is used to display each of the calibration points in the first a position in an infrared image or a second infrared image;

Determining three-dimensional motion trajectory information of each of the calibration points by using corresponding feature information of each of the calibration points in the first infrared image and corresponding feature information in the second infrared image, and passing each of the labels The fixed three-dimensional motion trajectory information performs virtual reality interaction.
A computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to execute The method of claims 1-6.