CN105005970A - Augmented reality implementation method and apparatus - Google Patents

Abstract

The invention discloses a method and device for realizing augmented reality. The method includes: acquiring field depth information of a target object in the real world; establishing a three-dimensional coordinate system for the real space according to the field depth information, and obtaining the three-dimensional image of the target object. image; fitting the virtual image with the obtained stereoscopic image of the target object to obtain a new three-dimensional fitting image; obtaining a two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system conversion, and converting the The 2D fitted image is displayed. The present invention can realize the interaction between the target object in the real world and the virtual object on the screen of the camera terminal, so as to solve a series of problems such as occlusion, illumination and shadow.

Description

A method and device for implementing augmented reality

technical field

The invention relates to the field of communication technology, in particular to a method and device for realizing augmented reality.

Background technique

Augmented Reality (Augmented Reality, AR) is a new computer application and human-computer interaction technology developed on the basis of virtual reality technology. It uses objects in the real world as target objects. Enhanced information related to the target object, such as advertising information, distance information or other virtual objects related to the target object, and then superimposing the target object and the enhanced information on the same screen with the help of computer and visualization technology, to achieve augmented reality.

Since the target object captured by the camera terminal is two-dimensional information and lacks three-dimensional information in the real world, the current AR technology can only establish a two-dimensional plane in the real space. If the enhanced information is a virtual object, the virtual object can only be simply The ground moves in a two-dimensional plane space and cannot interact with three-dimensional target objects in the real world. Phenomena such as occlusion, lighting, and shadows cannot be realized because there is no three-dimensional information in the real world.

Contents of the invention

In view of this, the embodiments of the present invention provide a method and device for implementing augmented reality. On the screen of the camera terminal, the interaction between the target object in the real world and the virtual object is realized to solve a series of problems such as occlusion, illumination, and shadow. question.

In the first aspect, the embodiment of the present invention provides a method for implementing augmented reality, including:

Obtain the depth information of the target object in the real world;

Establishing a three-dimensional coordinate system for the real space according to the depth of field information, and acquiring a three-dimensional image of the target object;

Fitting the virtual image with the acquired stereoscopic image of the target object to obtain a new three-dimensional fitting image;

A two-dimensional fitting image corresponding to the new three-dimensional fitting image is acquired through coordinate system conversion, and the two-dimensional fitting image is displayed.

In the second aspect, the embodiment of the present invention also provides a device for implementing augmented reality, including:

A depth information acquisition unit, configured to acquire depth information of a target object in the real world;

A stereoscopic image acquisition unit, configured to establish a stereoscopic coordinate system for the real space according to the depth of field information, and acquire a stereoscopic image of the target object;

a three-dimensional fitting image acquisition unit, configured to fit the virtual image with the acquired stereoscopic image of the target object, and obtain a new three-dimensional fitting image;

The two-dimensional fitting image display unit is configured to acquire a two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system conversion, and display the two-dimensional fitting image.

Embodiments of the present invention provide a method and device for augmented reality. According to the depth information of the target object in the real world, the method establishes a three-dimensional coordinate system for the real world to obtain a three-dimensional image of the target object; The image is fitted to obtain a new three-dimensional fitting image, and finally the three-dimensional fitting image is converted into a two-dimensional fitting image through coordinate system conversion and displayed. The embodiment of the present invention realizes the interaction between the target object in the real world and the virtual object on the screen of the camera terminal, so as to solve a series of problems such as occlusion, illumination, and shadow.

Description of drawings

Other characteristics, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

FIG. 1(a) is a flow chart of an augmented reality implementation method provided by Embodiment 1 of the present invention;

Fig. 1(b) is a schematic diagram of the state of a target object in real space provided by Embodiment 1 of the present invention;

FIG. 1(c) is a perspective view of a target object in real space acquired by the camera terminal provided in Embodiment 1 of the present invention;

Fig. 1(d) is a perspective view of the camera terminal provided by the first embodiment of the present invention to fit the target object in the real space with the virtual object;

Fig. 1(e) is a schematic diagram of a two-dimensional fitting image of a target object in real space and a virtual object displayed on the screen of the camera terminal provided by Embodiment 1 of the present invention;

FIG. 2 is a flowchart of an augmented reality implementation method provided by Embodiment 2 of the present invention;

FIG. 3 is a flowchart of an augmented reality implementation method provided by Embodiment 3 of the present invention;

Fig. 4 is a structural block diagram of an augmented reality realization device provided by Embodiment 4 of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, only parts related to the present invention are shown in the drawings but not all content.

Embodiment one

Fig. 1(a) is a flowchart of an augmented reality implementation method provided by Embodiment 1 of the present invention. The method is performed by an augmented reality implementation device, the device is implemented by means of software and/or hardware, and the device is set in the camera terminal, as shown in Figure 1(a), the method includes :

Step 101: Obtain the depth information of the target object in the real world.

Wherein, the depth of field information is the distance from each point in the target object to the camera terminal, and the depth of field information may be collected by a camera of the camera terminal.

Step 102: Establish a stereoscopic coordinate system for the real space according to the depth of field information, and acquire a stereoscopic image of the target object.

In this embodiment, taking the camera terminal as the origin, the camera terminal establishes a three-dimensional coordinate system for the real space, and obtains a three-dimensional image of the target object through the distance from each point of the target object to the camera terminal, wherein the three-dimensional coordinate system can be three-dimensional Coordinate System. For example, as shown in Figure 1(b), in the real space, there are three target objects on the ground, namely a book, a square box, and tape. The book is placed under the box, and the camera terminal acquires the three targets The depth of field information of the object is used to establish a three-dimensional coordinate system to obtain the three-dimensional images of these objects (as shown in Figure 1(c)).

Step 103: Fitting the virtual image with the acquired stereoscopic image of the target object to obtain a new three-dimensional fitting image;

In this embodiment, in the camera terminal, the virtual image is fitted to the stereoscopic image of the target object to obtain a new fitted image. Following the example in step 102, as shown in Figure 1(d), in the camera terminal, the avatar is fitted with the book, the box and the tape, wherein the first avatar 1 is inside the tape core, and the second avatar is The virtual character 2 stands on the book, the third virtual character 3 stands on the box, the fourth virtual character 4 stands behind the box, and the fifth virtual character 5 stands on the ground. It can be seen from Figure 1(d) It is found that in the new fitting image, due to the establishment of a three-dimensional coordinate system inside the camera terminal, books, square boxes and tapes are three-dimensional images. When the first virtual character 1 is inside the tape core, the tape core blocks the second A body part of a virtual character 1 . In addition, the square box also blocks a part of the body of the fourth virtual character 4 .

Step 104: Obtain a 2D fitting image corresponding to the new 3D fitting image through coordinate system conversion, and display the 2D fitting image.

In this embodiment, the camera terminal converts the three-dimensional coordinate system into a two-dimensional coordinate system, converts the three-dimensional fitting image into a two-dimensional fitting image, and displays the two-dimensional fitting image on the screen of the camera terminal, as As shown in Figure 1(e), the image displayed on the camera terminal screen is a two-dimensional fitting image, which realizes the interaction between the virtual character and the target object in the real world.

This embodiment provides a method for implementing augmented reality. According to the depth information of the target object in the real world, a three-dimensional coordinate system is established for the real world, and a stereoscopic image of the target object is obtained; the virtual image is fitted with the stereoscopic image of the target object. , to obtain a new 3D fitting image, and finally convert the 3D fitting image into a 2D fitting image through coordinate system transformation, and display it. The embodiment of the present invention realizes the interaction between the target object in the real world and the virtual object on the screen of the camera terminal, so as to solve a series of problems such as occlusion, illumination, and shadow.

Embodiment two

Fig. 2 is a flow chart of an augmented reality implementation method provided by the second embodiment; on the basis of the above embodiment, before obtaining the depth information of the target object in the real world, the following operation is added: obtain the target in the real world The image color information of the object. The above operations provide a basis for displaying the color image of the target object on the screen of the camera terminal.

Further, on the basis of the above embodiments, the two-dimensional fitting image corresponding to the new three-dimensional fitting image is acquired through coordinate system conversion, and the two-dimensional fitting image is displayed, including:

Obtain the two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system conversion, and perform color configuration for the target object in the two-dimensional fitting image according to the obtained image color information of the target object, and configure the color configuration for the target object in the two-dimensional fitting image. The 2D fitted image is displayed.

Based on the above optimization, the solution provided in this embodiment is specifically as follows:

Step 201: Obtain image color information of a target object in the real world.

The image color information may be the RGB value of each point of the target object.

Step 202: Obtain the depth information of the target object in the real world.

Step 203: According to the depth information, establish a three-dimensional coordinate system for the real space, and acquire a three-dimensional image of the target object.

Step 204: Obtain the 2D fitting image corresponding to the new 3D fitting image through coordinate system conversion, and color the target object in the 2D fitting image according to the acquired image color information of the target object Configuration, display the 2D fitting image after color configuration.

Specifically, the acquired RGB value of each point of the target object in the real world is fixed. In the camera terminal, according to the RGB value of each point of the target object, the color configuration of the target object in the two-dimensional fitting image is performed, and the camera terminal displayed in color on the screen.

The second embodiment provides a method for implementing augmented reality, which can display virtual images and target objects in the real world in color to increase the image quality of the camera.

Embodiment three

Fig. 3 is a flowchart of a method for implementing augmented reality provided by this embodiment; on the basis of the above embodiments, this embodiment "fits the virtual image with the acquired stereoscopic image of the target object, and acquires New 3D Fitting Image" step was optimized.

In the above solution, when the camera terminal is still, the fitting of the virtual image to the acquired stereoscopic image of the target object to obtain a new three-dimensional fitting image includes: inserting the virtual image at a preset coordinate position, Fitting the virtual image and the image of the target object into a new image is used as a new three-dimensional fitting image.

Specifically, in the camera terminal, there is a three-dimensional coordinate system established for the real space. Users can set the coordinate position for the virtual image according to their own needs. When adding a virtual image, the virtual image will be inserted at the preset coordinate position to realize virtual Fitting of image and target object.

On the basis of the above embodiments, when the camera terminal moves, this embodiment "displays the two-dimensional fitting image corresponding to the new three-dimensional fitting image acquired through coordinate system transformation" Optimized.

Further, based on the above optimization, when the camera terminal moves, the technical solution provided by this embodiment is specifically as follows:

Step 301: Obtain image color information of a target object in the real world.

Step 302: Obtain the depth information of the target object in the real world.

Step 303: Establish a stereo coordinate system for the real space according to the depth information, and acquire a stereo image of the target object.

Step 304: Fitting the virtual image and the acquired stereoscopic image of the target object to obtain a new three-dimensional fitting image.

Step 305: Obtain the location coordinates of the camera terminal in real time.

Wherein, the position coordinates of the camera terminal can be acquired through sensors such as GPS, acceleration, and gyroscope arranged on the camera terminal.

Step 306: According to the position coordinates of the camera terminal, calculate the position coordinates of the virtual image and the target object in the new three-dimensional fitting image through the six-axis coordinate system, and obtain the fitting image of the target object and the virtual image after the camera terminal moves.

Specifically, when the camera terminal moves, a six-axis coordinate system is established inside the camera terminal, and the position of each point of the virtual image and the target object is represented by six-axis coordinates. The position coordinates of the points have all changed compared with before, and the obtained target object and the fitting image of the virtual image after the camera terminal moves have all changed compared with those before the camera terminal moved. The camera terminal uses a six-axis coordinate system to determine the position coordinates of each point on the virtual image and the target object, which can clearly reflect the different occlusions, lighting, and shadows presented in different observation directions.

Step 307: Convert the acquired fitting image of the target object after the camera terminal moves and the virtual image into a three-dimensional fitting image.

In this embodiment, the obtained fitting image of the target object after the camera terminal moves and the virtual image is an image represented by six-axis coordinates, which is converted into a three-dimensional fitting image.

Step 308: Convert the converted 3D fitting image into a corresponding 2D fitting image, and display it.

This embodiment provides a method for implementing augmented reality. When the camera terminal is stationary, the method performs the task of "fitting the virtual image with the acquired stereoscopic image of the target object to obtain a new three-dimensional fitting image". The steps are optimized so that the user can set the position of the avatar according to their own needs. When the camera terminal moves, the method "acquires the two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system conversion" The step of "displaying the two-dimensional fitting image" has been optimized, which can present shooting pictures from different angles, and present different occlusions, lighting, shadows and other situations under different shooting angles.

Embodiment four

FIG. 4 is a structural block diagram of an augmented reality implementation device provided in Embodiment 4. As shown in FIG. 4, the device includes:

A depth information acquisition unit 401, configured to acquire depth information of a target object in the real world;

Stereoscopic image acquisition unit 402, configured to establish a stereoscopic coordinate system for the real space according to the depth of field information, and obtain a stereoscopic image of the target object; wherein, the depth of field information is the distance from each point in the target object to the camera terminal ;

A three-dimensional fitting image acquisition unit 403, configured to fit the virtual image with the acquired stereoscopic image of the target object, and obtain a new three-dimensional fitting image;

The two-dimensional fitting image display unit 404 is configured to acquire a two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system transformation, and display the two-dimensional fitting image.

On the basis of the foregoing embodiments, the device further includes:

The image color acquiring unit 405 is configured to acquire the image color information of the target object in the real world before acquiring the depth information of the target object in the real world.

The two-dimensional fitting image display unit 404 is specifically configured to obtain a two-dimensional fitting image corresponding to a new three-dimensional fitting image through coordinate system conversion, and to obtain a two-dimensional fitting image according to the obtained image color information of the target object. The target object in the image is color-configured, and the two-dimensional fitting image after the color-configuration is displayed.

On the basis of the foregoing embodiments, when the camera terminal is stationary;

The three-dimensional fitting image acquisition unit 403 is specifically configured to: insert the virtual image at a preset coordinate position, so that the virtual image and the image of the target object are fitted into a new image.

On the basis of the above embodiments, when the camera terminal moves; the two-dimensional fitting image display unit 404 is specifically used to:

Obtain the location coordinates of the camera terminal in real time;

According to the position coordinates of the camera terminal, calculate the position coordinates of the virtual image and the target object in the new three-dimensional fitting image through the six-axis coordinate system, and obtain the fitting image of the target object and the virtual image after the camera terminal moves;

converting the obtained fitting image of the target object and the virtual image after the camera terminal moves into a three-dimensional fitting image;

The converted 3D fitting image is converted into a corresponding 2D fitting image and displayed.

This embodiment provides a device for implementing augmented reality. The device establishes a three-dimensional coordinate system for the real world according to the depth information of the target object in the real world, and obtains a stereoscopic image of the target object; Fitting, to obtain a new 3D fitting image, and finally convert the 3D fitting image into a 2D fitting image through coordinate system conversion, and display it. The embodiment of the present invention realizes the interaction between the target object in the real world and the virtual object in the display screen of the camera terminal, so as to solve a series of problems such as occlusion, illumination, and shadow.

The device can be used to implement the augmented reality realization method of the present invention, and has corresponding functions and beneficial effects.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention, and the present invention The scope is determined by the scope of the appended claims.

Claims (10)

1. A method for implementing augmented reality, comprising: Obtain the depth information of the target object in the real world; Establishing a three-dimensional coordinate system for the real space according to the depth of field information, and acquiring a three-dimensional image of the target object; Fitting the virtual image with the acquired stereoscopic image of the target object to obtain a new three-dimensional fitting image; A two-dimensional fitting image corresponding to the new three-dimensional fitting image is acquired through coordinate system conversion, and the two-dimensional fitting image is displayed. 2. The method according to claim 1, characterized in that, before acquiring the depth information of the target object in the real world, further comprising, acquiring image color information of the target object in the real world; Obtaining the two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system conversion, and displaying the two-dimensional fitting image, including: Obtain a two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system conversion, and perform color configuration for the target object in the two-dimensional fitting image according to the acquired image color information of the target object, and configure the color The final two-dimensional fitting image is displayed. 3. The method according to claim 1, wherein the depth information is the distance from each point in the target object to the camera terminal. 4. The method according to claim 1, characterized in that, when the camera terminal is stationary, The fitting of the virtual image with the acquired stereoscopic image of the target object, and acquiring a new three-dimensional fitting image includes: inserting the virtual image at a preset coordinate position, fitting the virtual image and the image of the target object into a new The image of , as the new 3D fitting image. 5. The method according to claim 1, wherein when the camera terminal moves, The step of obtaining a two-dimensional fitting image corresponding to a new three-dimensional fitting image through coordinate system conversion, and displaying the two-dimensional fitting image includes: Obtain the location coordinates of the camera terminal in real time; According to the position coordinates of the camera terminal, calculate the position coordinates of the virtual image and the target object in the new three-dimensional fitting image through the six-axis coordinate system, and obtain the fitting image of the target object and the virtual image after the camera terminal moves; converting the obtained fitting image of the target object and the virtual image after the camera terminal moves into a three-dimensional fitting image; The converted 3D fitting image is converted into a corresponding 2D fitting image and displayed. 6. A device for realizing augmented reality, comprising: A depth information acquisition unit, configured to acquire depth information of a target object in the real world; A stereoscopic image acquisition unit, configured to establish a stereoscopic coordinate system for the real space according to the depth of field information, and acquire a stereoscopic image of the target object; A three-dimensional fitting image acquisition unit, configured to fit the virtual image with the acquired stereoscopic image of the target object to obtain a new three-dimensional fitting image; The two-dimensional fitting image display unit is configured to acquire a two-dimensional fitting image corresponding to the new three-dimensional fitting image through coordinate system conversion, and display the two-dimensional fitting image. 7. The device according to claim 6, further comprising, The image color acquisition unit is used to acquire the image color information of the target object in the real world before acquiring the depth information of the target object in the real world; The two-dimensional fitting image display unit is specifically used to obtain a two-dimensional fitting image corresponding to a new three-dimensional fitting image through coordinate system conversion, and obtain a two-dimensional fitting image according to the acquired image color information of the target object The color configuration of the target object is performed, and the two-dimensional fitting image after the color matching is displayed. 8. The device according to claim 6, wherein the depth information is the distance from each point in the target object to the camera terminal. 9. The device according to claim 6, wherein when the camera terminal is stationary; The three-dimensional fitting image acquisition unit is specifically used for: inserting the virtual image at a preset coordinate position, so that the virtual image and the image of the target object are fitted into a new image as a new three-dimensional fitting image. 10. The device according to claim 6, wherein when the camera terminal moves, The two-dimensional fitting image display unit is specifically used for, Obtain the location coordinates of the camera terminal in real time; According to the position coordinates of the camera terminal, calculate the position coordinates of the virtual image and the target object in the new three-dimensional fitting image through the six-axis coordinate system, and obtain the fitting image of the target object and the virtual image after the camera terminal moves; converting the obtained fitting image of the target object and the virtual image after the camera terminal moves into a three-dimensional fitting image; The converted 3D fitting image is converted into a corresponding 2D fitting image and displayed.