WO2021072702A1

WO2021072702A1 - Augmented reality scene implementation method, apparatus, device, and storage medium

Info

Publication number: WO2021072702A1
Application number: PCT/CN2019/111692
Authority: WO
Inventors: 邓生全; 吴少滨
Original assignee: 深圳盈天下视觉科技有限公司
Priority date: 2019-10-17
Filing date: 2019-10-17
Publication date: 2021-04-22
Also published as: CN110914873A; CN110914873B

Abstract

An augmented reality method, comprising: acquiring a first image, collected by a camera of mixed reality glasses, in a real scenario (S101); determining, on the basis of first real coordinates of the mixed reality glasses, first virtual coordinates of the mixed reality glasses and second real coordinates of a target object, second virtual coordinates of the target object in a preview picture of the camera (S102); and determining, on the basis of a first mapping relationship, second attribute information matching first attribute information, and superposing the first attribute information and the second attribute information in a target region (S103). The method realizes reality augmentation by means of combining real coordinates with virtual coordinates of a target object, without it being necessary to calculate the position and the angle of an image in real time, such that an augmented image has a higher stability.

Description

Augmented reality method, device, mixed reality glasses and storage medium

Technical field

This application belongs to the field of computer technology, and in particular relates to an augmented reality method, device, mixed reality glasses, and storage medium.

Background technique

Augmented Reality (AR) is a technology that calculates the position and angle of the camera image in real time and adds the corresponding image. The goal of this technology is to put the virtual world on the screen in the process of interacting with the real world. Since AR technology needs to calculate the position and angle of the camera image in real time, the image after enhancement is unstable.

Summary of the invention

technical problem

One of the objectives of the embodiments of the present application is to provide an augmented reality method, device, hybrid display glasses, and storage medium, aiming to solve the problem of image instability after enhancement.

The solution to the problem

Technical solutions

In order to solve the above technical problems, the embodiments of the present application provide an augmented reality method, device, augmented reality glasses, and storage medium to solve the problem that the existing augmented reality technology needs to calculate the position and angle of the camera image in real time, resulting in an enhanced image Instability issues.

The first aspect of the embodiments of the present application provides an augmented reality method, including:

Obtain a first image in a real scene collected by the camera of the mixed reality glasses, the first image including the first attribute information of the target object, and the difference between the first attribute information of the target object and the second attribute information of the object to be superimposed There is a first mapping relationship, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses;

Based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object, determine the first position of the target object in the display area of the augmented reality glasses Two virtual coordinates;

Wherein, the first real coordinates and the second real coordinates are both coordinates relative to the origin of the first coordinates, and the first virtual coordinates and the second virtual coordinates are both coordinates relative to the origin of the second coordinates; The first coordinate origin is the coordinate origin of the first image, and the second coordinate origin is the coordinate origin of the display area of the augmented reality glasses;

Based on the first mapping relationship, the second attribute information that matches the first attribute information is determined, and the first attribute information and the second attribute information are superimposed in a target area, and the target area Is the area corresponding to the second virtual coordinate in the display area of the augmented reality glasses.

In an optional implementation manner, the target object is determined based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinates in the display area of the augmented reality glasses include:

Acquiring first reality coordinates of the mixed reality glasses relative to the origin of the first coordinates, and acquiring first virtual coordinates of the mixed reality glasses relative to the origin of the second coordinates;

Determining a first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate;

According to the first coordinate conversion relational expression, the second real coordinates are converted into the second virtual coordinates to obtain the second virtual coordinates of the target object in the display area of the augmented reality glasses.

In an optional implementation manner, the first virtual coordinate of the mixed reality glasses is the origin of the second coordinate;

The determining the first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate includes:

The first real coordinate is mapped to the second coordinate origin to obtain the first coordinate conversion relational expression between the first coordinate origin and the second coordinate origin.

In an optional implementation manner, based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object, it is determined that the target object is in the place. The second virtual coordinates in the display area of the augmented reality glasses include:

Determining a second coordinate conversion relationship between the first real coordinate and the first virtual coordinate according to a preset coordinate calibration rule;

According to the second coordinate conversion relational expression, the second virtual coordinate of the target object in the display area of the augmented reality glasses is determined.

In an optional implementation manner, both the augmented reality glasses and the target object have a GPS positioning system, the first reality coordinates are the first GPS coordinates of the augmented reality glasses, and the second reality coordinates Is the second GPS coordinate of the target object.

In an optional implementation manner, the target object is determined based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinates in the camera preview image include:

Acquiring the first GPS coordinates of the augmented reality glasses and the second GPS coordinates of the target object;

Mapping the first GPS coordinates to a virtual coordinate origin;

Calculating a coordinate conversion relationship between the first GPS coordinate and the second GPS coordinate;

Based on the origin of the virtual coordinates and the coordinate conversion relationship between the first GPS coordinates and the second GPS coordinates, the second virtual coordinates of the target object are determined.

The second aspect of the embodiments of the present application provides an augmented reality device, including:

The acquisition module is configured to acquire a first image in a real scene collected by a camera of the mixed reality glasses, the first image including the first attribute information of the target object, the first attribute information of the target object and the first image of the object to be superimposed There is a first mapping relationship between the two attribute information, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses;

The determining module is configured to determine the position of the target object in the augmented reality glasses based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinate in the display area;

An overlay module, configured to determine the second attribute information that matches the first attribute information based on the first mapping relationship, and superimpose the first attribute information and the second attribute information in a target area The target area is an area corresponding to the second virtual coordinate in the display area of the augmented reality glasses.

In an optional implementation manner, the determining module includes:

A first acquiring unit, configured to acquire first reality coordinates of the mixed reality glasses relative to the origin of the first coordinates, and acquire first virtual coordinates of the mixed reality glasses relative to the origin of the second coordinates;

A first determining unit, configured to determine a first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate;

The conversion unit is configured to convert the second real coordinates into the second virtual coordinates according to the first coordinate conversion relational expression to obtain the first position of the target object in the display area of the augmented reality glasses Two virtual coordinates.

The first determining unit includes:

The mapping subunit is configured to map the first real coordinate to the second coordinate origin to obtain the first coordinate conversion relational expression between the first coordinate origin and the second coordinate origin.

In an optional implementation manner, the determining module includes:

A second determining unit, configured to determine a second coordinate conversion relational expression between the first real coordinate and the first virtual coordinate according to a preset coordinate calibration rule;

The third determining unit is configured to determine the second virtual coordinates of the target object in the display area of the augmented reality glasses according to the second coordinate conversion relational expression.

In an optional implementation manner, the determining module includes:

A second acquiring unit, configured to acquire the first GPS coordinates of the augmented reality glasses and the second GPS coordinates of the target object;

A mapping unit, configured to map the first GPS coordinates to a virtual coordinate origin;

A calculation unit, configured to calculate a coordinate conversion relationship between the first GPS coordinate and the second GPS coordinate;

The fourth determining unit is configured to determine the second virtual coordinates of the target object based on the origin of the virtual coordinates and the coordinate conversion relationship between the first GPS coordinates and the second GPS coordinates.

The third aspect of the embodiments of the present application provides an augmented reality glasses, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor executes the computer program When realizing the augmented reality method as described in the embodiment provided in the first aspect above.

The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned embodiment provided in the first aspect is implemented. Augmented reality method.

Compared with the prior art, the augmented reality method provided by the first aspect of the present application acquires a first image in a real scene collected by a camera of a mixed reality glasses. The first image includes the first attribute information of the target object. There is a first mapping relationship between the first attribute information of the target object and the second attribute information of the object to be superimposed, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses; based on the mixed reality glasses The first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object to determine the second virtual coordinates of the target object in the display area of the augmented reality glasses; In the first mapping relationship, the second attribute information that matches the first attribute information is determined, and the first attribute information and the second attribute information are superimposed in a target area, where the target area is all The area corresponding to the second virtual coordinate in the display area of the augmented reality glasses. Because based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object, the second reality of the target object in the display area of the augmented reality glasses is determined. Virtual coordinates; and superimpose the matched first attribute information and second attribute information in a target area, where the target area is the area corresponding to the second virtual coordinates in the display area of the augmented reality glasses, which realizes Combining the real coordinates of the target object with the virtual coordinates for reality augmentation does not require real-time calculation of the position and angle of the image, so that the enhanced image has better stability.

Compared with the prior art, the embodiments provided in the second to fourth aspects of the present application have the same beneficial effects as the embodiments provided in the first aspect of the present application. Go into details.

The beneficial effects of the invention

Brief description of the drawings

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only of the present application. For some embodiments, those of ordinary skill in the art can obtain other drawings based on these drawings without creative labor.

Fig. 1 is an implementation flowchart of the augmented reality method provided by the first embodiment of the present application;

Figure 2 is a first specific implementation flow chart of S102 in Figure 1;

Figure 3 is a second specific implementation flow chart of S102 in Figure 1;

Figure 4 is a third specific implementation flow chart of S102 in Figure 1;

Figure 5 is a specific implementation flow chart of S405 in Figure 4;

FIG. 6 is a schematic diagram of the augmented reality device provided by the present application;

Fig. 7 is a schematic diagram of augmented reality glasses provided by an embodiment of the present application.

Invention embodiment

Embodiments of the present invention

In the following description, for the purpose of illustration rather than limitation, specific details such as a specific system structure and technology are proposed for a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obstructing the description of this application.

It should be noted that Augmented Reality (AR) technology is a technology that ingeniously integrates virtual information with the real world, and widely uses multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, sensing, etc. Technical means, after computer-generated text, images, three-dimensional models, music, video and other virtual information are simulated and applied to the real world, the two kinds of information complement each other, so as to realize the "enhancement" of the real world.

At present, the common augmented reality technology requires real-time calculation of the position and angle of the camera image, and the combination of real coordinates and virtual coordinates is not performed, resulting in unstable images after enhancement.

In order to solve the foregoing technical problems, embodiments of the present application provide an augmented reality method, device, augmented reality glasses, and storage medium.

In order to illustrate the technical solution described in the present application, specific embodiments are used for description below. As shown in FIG. 1, it is an implementation flowchart of the augmented reality method provided by the first embodiment of the present application. The execution subject of this embodiment is augmented reality glasses, which can be implemented by software or hardware in the augmented reality glasses. The details are as follows:

S101: Acquire a first image in a real scene collected by a camera of the mixed reality glasses. Wherein, the first image includes first attribute information of the target object, there is a first mapping relationship between the first attribute information of the target object and the second attribute information of the object to be superimposed, and the object to be superimposed is a preview image Virtual objects in.

Among them, the mixed reality glasses are glasses that can complete the mixed reality technology. The user wearing the mixed reality glasses in the embodiment of this application can observe the presentation of virtual scene information in the real scene, which connects the real world, the virtual world and the user. An information loop of interactive feedback enhances the realism of the user experience.

In an optional implementation manner, the mixed reality glasses include a camera, the camera is used to collect a first image in a real scene, and the first image includes a target object that needs to be augmented. Generally, the target object is Persons and objects with fixed first attribute information; wherein the first attribute information is the feature information of the target object in the application scene. For example, assuming that the target object is a player at the game scene, the first attribute information of the target object An attribute information includes the position of the player at the game scene, the color of the team uniform, and so on. Corresponding to the aforementioned game application scenario, the object to be superimposed is auxiliary information of the game scene, such as auxiliary information such as the stadium and the number of players, and the second attribute information of the auxiliary information matches the first attribute information of the player. For example, if the auxiliary information is a stadium, the second attribute information of the auxiliary information includes the center position of the stadium and the position of the boundary point of the stadium that match the position of the player. The augmented reality technology can assist the game scene. The information is superimposed and displayed with the first image where the player is, so as to realize the enhanced display of the real scene and the virtual scene.

S102: Determine the second position of the target object in the camera preview image based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object. Virtual coordinates.

Wherein, the first real coordinates and the second real coordinates are both coordinates relative to the origin of the first coordinates, and the first virtual coordinates and the second virtual coordinates are both coordinates relative to the origin of the second coordinates; The first coordinate origin is the coordinate origin of the first image, and the second coordinate origin is the coordinate origin of the display area of the augmented reality glasses.

It should be noted that the display area of the augmented reality glasses is a picture displayed in the preset display area for displaying information about virtual objects (objects to be superimposed). In the embodiment of the present application, the reality is obtained through the camera of the augmented reality glasses. In the first image of the scene, the preview screen of the augmented reality glasses displays the object to be superimposed. Based on the coordinate relationship between the target object in the first image and the object to be superimposed in the preview screen, it can be observed through the screen of the augmented reality glasses. The object to be superimposed is superimposed on the target object to achieve the effect of augmented reality.

In an optional implementation manner, as shown in FIG. 2, it is the first specific implementation flowchart of S102 in FIG. 1. As can be seen from FIG. 2, S102 includes:

S1021: Acquire first reality coordinates of the mixed reality glasses relative to the origin of the first coordinates, and acquire first virtual coordinates of the mixed reality glasses relative to the origin of the second coordinates.

Wherein, the first reality coordinates are based on the real scene, and the first virtual coordinates are based on the screen of the augmented reality glasses. If the position relationship between the real scene and the screen of the augmented reality glasses can be obtained, it can be determined that the The display area (the screen of the augmented reality glasses) displays the object to be superimposed that needs to be superimposed, so that when the real scene information is observed through the augmented reality glasses, the fusion of the object to be superimposed and the target object can be realized.

S1022: Determine a first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate.

Understandably, the coordinates are all determined by the corresponding coordinate origin, and therefore, the conversion relationship between the coordinates can also be considered as the conversion relationship between the coordinate origins. In this embodiment, the first coordinate conversion relationship between the first real coordinate and the first virtual coordinate is determined by determining the coordinate relationship between the first coordinate origin and the second coordinate origin.

In an optional implementation manner, S1022 includes:

Wherein, mapping the first real coordinate to the second coordinate origin refers to a process of initializing the second coordinate origin so that the first real coordinate and the second coordinate origin coincide.

S1023: According to the first coordinate conversion relational expression, convert the second real coordinates into the second virtual coordinates to obtain the second virtual coordinates of the target object in the display area of the augmented reality glasses .

In an optional implementation manner, the first virtual coordinate of the mixed reality glasses is the origin of the second coordinate.

It is understandable that the embodiment shown in FIG. 2 only shows a way of determining the second virtual coordinates of the target object in the camera preview image. In different embodiments, different ways can be selected for determination.

For example, as shown in FIG. 3, it is the second specific implementation flowchart of S102 in FIG. 1. As can be seen from FIG. 3, in this embodiment, S102 includes:

S1024: Determine a second coordinate conversion relationship between the first real coordinate and the first virtual coordinate according to a preset coordinate calibration rule.

Among them, the preset coordinate calibration law is to place the preset calibration map at a certain distance from the front of the augmented reality glasses, so that the camera of the augmented reality glasses can also collect the calibration when collecting the first image in the real scene. Figure, the calibration diagram is marked with calibration points.

S1025: Determine the second virtual coordinates of the target object in the camera preview image according to the second coordinate conversion relational expression.

As shown in FIG. 4, it is the third specific implementation flowchart of S102 in FIG. 1. As can be seen from FIG. 4, in this embodiment, S102 includes:

S401: Acquire the first GPS coordinates of the augmented reality glasses and the second GPS coordinates of the target object.

Wherein, the first GPS coordinates of the augmented reality glasses are obtained by positioning by the GPS positioning system of the augmented reality glasses, and the second GPS coordinates of the target object are obtained by positioning by the GPS positioning system of the target object.

S402: Map the first GPS coordinates to a virtual coordinate origin.

Wherein, after acquiring the first GPS coordinates, the virtual coordinate origin is initialized. In this embodiment, the first GPS coordinates are mapped to the initialized virtual coordinate origin, and the mapping relationship between the GPS coordinates and the virtual coordinates is established. Understandably, usually during the use of the augmented reality glasses, after the origin of the virtual coordinates is initialized, the origin of the virtual coordinates is a fixed position and no longer changes.

S403: Calculate a coordinate conversion relationship between the first GPS coordinate and the second GPS coordinate.

Understandably, the first GPS coordinates and the second GPS coordinates are respectively position coordinates in the GPS coordinate system, which represent the position of the object in the GPS coordinate system, and the first GPS coordinates and the second GPS coordinates are calculated by calculating the first GPS coordinates and the second GPS coordinates. Second, the coordinate conversion relationship between GPS coordinates can determine the position relationship between the target object and the augmented reality setting.

S404: Determine the second virtual coordinate of the target object based on the coordinate conversion relationship between the virtual coordinate origin and the first GPS coordinate and the second GPS coordinate.

S103. Determine the second attribute information that matches the first attribute information based on the first mapping relationship, and superimpose the first attribute information and the second attribute information in a target area, and The target area is an area corresponding to the second virtual coordinate in the display area of the augmented reality glasses.

The coordinate conversion relationship between the object to be superimposed in the camera preview screen and the target object in the real scene obtained by the above method, for any point in the area where the augmented reality can be superimposed in the display area of the augmented reality glasses, the obtained coordinate conversion The relational expression can obtain the coordinates of the corresponding point of the point in the real scene, and then perform the second attribute information of the object to be superimposed and the first attribute information of the target object at the coordinates of the corresponding point in the display area of the augmented reality glasses. Superimposition can realize the fusion of virtual and real scenes.

It can be seen from the above analysis that the augmented reality method provided in this embodiment acquires the first image in the real scene collected by the camera of the mixed reality glasses. The first image includes the first attribute information of the target object. There is a first mapping relationship between the first attribute information and the second attribute information of the object to be superimposed, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses; based on the first reality of the mixed reality glasses Coordinates, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object to determine the second virtual coordinates of the target object in the display area of the augmented reality glasses; based on the first mapping Relationship, determine the second attribute information that matches the first attribute information, and superimpose the first attribute information and the second attribute information in a target area, where the target area is the second virtual The coordinate corresponds to the area in the display area of the augmented reality glasses. Determining the second virtual coordinates of the target object in the camera preview screen based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object; And superimpose the matched first attribute information and second attribute information in a target area, where the target area is the area corresponding to the second virtual coordinate in the display area of the augmented reality glasses, which realizes the target object Real-time coordinates and virtual coordinates are combined for reality augmentation, without real-time calculation of the position and angle of the image, so that the enhanced image has better stability.

Corresponding to the augmented reality method provided in the first embodiment of the present application, as shown in FIG. 6, is a schematic diagram of the augmented reality device provided in the present application.

It can be seen from FIG. 6 that the augmented reality device 6 provided by the embodiment of the present application includes:

The acquiring module 601 is configured to acquire a first image in a real scene collected by the camera of the mixed reality glasses, the first image including the first attribute information of the target object, the first attribute information of the target object and the object to be superimposed There is a first mapping relationship between the second attribute information, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses;

The determining module 602 is configured to determine that the target object is in the augmented reality glasses based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object. The second virtual coordinate in the display area of;

The overlay module 603 is configured to determine, based on the first mapping relationship, the second attribute information that matches the first attribute information, and compare the first attribute information with the second attribute information in the target area Superimposed, the target area is an area corresponding to the second virtual coordinate in the display area of the augmented reality glasses.

In an optional implementation manner, the determining module 602 includes:

The first determining unit includes:

The mapping subunit is configured to map the first actual coordinate to the second coordinate origin to obtain the first coordinate conversion relational expression between the first coordinate origin and the second coordinate origin.

In an optional implementation manner, the determining module 602 includes:

In an optional implementation manner, both the augmented reality glasses and the target object have a GPS positioning system, the first reality coordinates are the first GPS coordinates of the augmented reality glasses, and the second reality coordinates are Is the second GPS coordinate of the target object.

In an optional implementation manner, the determining module includes:

As shown in FIG. 7, it is a schematic diagram of augmented reality glasses provided by an embodiment of the present application. It can be seen from FIG. 7 that the augmented reality glasses 7 of this embodiment includes a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and running on the processor 70, such as an augmented reality program. When the processor 70 executes the computer program 72, the steps in the foregoing embodiments of the augmented reality method are implemented, for example, steps 101 to 103 shown in FIG. 1. Alternatively, when the processor 70 executes the computer program 72, the function of each module/unit in the above-mentioned augmented reality device embodiment is realized, for example, the functions of the modules 601 to 603 shown in FIG. 6.

Exemplarily, the computer program 72 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 71 and executed by the processor 70 to complete This application. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 72 in the augmented reality glasses 7. For example, the computer program 72 may be divided into an acquisition module, a determination module, and an overlay module (a module in a virtual device), and the specific functions of each module are as follows:

A determining module, configured to determine that the target object is in the camera preview image based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinate;

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the division of the above functional units and modules is used as an example. In practical applications, the above functions can be allocated to different functional units and modules as needed. Module completion, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist alone physically, or two or more units can be integrated into one unit. The above-mentioned integrated units can be hardware-based Formal realization can also be realized in the form of software functional units. In addition, the specific names of the functional units and modules are only used to facilitate distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the foregoing system, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail or recorded in an embodiment, reference may be made to related descriptions of other embodiments.

A person of ordinary skill in the art may realize that the units and algorithm steps of the examples described in the embodiments disclosed in this document can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the embodiments provided in this application, it should be understood that the disclosed device/terminal device and method may be implemented in other ways. For example, the device/terminal device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division, and there may be other divisions in actual implementation, such as multiple units. Or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple communication units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, it can implement the steps of the foregoing method embodiments. . Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (Read-Only Memory, ROM) , Random Access Memory (RAM), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted according to the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, a person of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions recorded in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application, and should be included in Within the scope of protection of this application.

Claims

An augmented reality method, characterized in that it includes:

Obtain a first image in a real scene collected by the camera of the mixed reality glasses, the first image including the first attribute information of the target object, and the difference between the first attribute information of the target object and the second attribute information of the object to be superimposed There is a first mapping relationship, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses;

Based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object, the second virtual coordinates of the target object in the camera preview screen are determined ；

Wherein, the first real coordinates and the second real coordinates are both coordinates relative to the origin of the first coordinates, and the first virtual coordinates and the second virtual coordinates are both coordinates relative to the origin of the second coordinates; The first coordinate origin is the coordinate origin of the first image, and the second coordinate origin is the coordinate origin of the display area of the augmented reality glasses;

Based on the first mapping relationship, the second attribute information that matches the first attribute information is determined, and the first image and the second attribute information are superimposed and displayed in the target area to obtain the enhanced image In the second image, the target area is an area corresponding to the second virtual coordinate in the display area of the augmented reality glasses.
8. The augmented reality method of claim 1, wherein the first reality coordinates based on the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object are based on, Determining the second virtual coordinates of the target object in the display area of the augmented reality glasses includes:

Acquiring first reality coordinates of the mixed reality glasses relative to the origin of the first coordinates, and acquiring first virtual coordinates of the mixed reality glasses relative to the origin of the second coordinates;

Determining a first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate;

According to the first coordinate conversion relational expression, the second real coordinates are converted into the second virtual coordinates to obtain the second virtual coordinates of the target object in the display area of the augmented reality glasses.
3. The augmented reality method of claim 2, wherein the first virtual coordinate of the mixed reality glasses is the origin of the second coordinate;

The determining the first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate includes:

The first real coordinate is mapped to the second coordinate origin to obtain the first coordinate conversion relational expression between the first coordinate origin and the second coordinate origin.
The augmented reality method according to claim 1, wherein the determination is based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object. The second virtual coordinates of the target object in the display area of the augmented reality glasses include:

Determining a second coordinate conversion relationship between the first real coordinate and the first virtual coordinate according to a preset coordinate calibration rule;

According to the second coordinate conversion relational expression, the second virtual coordinate of the target object in the display area of the augmented reality glasses is determined.
The augmented reality method of claim 4, wherein the augmented reality glasses and the target object both have a GPS positioning system, and the first reality coordinates are the first GPS coordinates of the augmented reality glasses, so The second actual coordinate is the second GPS coordinate of the target object.
8. The augmented reality method of claim 5, wherein the first reality coordinates based on the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object are based on, Determining the second virtual coordinates of the target object in the camera preview image includes:

Acquiring the first GPS coordinates of the augmented reality glasses and the second GPS coordinates of the target object;

Mapping the first GPS coordinates to a virtual coordinate origin;

Calculating a coordinate conversion relationship between the first GPS coordinate and the second GPS coordinate;

Based on the origin of the virtual coordinates and the coordinate conversion relationship between the first GPS coordinates and the second GPS coordinates, the second virtual coordinates of the target object are determined.
An augmented reality device, characterized in that it comprises:

The acquisition module is configured to acquire a first image in a real scene collected by a camera of the mixed reality glasses, the first image including the first attribute information of the target object, the first attribute information of the target object and the first image of the object to be superimposed There is a first mapping relationship between the two attribute information, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses;

A determining module, configured to determine that the target object is in the camera preview image based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinate;

An overlay module, configured to determine the second attribute information that matches the first attribute information based on the first mapping relationship, and superimpose the first attribute information and the second attribute information in a target area The target area is an area corresponding to the second virtual coordinate in the display area of the augmented reality glasses.
8. The augmented reality device of claim 7, wherein the determining module comprises:

A first acquiring unit, configured to acquire first reality coordinates of the mixed reality glasses relative to the origin of the first coordinates, and acquire first virtual coordinates of the mixed reality glasses relative to the origin of the second coordinates;

A first determining unit, configured to determine a first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate;

The conversion unit is configured to convert the second real coordinates into the second virtual coordinates according to the first coordinate conversion relational expression to obtain the first position of the target object in the display area of the augmented reality glasses Two virtual coordinates.
A mixed reality glasses comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the following steps when the processor executes the computer program:

Obtain a first image in a real scene collected by the camera of the mixed reality glasses, the first image including the first attribute information of the target object, and the difference between the first attribute information of the target object and the second attribute information of the object to be superimposed There is a first mapping relationship, and the object to be superimposed is a virtual object in the display area of the augmented reality glasses;

Based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object, the second virtual coordinates of the target object in the camera preview screen are determined ；

Wherein, the first real coordinates and the second real coordinates are both coordinates relative to the origin of the first coordinates, and the first virtual coordinates and the second virtual coordinates are both coordinates relative to the origin of the second coordinates; The first coordinate origin is the coordinate origin of the first image, and the second coordinate origin is the coordinate origin of the display area of the augmented reality glasses;

Based on the first mapping relationship, the second attribute information that matches the first attribute information is determined, and the first image and the second attribute information are superimposed and displayed in the target area to obtain the enhanced image In the second image, the target area is an area corresponding to the second virtual coordinate in the display area of the augmented reality glasses.
The calculation setting according to claim 9, wherein the determining is based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinates of the target object in the display area of the augmented reality glasses include:

Acquiring first reality coordinates of the mixed reality glasses relative to the origin of the first coordinates, and acquiring first virtual coordinates of the mixed reality glasses relative to the origin of the second coordinates;

Determining a first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate;

According to the first coordinate conversion relational expression, the second real coordinates are converted into the second virtual coordinates to obtain the second virtual coordinates of the target object in the display area of the augmented reality glasses.
The computing device of claim 10, wherein the first virtual coordinate of the mixed reality glasses is the origin of the second coordinate;

The determining the first coordinate conversion relational expression between the first real coordinate and the first virtual coordinate includes:

The first real coordinate is mapped to the second coordinate origin to obtain the first coordinate conversion relational expression between the first coordinate origin and the second coordinate origin.
The computing device according to claim 9, wherein the determination of the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinates of the target object in the display area of the augmented reality glasses include:

Determining a second coordinate conversion relationship between the first real coordinate and the first virtual coordinate according to a preset coordinate calibration rule;

According to the second coordinate conversion relational expression, the second virtual coordinate of the target object in the display area of the augmented reality glasses is determined.
The computing device of claim 12, wherein the augmented reality glasses and the target object both have a GPS positioning system, the first reality coordinates are the first GPS coordinates of the augmented reality glasses, and the The second actual coordinate is the second GPS coordinate of the target object.
The computing device according to claim 13, wherein the determining is based on the first reality coordinates of the mixed reality glasses, the first virtual coordinates of the mixed reality glasses, and the second reality coordinates of the target object The second virtual coordinates of the target object in the camera preview image include:

Acquiring the first GPS coordinates of the augmented reality glasses and the second GPS coordinates of the target object;

Mapping the first GPS coordinates to a virtual coordinate origin;

Calculating a coordinate conversion relationship between the first GPS coordinate and the second GPS coordinate;

Based on the origin of the virtual coordinates and the coordinate conversion relationship between the first GPS coordinates and the second GPS coordinates, the second virtual coordinates of the target object are determined.
A computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the steps of the augmented reality method according to any one of claims 1 to 7 .