WO2020019962A1

WO2020019962A1 - Coordinate system calibration method and device for augmented reality device

Info

Publication number: WO2020019962A1
Application number: PCT/CN2019/094835
Authority: WO
Inventors: 周轶霄
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2018-07-27
Filing date: 2019-07-05
Publication date: 2020-01-30
Also published as: CN109272454B; TW202008305A; CN109272454A; TWI712004B

Abstract

Provided by the present description is a coordinate system calibration method and device for an augmented reality device. The method comprises: scanning a preset pattern using an augmented reality device to obtain image information of the preset pattern in the augmented reality device; acquiring image coordinate information of a reference point in the image information within an image coordinate system of the augmented reality device and real coordinate information of the reference point in a real coordinate system; obtaining the real coordinate system, the image coordinate system, and a conversion relationship between the device coordinate systems of the augmented reality device on the basis of the image coordinate information and the real coordinate information; and calibrating the device coordinate system according to the conversion relationship. By employing each embodiment in the present description, an augmented reality coordinate system may be calibrated without restricting the configuration of augmented reality devices to a same position and a same orientation; the operation is convenient, the the method is highly practical, and the requirement of calibrating the coordinate system of the AR devices of users at different positions is met.

Description

Method and device for calibrating coordinate system of augmented reality equipment

Technical field

This specification belongs to the field of augmented reality technology, and particularly relates to a coordinate system calibration method and device of an augmented reality device.

Background technique

With the development of science and technology, AR (Augmented Reality) technology is increasingly used. AR technology applies virtual information to the real world, and real environments and virtual objects are superimposed on the same screen or space in real time. At the same time, people can use AR equipment to experience the combination of virtual and reality.

However, the AR coordinate system in the AR device is usually related to the posture and position of the AR device during initialization. Different AR devices may have different positions and postures during initialization, and the AR coordinate system may be different. Different users cannot interact in the same AR space. Therefore, AR applications are usually stand-alone applications. In the prior art, for a method of calibrating a coordinate system of an AR device, generally, each device is placed at the same position, the same posture is maintained, and then the device is initialized. It can be seen that this method is relatively limited and inconvenient to operate, and it is not possible to calibrate the AR coordinate system of users at different positions. Therefore, there is an urgent need for an implementation that can facilitate the calibration of multiple AR device coordinate systems.

Summary of the Invention

The purpose of this specification is to provide a coordinate system calibration method and device for an augmented reality device. The method is simple, and the calibration of the AR coordinate system of different AR devices is achieved.

On the one hand, embodiments of the present specification provide a coordinate system calibration method for an augmented reality device, including:

Scanning the preset pattern using an augmented reality device to obtain image information of the preset pattern in the augmented reality device;

Acquiring image coordinate information of a reference point in the image information in an image coordinate system of the enhancement device, and real coordinate information of the reference point in a real coordinate system;

Determining a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device based on the image coordinate information and the real coordinate information;

The device coordinate system is calibrated according to the conversion relationship.

Further, in another embodiment of the method, a method for acquiring real coordinate information of the reference point in a real coordinate system includes:

Determining a calibration point in the image information;

Determine the real coordinate information of the reference point in the real coordinate system according to the relative position between the reference point and the calibration point in the preset pattern.

Further, in another embodiment of the method, the determining a calibration point in the image information includes:

Selecting one of the reference points as the calibration point;

Correspondingly, determining the real coordinate information of the reference point in the real coordinate system includes:

Using the calibration point as a coordinate origin, selecting a straight line where a line connecting two reference points outside the calibration point and the calibration point is located as a coordinate axis, and constructing the true coordinate system;

Further, in another embodiment of the method, the determining a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device includes:

Acquiring a device transformation matrix for converting the device coordinate system of the augmented reality device into the image coordinate system;

Calculating the real transformation matrix transformed from the real coordinate system into the image coordinate system according to the image coordinate information, the real coordinate information, and the camera internal reference matrix of the augmented real device;

Accordingly, the method for calibrating the device coordinate system according to the conversion relationship includes:

The device coordinate system is calibrated according to the device conversion matrix and the real conversion matrix.

Further, in another embodiment of the method, the calibrating the device coordinate system according to the device conversion matrix and the real conversion matrix includes:

The device coordinate system of the first augmented reality device and the second augmented reality device is calibrated according to the following formula:

P _{cameraA_real} ^-1 P _{cameraA_AR} O _ARA = P _{cameraB_real} ^-1 P _{cameraB_AR} O _ARB

In the above formula, P _{cameraA_real} represents a real transformation matrix corresponding to the first augmented reality device, P _{cameraA_AR} represents a device transformation matrix corresponding to the first augmented reality device, and O _ARA represents a point O on the first augmented reality device. Coordinates in the device coordinate system, P _{cameraB_real} represents the real transformation matrix corresponding to the second augmented reality device, P _{cameraB_AR} represents the device transformation matrix corresponding to the second augmented reality device, and O _ARB represents the point O at the second augmentation Coordinates in the device coordinate system of the real device.

Further, in another embodiment of the method, the method for calibrating the device coordinate system according to the conversion relationship further includes:

If there are more than two augmented reality devices for device coordinate system calibration, the device coordinate system is calibrated based on the device coordinate system of any one of the augmented reality devices.

Further, in another embodiment of the method, the calculation of the real coordinate system into the image coordinates is performed according to the image coordinate information, the real coordinate information, and a camera internal parameter matrix of the augmented real device. The system's true transformation matrix, including:

According to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented real device, the SolvePnP algorithm is used to calculate and obtain the real transformation matrix.

Further, in another embodiment of the method, the preset pattern includes at least one of a two-dimensional code and a barcode.

On the other hand, this specification provides a coordinate system calibration device for an augmented reality device, including:

An image information acquisition module, configured to scan a preset pattern using an augmented reality device to acquire image information of the preset pattern in the augmented reality device;

A coordinate information acquisition module, configured to acquire image coordinate information of a reference point in the image information in an image coordinate system of the enhancement device, and real coordinate information of the reference point in a real coordinate system;

A conversion relationship determining module, configured to obtain a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device based on the image coordinate information and the real coordinate information;

The device coordinate system calibration module is configured to calibrate the device coordinate system according to the conversion relationship.

Further, in another embodiment of the apparatus, the coordinate information acquisition module includes a real coordinate acquisition unit,

The real coordinate obtaining unit is specifically configured to:

Determining a calibration point in the image information;

Further, in another embodiment of the apparatus, the real coordinate obtaining unit is further configured to:

Selecting one of the reference points as the calibration point;

Further, in another embodiment of the apparatus, the conversion relationship determination module includes:

A device transformation matrix calculation unit, configured to obtain a device transformation matrix for converting the device coordinate system of the augmented reality device into the image coordinate system;

A real transformation matrix calculation unit, configured to calculate a real transformation matrix transformed from the real coordinate system into the image coordinate system according to the image coordinate information, the real coordinate information, and a camera internal reference matrix of the augmented real device;

Correspondingly, the equipment coordinate system calibration module is configured to calibrate the equipment coordinate system according to the equipment transformation matrix and the real transformation matrix.

Further, in another embodiment of the apparatus, the device coordinate system calibration module is specifically configured to:

Further, in another embodiment of the apparatus, the device coordinate system calibration module is further configured to:

Further, in another embodiment of the apparatus, the coordinate system calibration processing unit is specifically configured to: use the SolvePnP algorithm according to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented real device To calculate the real transformation matrix.

Further, in another embodiment of the device, the preset pattern in the image information of the preset pattern acquired by the image information acquisition module includes at least one of a two-dimensional code and a barcode.

In another aspect, an embodiment of the present specification provides a computer storage medium on which a computer program is stored. When the computer program is executed, the method for calibrating a coordinate system of an augmented reality device according to the claim is implemented.

In another aspect, an embodiment of the present specification provides a coordinate system calibration system for an augmented reality device, including at least one processor and a memory for storing processor-executable instructions. When the processor executes the instructions, the AR device is implemented Coordinate system calibration method.

The coordinate system calibration method, device, and system of the augmented reality device provided in this manual can obtain the conversion between the image coordinate system, AR coordinate system, and real coordinate system of the AR device based on the image information of preset patterns in different AR devices. Relationship, according to the conversion relationship between the coordinate systems, the AR coordinate system of different AR devices can be calibrated, so that different users can share the position and orientation of virtual objects, get the same experience, and make the AR experience interaction between multiple users It becomes possible and provides a theoretical basis for the subsequent online development of AR applications. The method provided in this embodiment of the present invention can not restrict the AR device to be set at the same position and maintain the same posture for calibration of the AR coordinate system. The operation is convenient and practical, and it meets the needs of the AR device coordinate system calibration of users at different positions .

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly explain the embodiments of the present specification or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely These are some of the embodiments described in this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without paying creative labor.

FIG. 1 is a schematic flowchart of a coordinate system calibration method of an augmented reality device according to an embodiment provided in the present specification; FIG.

2 is a schematic diagram of constructing a real coordinate system by using a two-dimensional code image in an embodiment of the present specification;

3 is a schematic flowchart of a coordinate system calibration method for an augmented reality device in another embodiment of the present specification;

FIG. 4 is a schematic structural diagram of a module of an embodiment of a coordinate system calibration apparatus for an augmented reality device provided in this specification; FIG.

5 is a schematic structural diagram of a device coordinate system calibration module according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram of a module structure of an embodiment of a coordinate system calibration system of an augmented reality device provided in the present specification.

detailed description

In order to enable those skilled in the art to better understand the technical solutions in the present specification, the technical solutions in the embodiments of the present specification will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present specification. Obviously, the described The examples are only a part of examples of this specification, but not all examples. Based on the embodiments in this specification, all other embodiments obtained by a person of ordinary skill in the art without creative efforts should fall within the protection scope of this specification.

Augmented reality technology, that is, AR technology is increasingly used in engineering, medical, military, marine aviation, and more and more applications developed by AR technology, such as an augmented reality application software launched by Apple Development Kit ARKit. The augmented reality device in the embodiments of the present specification may also be referred to as an AR device, and may include a client with an AR application, such as: a smartphone, a tablet, a smart wearable device (a smart watch, a virtual reality glasses, a virtual reality helmet, etc.) ) And other electronic equipment.

When using an AR device, the device coordinate system (that is, the AR coordinate system) in the AR device needs to be determined. The AR coordinate system is the Camera coordinate system (that is, the image coordinate system in the embodiment of the present specification) in the AR device. A bridge between real world real coordinate systems. The AR device can realize the interaction between the virtual world and the real world, and the AR coordinate system in the AR device can be used to connect the virtual world with the real world. Generally, the AR coordinate system in the AR device is the position of the camera in the AR device in the real world after the AR device is successfully initialized. The direction of the Y axis of the AR coordinate system is the opposite of gravity, and the directions of the X and Z axes are The posture of AR devices is related. Therefore, even if it is the same AR device, as long as the position of the AR device changes or the posture of the AR device is different during the startup process, the AR coordinate system after initialization will be different. Therefore, the position and orientation of virtual objects cannot be shared between different AR devices, which will cause isolation of the AR experience, and different users cannot interact in the same AR space.

The method for calibrating the coordinate system of an augmented reality device provided in the embodiments of this specification scans different AR devices for preset patterns, and based on the image information of the preset patterns scanned in the AR device, the AR coordinate system and real coordinates of the AR device can be obtained. The conversion relationship between the image system and the image coordinate system, according to the conversion relationship between the coordinate systems, the calibration of the AR coordinate system of different AR devices can be achieved. There is no need to limit the position and posture of the AR device, and it is not necessary to fix different AR devices at the same fixed point and maintain the same posture to achieve the calibration of the coordinate system of the AR device. The method is simple and flexible, and has wide applicability, which meets the needs of AR device coordinate system calibration between different users.

Specifically, FIG. 1 is a schematic flowchart of a coordinate system calibration method of an augmented reality device according to an embodiment provided in this specification. As shown in FIG. 1, a coordinate system calibration method of an augmented reality device provided by an embodiment of this specification includes:

S2. Scan the preset pattern by using the augmented reality device to obtain image information of the preset pattern in the augmented reality device.

Specifically, the AR device can be controlled to scan a preset pattern, and after scanning, the camera data in the AR device is processed, that is, each frame of image information in the Camera is processed to obtain image information of the preset pattern in the AR device. When using the AR device to scan a preset pattern, different AR devices can scan the same preset pattern, or send information (such as a photo) of the preset pattern to different AR devices, or copy the preset pattern Multiple copies, different AR devices scan the copied image of a preset pattern, or different AR devices can scan patterns with the same image characteristics and the same size.

In one embodiment of the present specification, the preset pattern may be a two-dimensional code, a bar code, or other planar patterns with characteristics. Flat patterns such as two-dimensional codes and barcodes are more common in daily life. It is easier to guide users to scan preset patterns using AR devices, and it is easier for users to scan two-dimensional codes or barcodes. For example: if users A, B, and C are located in different cities, users A, B, and C can agree to use the same QR code image as a preset pattern, and users A, B, and C can use their respective AR devices to scan and agree Image of the QR code, the AR device of users A, B, and C can obtain the image information of the QR code image.

S4. Acquire image coordinate information of a reference point in the image information in an image coordinate system of the enhancement device, and real coordinate information of the reference point in a real coordinate system.

Specifically, a reference point can be selected in the preset pattern, and the reference point can be any pixel point in the preset pattern, such as: a corner point, a center point in the preset pattern, or a special feature in the preset pattern. The number of reference points can be set according to actual needs, such as four, which are not specifically limited in the embodiments of the present specification. By analyzing the image information of the preset pattern in the AR device, the image coordinate information of the reference point in the image information of the preset pattern in the image coordinate system of the AR device and the real coordinate information of the reference point in the real coordinate system . The image coordinate information of the reference point in the image coordinate system of the AR device can be obtained from the AR device's own system by analyzing the image information of the preset pattern in the AR device. For example, the preset pattern can be obtained by detecting the barcode of the AR device. The coordinate information of the reference point in the image information in the image coordinate system of the AR device. The image coordinate system can represent the Camera coordinate system of the AR device, the real coordinate system can represent the real world coordinate system, and the specific construction method of the real coordinate system can be selected according to actual needs, which are not specifically limited in the embodiments of this specification.

The coordinate information of the reference point in the image information of the preset pattern in the real coordinate system may be determined according to the actual size of the preset image and the actual position of the reference point in the preset pattern.

In an embodiment of the present specification, a method for acquiring real coordinate information of the reference point in a real coordinate system includes:

Determining a calibration point in the image information;

The calibration point may be one of the reference points, or may be another pixel point other than the reference point in the preset pattern. The calibration point can be used to determine the relative position of the real coordinate system, and can be used as the reference point of the reference point in the preset pattern in the real coordinate system. According to the relative positions of the calibration point and the reference point in the preset pattern, the real coordinate information of the reference point in the real coordinate system is determined. For example, you can use the calibration point as the coordinate origin to build a real coordinate system. For example, if the preset pattern is rectangular, square, or parallelogram, you can use the direction parallel to the length of the side of the preset pattern as the coordinate axis to construct the real coordinate system. According to the actual size of the preset pattern and the position of the reference point in the preset pattern, the true coordinate information of the reference point is determined.

In an embodiment of the present specification, the determining a calibration point in the image information includes:

Selecting one of the reference points as the calibration point;

FIG. 2 is a schematic diagram of constructing a real coordinate system by using a QR code image in an embodiment of the specification. As shown in FIG. 2, in an embodiment of the specification, four corner points of the QR code image may be selected as reference points, and The reference point in the lower left corner can be used as the calibration point. The reference point in the lower left corner is used as the origin, and the straight line on which the two sides of the calibration point are located in the two-dimensional code image is used as the coordinate axis. As shown in FIG. Real coordinate information. For example, if the side length of a QR code image is 1 decimeter, the coordinate information of the four reference points in the real coordinate system can be expressed as: (0,0), (1,0), (1,1), (0,1). Of course, according to actual needs, the reference point can also choose other positions in the QR code image, such as the center point of the QR code image. The method of constructing the real coordinate system can also be adjusted according to actual needs, such as: A point other than the dimension code image is used as a coordinate origin to construct a real coordinate system, etc., which are not specifically limited in the embodiments of the present specification.

It should be noted that the method of constructing the real coordinate system of different AR devices in the embodiments of the present specification is the same, so that the true coordinate information of the reference point in different AR devices can be ensured.

If the two QR code images have the same size, as shown in FIG. 2, if four corner points of the two-dimensional code image are selected as reference points, different AR devices can scan the same-sized two-dimensional code image to obtain a preset. Image information of the pattern. At this time, the true coordinate information of the reference point in the two-dimensional code image obtained by scanning by different AR devices may be considered to be the same.

S6. Based on the image coordinate information and the real coordinate information, determine a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device.

After obtaining the image coordinate information of the reference point in the image coordinate system of the AR device and the real coordinate information in the real coordinate system, the real coordinate system, image coordinate system, and The AR coordinate system is a conversion relationship between device coordinate systems. For example, according to the real coordinate information and image coordinate information of the reference point, how to perform offset, rotation, and other operations on the reference point can be used to convert the real coordinate information of the reference point in the real coordinate system to the position in the image coordinate system. Obtain the transformation relationship between the real coordinate system and the image coordinate system. At the same time, the AR device system can be used to obtain the conversion relationship between the AR coordinate system of the AR device and the image coordinate system. For example, the AR coordinate system and image of the AR device can be obtained through a view transformation matrix based on OpenGL (Open Graphics Interface) The conversion relationship between the coordinate systems can also be directly obtained through the ARKit API (Application Program Interface). (For example, it can be used through the ARKit API View matrix in ARCamera). ARKit can represent an augmented reality application software development kit launched by Apple, and API can represent a calling interface left by the operating system to applications.

S8. Calibrate the device coordinate system according to the conversion relationship.

In an embodiment of the present specification, the conversion relationship between the AR coordinate system and the real coordinate system can be obtained indirectly by using the conversion relationship between the real coordinate system and the image coordinate system and the conversion relationship between the AR coordinate system and the image coordinate system. Relationship, the AR coordinate system of different AR devices is calibrated to the real coordinate system, and the coordinate system of different AR devices is calibrated.

For example, if the conversion relationship between the real coordinate system and the image coordinate system of an AR device is: A × k = B, the conversion relationship between the AR coordinate system and the image coordinate system is: C × p = B, where: A can represent the coordinates of pixel O in the real coordinate system, B can represent the coordinates of pixel O in the image coordinate system, C can represent the coordinates of pixel O in the AR coordinate system, and k and p each represent a conversion coefficient. Then, the conversion relationship between the AR coordinate system and the real coordinate system can be obtained as: C = A × (k / p). The AR coordinate system of the AR device can be calibrated into the real coordinate system. Using this method, the coordinate system of different AR devices can be calibrated, and the conversion relationship between the AR coordinate system of different AR devices and the real coordinate system can be obtained. The calibration of the device coordinate system enables the AR coordinate systems of different AR devices to be unified.

The coordinate system calibration method of the augmented reality device provided in the embodiment of the present specification can obtain the conversion relationship between the image coordinate system, the AR coordinate system, and the real coordinate system of the AR device based on the image information of preset patterns in different AR devices. According to the transformation relationship between the coordinate systems, the calibration of the AR coordinate system of different AR devices can be achieved, so that different users can share the position and orientation of virtual objects and get the same experience, providing a theoretical basis for the subsequent online development of AR applications. . The method provided in this embodiment of the present invention can not restrict the AR device to be set at the same position and maintain the same posture for calibration of the AR coordinate system. The operation is convenient and practical, and it meets the needs of the AR device coordinate system calibration of users at different positions. .

Based on the above embodiment, in an embodiment of the present specification, the determining a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device includes:

Specifically, when calculating the conversion relationship between the real coordinate system and the image coordinate system, an embodiment of the present specification may use the obtained image coordinate information of the reference point, the real coordinate information, and the camera internal parameter matrix of the AR device to calculate the reference. Point displacement, offset, rotation, etc., to obtain the real transformation matrix between the real coordinate system and the image coordinate system. The camera internal parameter matrix can represent the transformation relationship from the camera coordinate system to the image plane coordinate system. The image plane coordinate system is expressed in pixel units, and the camera coordinate system (that is, the image coordinate system) is expressed in millimeters. To complete the change process, you must first obtain the linear relationship between the pixel unit and the millimeter unit of the image plane. The camera internal reference matrix in the embodiment of the present specification may be obtained from parameters of a camera in an AR device.

In one embodiment of the present specification, the real transformation matrix between the real coordinate system and the image coordinate system may be calculated by using the SolvePnP algorithm through the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented real device. SolvePnP can represent an algorithm for solving camera rotation and translation provided in the OpenCV library. OpenCV can represent a function library mainly for real-time computer vision programming.

The SolvePnP algorithm can be used to calculate the pose of the camera of the AR device, and calculate the real transformation matrix between the real coordinate system and the image coordinate system.

At the same time, an AR device can be used to obtain a device transformation matrix between the AR coordinate system and the image coordinate system. Using the real transformation matrix and the device transformation matrix, the AR coordinate systems of different AR devices can be calibrated.

In an embodiment of the present specification, the calibrating the device coordinate system according to the device conversion matrix and the real conversion matrix may include:

Calibration of the device coordinate system of the first augmented reality device and the second augmented reality device is performed according to the following formula (1):

P _{cameraA_real} ^-1 P _{cameraA_AR} O _ARA = P _{cameraB_real} ^-1 P _{cameraB_AR} O _ARB (1)

In the above formula, P _{cameraA_real} may represent a real transformation matrix corresponding to the first augmented reality device, P _{cameraA_AR} may represent a device transformation matrix corresponding to the first augmented reality device, and O _ARA may represent a point O at the first augmentation Coordinates in the device coordinate system of a real device, P _{cameraB_real} may represent a real transformation matrix corresponding to the second augmented reality device, P _{cameraB_AR} may represent a device transformation matrix corresponding to the second augmented reality device, and O _ARB may represent point O Coordinates in a device coordinate system of the second augmented reality device.

Suppose there are two AR devices A and B. We already know a point O in the real world, and define it as follows: The coordinates of point O in the image coordinate system of AR device A are O _cameraA and the image coordinates of point O in AR device B. The coordinates in the system are O _cameraB , the coordinates of point O in the real coordinate system are O _real , the real coordinate system is transformed into the real transformation matrix of the image coordinate system of AR device A as P _{cameraA_real} , and the real coordinate system is transformed into that of AR device B The real transformation matrix of the image coordinate system is P _{cameraB_real} , the coordinates of point O in the AR coordinate system of AR device A is O _ARA , the coordinates of point O in the AR coordinate system of AR device B is O _ARB , and the AR of AR device A The device matrix of the coordinate system converted to the image coordinate system of AR device A is P _{cameraA_AR} , and the device matrix of the AR coordinate system of AR device B to the image coordinate system of AR device B is P _{cameraB_AR} . You can get the following equations:

The above formula (1) can be obtained by transforming and changing the above formula (2), so the real conversion matrix between the real coordinate system and image coordinate system of different AR devices, and between the AR coordinate system and the image coordinate system can be calculated. After the conversion matrix of the device, the coordinate information of the point O can be calibrated by using formula (1). The method is simple and accurate, and the coordinate system calibration of different AR devices is realized.

Based on the above embodiment, in an embodiment of the present specification, the method for calibrating the device coordinate system further includes:

For example, if coordinate systems of 3 AR devices need to be calibrated, the real transformation matrix between the real coordinate system and the image coordinate system corresponding to the 3 AR devices, and the device conversion between the AR coordinate system and the image coordinate system can be obtained. The matrix can use the AR coordinate system of any one of the AR devices as a reference to obtain the conversion relationship between the AR coordinate system of the other 2 AR devices and the AR coordinate system. For example, the following formula (3) can be obtained:

The AR coordinate systems of AR devices B and C were calibrated to the AR coordinate system of AR device A, and the coordinate systems of the three AR devices were calibrated.

FIG. 3 is a schematic flowchart of a coordinate system calibration method for an augmented reality device in another embodiment of the present specification. The following describes a calibration process of an AR device coordinate system in detail with reference to FIG. 3 in the following:

(1) The AR device is initialized. The AR device needs to be initialized for startup.

(2) Process the frame data in the camera of the AR device. The user can use the AR device to scan a preset QR code image, and after scanning, the frame data of the image information in the camera of the AR device can be processed.

(3) Identify whether there is a two-dimensional code image in the image information of the AR device. If so, perform step (4); if not, return to step (2) until the two-dimensional code image is recognized.

(4) Acquire the image coordinate information of the four corner points in the two-dimensional code image in the image coordinate system, and the real coordinate information of the four corner points in the two-dimensional code image in the real coordinate system.

(5) According to the image coordinate information of the two-dimensional image, the real coordinate information, and the camera internal reference matrix of the AR device, the SolvePnP algorithm is used to calculate the camera pose of the AR device to obtain the real transformation matrix of the real coordinate system transformed into the image coordinate system of the AR device.

(6) Acquire the AR space view transformation matrix of the AR device, that is, the device transformation matrix of the AR device's AR coordinate system to an image coordinate system.

(7) Use the above formula (1) to calculate the conversion relationship of the AR coordinate systems of different AR devices, and complete the calibration of the AR device's coordinate system.

The method provided in the embodiment of the present specification can implement the calibration of the AR coordinate system of different AR devices based on the image information of preset patterns in different AR devices, so that different users can share the position and orientation of virtual objects and get the same experience. This makes it possible for the AR experience interaction between multiple users to provide a theoretical basis for the subsequent online development of AR applications. The method provided in this embodiment of the present invention can be used to calibrate the AR coordinate system without restricting the AR devices to be set at the same position and the same posture. The operation is convenient and practical. Improved user experience.

Each embodiment of the above method in this specification is described in a progressive manner, and the same or similar parts between the various embodiments may refer to each other. Each embodiment focuses on the differences from other embodiments. For related points, refer to the description of the method embodiments.

Based on the coordinate system calibration method of the augmented reality device described above, one or more embodiments of the present specification further provide a coordinate system calibration device of the augmented reality device. The device may include a system (including a distributed system), software (application), module, component, server, client, etc. that uses the method described in the embodiments of the present specification, and a device that implements necessary hardware. Based on the same innovative concept, the devices in one or more embodiments provided in the embodiments of this specification are as described in the following embodiments. Since the implementation solution of the device to solve the problem is similar to the method, the implementation of the specific device in the embodiment of this specification may refer to the implementation of the foregoing method, and the duplicated details are not described again. As used below, the term "unit" or "module" may be a combination of software and / or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and conceived.

Specifically, FIG. 4 is a schematic block diagram of an embodiment of a coordinate system calibration apparatus for an augmented reality device provided in this specification. As shown in FIG. 4, the coordinate system calibration apparatus for an augmented reality device provided in this specification includes: image information acquisition Module 41, coordinate information acquisition module 42, conversion relationship determination module 43, and equipment coordinate system calibration module 44, wherein:

The image information acquisition module 41 may be configured to scan a preset pattern by using an augmented reality device to acquire image information of the preset pattern in the augmented reality device;

The coordinate information obtaining module 42 may be configured to obtain image coordinate information of a reference point in the image information in an image coordinate system of the enhancement device, and real coordinate information of the reference point in a real coordinate system;

A conversion relationship determining module 43 may be configured to determine a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device based on the image coordinate information and the real coordinate information;

The device coordinate system calibration module 44 may be configured to calibrate the device coordinate system according to the conversion relationship.

The coordinate system calibration device of the augmented reality device provided in the embodiment of the present specification can obtain the conversion relationship between the image coordinate system, the AR coordinate system, and the real coordinate system of the AR device based on image information of preset patterns in different AR devices. According to the transformation relationship between the coordinate systems, the calibration of the AR coordinate system of different AR devices can be achieved, so that different users can share the position and orientation of virtual objects and get the same experience, providing a theoretical basis for the subsequent online development of AR applications. . The method provided in this embodiment of the present invention can be used to calibrate the AR coordinate system without restricting the AR devices to be set at the same position and the same posture. The operation is convenient, and the practicability is strong, which meets the needs of the coordinate system calibration of AR devices for users at different positions.

Based on the above embodiment, the coordinate information acquisition module includes a real coordinate acquisition unit,

The real coordinate obtaining unit is specifically configured to:

Determining a calibration point in the image information;

Based on the above embodiment, the real coordinate obtaining unit is further configured to:

Selecting one of the reference points as the calibration point;

The coordinate system calibration device of the augmented reality device provided in the embodiment of the present specification obtains coordinate information of the reference point in the real coordinate system based on the actual size of the preset pattern and the relative position between the reference point and the calibration point. Because different ARs use the same size preset pattern, it is possible to calibrate the coordinate system of the AR device based on the real coordinate information of the reference point in the preset pattern.

FIG. 5 is a schematic structural diagram of a device coordinate system calibration module in an embodiment of the description. As shown in FIG. 5, based on the above embodiment, the conversion relationship determination module 43 includes:

The device conversion matrix calculation unit 51 may be configured to obtain a device conversion matrix for converting the device coordinate system of the augmented reality device into the image coordinate system;

The real transformation matrix calculation unit 52 may be configured to calculate a real transformation matrix converted from the real coordinate system to the image coordinate system according to the image coordinate information, the real coordinate information, and a camera internal parameter matrix of the augmented real device. ;

Accordingly, the device coordinate system calibration module 44 may be configured to calibrate the device coordinate system according to the device transformation matrix and the real transformation matrix.

In the embodiment of the present specification, the transformation relationship between AR coordinate systems between different AR devices can be obtained by using the real transformation matrix of the image coordinate system and the real coordinate system of the AR device, and the transformation matrix between the AR coordinate system and the image coordinate system. Provides an accurate theoretical basis for the calibration of the coordinate system of AR equipment.

Based on the above embodiments, the device coordinate system calibration module is specifically configured to:

The embodiments of this specification provide coordinate system conversion formulas of different AR devices, which can accurately calibrate the coordinate systems of each AR device.

Based on the above embodiments, the device coordinate system calibration module is further configured to:

Based on the above embodiment, the coordinate system calibration processing unit is specifically configured to: according to the image coordinate information, the real coordinate information, and a camera internal parameter matrix of the augmented real device, use SolvePnP algorithm to calculate and obtain the Real transformation matrix.

In the embodiment of the present specification, the pose of the camera in the AR device can be calculated by using the SolvePnP algorithm, and the real transformation matrix between the real coordinate system of the AR device and the image coordinate system is obtained.

Based on the above embodiment, the preset pattern in the image information of the preset pattern acquired by the image information acquisition module includes at least one of a two-dimensional code and a barcode.

In the embodiment of the present specification, a two-dimensional code, a bar code, and other flat patterns that are common in daily life are used as preset patterns, which is convenient for guiding a user to scan, convenient in operation, and improving the user experience.

It should be noted that the above-mentioned device according to the description of the method embodiments may also include other implementations. For specific implementation manners, reference may be made to the description of related method embodiments, and details are not described herein.

In an embodiment of the present specification, a computer storage medium may also be provided, in which a computer program is stored. When the computer program is executed, the method for processing video data in the foregoing embodiment is implemented. For example, the following method may be implemented:

The specific embodiments of the present specification have been described above. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and the desired result may still be achieved. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The method or device described in the foregoing embodiments provided in this specification may implement business logic through a computer program and record it on a storage medium, and the storage medium may be read and executed by a computer to achieve the effect of the solution described in the embodiments of this specification.

The method or device for calibrating the coordinate system of the above-mentioned augmented reality device provided in the embodiment of the present specification may be implemented by a processor executing corresponding program instructions in a computer, such as using a C ++ language of a Windows operating system on a PC, a Linux system, or Others, for example, use android and iOS system programming languages to implement in smart terminals, and to implement processing logic based on quantum computers. In an embodiment of a coordinate system calibration system for an augmented reality device provided in this specification, FIG. 6 is a schematic diagram of a module structure of an embodiment of a coordinate system calibration system for an augmented reality device provided in this specification, as shown in FIG. 6, The coordinate system calibration system of the augmented reality device provided by the embodiment of the present specification may include a processor 61 and a memory 62 for storing processor-executable instructions.

The processor 61 and the memory 62 complete communication with each other through a bus 63;

The processor 61 is configured to call program instructions in the memory 62 to execute the methods provided in the foregoing seismic data processing method embodiments. For example, the processor 61 scans a preset pattern using an augmented reality device, and obtains the information in the augmented reality device. The image information of the preset pattern; acquiring image coordinate information of a reference point in the image information in an image coordinate system of the enhancement device, and real coordinate information of the reference point in a real coordinate system; based on the The image coordinate information and the real coordinate information, determine a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device, and perform the device coordinate system according to the conversion relationship calibration.

It should be noted that the description of the device, the computer storage medium, and the system described in the description according to the related method embodiments may also include other implementations. For specific implementations, reference may be made to the description of the method embodiments, and details are not described herein.

Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the hardware + program embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the related parts, refer to the description of the method embodiment.

The embodiments of the present specification are not limited to situations that must conform to industry communication standards, standard computer data processing and data storage rules, or one or more embodiments described in this specification. Certain industry standards or implementations that are slightly modified based on implementations described in custom methods or embodiments can also achieve the same, equivalent or similar, or predictable implementation effects of the above embodiments. The embodiments obtained by applying these modified or deformed data acquisitions, storages, judgments, and processing methods may still fall within the scope of the optional implementations of the examples in this specification.

In the 1990s, for a technical improvement, it can be clearly distinguished whether it is an improvement in hardware (for example, the improvement of circuit structures such as diodes, transistors, switches, etc.) or an improvement in software (for the improvement of method flow). However, with the development of technology, the improvement of many methods and processes can be regarded as a direct improvement of the hardware circuit structure. Designers almost always get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware entity modules. For example, a programmable logic device (Programmable Logic Device (PLD)) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system on a PLD, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of making integrated circuit chips manually, this programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used in program development and writing, but before compilation The original code must also be written in a specific programming language, which is called Hardware Description Language (HDL), and HDL is not the only one, but there are many types, such as ABEL (Advanced Boolean ExpressionLanguage) , AHDL (Altera, Hardware, Description, Language), Confluence, CUPL (Cornell, University Programming, Language), HDCal, JHDL (Java, Hardware, Description, Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby, Hardware, Description), etc. VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be clear that as long as the method flow is logically programmed and integrated into the integrated circuit using the above-mentioned several hardware description languages, a hardware circuit that implements the logic method flow can be easily obtained.

The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro) processor. , Logic gates, switches, Application Specific Integrated Circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, With the Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in a purely computer-readable program code manner, it is entirely possible to make the controller logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded controllers by logically programming the method steps. Microcontrollers, etc. to achieve the same function. Therefore, such a controller can be regarded as a hardware component, and a device included in the controller for implementing various functions can also be regarded as a structure within the hardware component. Or even, the means for implementing various functions can be regarded as a structure that can be both a software module implementing the method and a hardware component.

The system, device, module, or unit described in the foregoing embodiments may be specifically implemented by a computer chip or entity, or a product with a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, and a tablet. A computer, a wearable device, or a combination of any of these devices.

Although one or more embodiments of the present specification provide method operation steps as described in the embodiments or flowcharts, more or less operation steps may be included based on conventional or non-creative means. The sequence of steps listed in the embodiments is only one way of executing the steps, and does not represent the only sequence of execution. When the actual device or terminal product is executed, it may be executed sequentially or in parallel according to the method shown in the embodiment or the accompanying drawings (for example, a parallel processor or multi-threaded environment, or even a distributed data processing environment). The terms "including," "including," or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, product, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed Elements, or elements that are inherent to such a process, method, product, or device. Without further limitation, it does not exclude that there are other identical or equivalent elements in the process, method, product or equipment including the elements. Words such as first and second are used to indicate names, but not in any particular order.

For the convenience of description, when describing the above device, the functions are divided into various modules and described separately. Of course, when implementing one or more of this specification, the functions of each module may be implemented in the same or multiple software and / or hardware, or the module that implements the same function may be implemented by a combination of multiple submodules or subunits, etc. . The device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be another division manner. For example, multiple units or components may be combined or integrated. To another system, or some features can be ignored or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, which may be electrical, mechanical or other forms.

The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and combinations of processes and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, so that the instructions generated by the processor of the computer or other programmable data processing device are used to generate Means for implementing the functions specified in one or more flowcharts and / or one or more blocks of the block diagrams.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to work in a particular manner such that the instructions stored in the computer-readable memory produce a manufactured article including an instruction device, the instructions The device implements the functions specified in one or more flowcharts and / or one or more blocks of the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing device, so that a series of steps can be performed on the computer or other programmable device to produce a computer-implemented process, which can be executed on the computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

Memory may include non-persistent memory, random access memory (RAM), and / or non-volatile memory in computer-readable media, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both permanent and non-persistent, removable and non-removable media. Information can be stored by any method or technology. Information may be computer-readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), and read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, Magnetic tape cartridges, magnetic tape magnetic disk storage, graphene storage, or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. As defined herein, computer-readable media does not include temporary computer-readable media, such as modulated data signals and carrier waves.

Those skilled in the art should understand that one or more embodiments of the present specification may be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of the present specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of the present specification may adopt a computer program implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code therein. The form of the product.

One or more embodiments of the specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present specification may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant part, refer to the description of the method embodiment. In the description of this specification, the description with reference to the terms “one embodiment”, “some embodiments”, “examples”, “specific examples”, or “some examples” and the like means specific features described in conjunction with the embodiments or examples , Structure, materials, or features are included in at least one embodiment or example of this specification. In this specification, the schematic expressions of the above terms are not necessarily directed to the same embodiment or example. Moreover, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, without any contradiction, those skilled in the art may combine and combine different embodiments or examples and features of the different embodiments or examples described in this specification.

The above description is only an embodiment of one or more embodiments of the present specification, and is not intended to limit one or more embodiments of the present specification. For those skilled in the art, various modifications and changes can be made to one or more embodiments of the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims.

Claims

A coordinate system calibration method for an augmented reality device includes:

Scanning the preset pattern using an augmented reality device to obtain image information of the preset pattern in the augmented reality device;

Acquiring image coordinate information of a reference point in the image information in an image coordinate system of the enhancement device, and real coordinate information of the reference point in a real coordinate system;

Determining a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device based on the image coordinate information and the real coordinate information;

The device coordinate system is calibrated according to the conversion relationship.
The method according to claim 1, wherein the method for acquiring real coordinate information of the reference point in a real coordinate system comprises:

Determining a calibration point in the image information;

Determine the real coordinate information of the reference point in the real coordinate system according to the relative position between the reference point and the calibration point in the preset pattern.
The method according to claim 2, wherein determining a calibration point in the image information comprises:

Selecting one of the reference points as the calibration point;

Correspondingly, determining the real coordinate information of the reference point in the real coordinate system includes:

Using the calibration point as a coordinate origin, selecting a straight line where a line connecting two reference points outside the calibration point and the calibration point is located as a coordinate axis, and constructing the true coordinate system;

Determine the real coordinate information of the reference point in the real coordinate system according to the relative position between the reference point and the calibration point in the preset pattern.
The method according to claim 1, wherein determining a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device comprises:

Acquiring a device transformation matrix for converting the device coordinate system of the augmented reality device into the image coordinate system;

Calculating the real transformation matrix transformed from the real coordinate system into the image coordinate system according to the image coordinate information, the real coordinate information, and the camera internal reference matrix of the augmented real device;

Accordingly, the method for calibrating the device coordinate system according to the conversion relationship includes:

The device coordinate system is calibrated according to the device conversion matrix and the real conversion matrix.
The method according to claim 4, wherein calibrating the device coordinate system based on the device transformation matrix and the real transformation matrix comprises:

The device coordinate system of the first augmented reality device and the second augmented reality device is calibrated according to the following formula:

P cameraA_real -1 P cameraA_AR O ARA = P cameraB_real -1 P cameraB_AR O ARB

In the above formula, P cameraA_real represents a real transformation matrix corresponding to the first augmented reality device, P cameraA_AR represents a device transformation matrix corresponding to the first augmented reality device, and O ARA represents a point O on the first augmented reality device. Coordinates in the device coordinate system, P cameraB_real represents the real transformation matrix corresponding to the second augmented reality device, P cameraB_AR represents the device transformation matrix corresponding to the second augmented reality device, and O ARB represents the point O at the second augmentation Coordinates in the device coordinate system of the real device.
The method according to claim 5, wherein the method for calibrating the device coordinate system according to the conversion relationship further comprises:

If there are more than two augmented reality devices for device coordinate system calibration, the device coordinate system is calibrated based on the device coordinate system of any one of the augmented reality devices.
The method according to claim 4, wherein said calculating a true transformation of said real coordinate system into said image coordinate system based on said image coordinate information, said real coordinate information, and said camera internal parameter matrix of said augmented real device. Matrix, including:

According to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented real device, the SolvePnP algorithm is used to calculate and obtain the real transformation matrix.
The method according to any one of claims 1 to 7, wherein the preset pattern comprises at least one of a two-dimensional code and a bar code.
A coordinate system calibration device for an augmented reality device includes:

An image information acquisition module, configured to scan a preset pattern using an augmented reality device to acquire image information of the preset pattern in the augmented reality device;

A coordinate information acquisition module, configured to acquire image coordinate information of a reference point in the image information in an image coordinate system of the enhancement device, and real coordinate information of the reference point in a real coordinate system;

A conversion relationship determining module, configured to determine a conversion relationship between the real coordinate system, the image coordinate system, and a device coordinate system of the augmented reality device based on the image coordinate information and the real coordinate information;

The device coordinate system calibration module is configured to calibrate the device coordinate system according to the conversion relationship.
The apparatus according to claim 9, wherein the coordinate information acquisition module includes a real coordinate acquisition unit,

The real coordinate obtaining unit is specifically configured to:

Determining a calibration point in the image information;

Determine the real coordinate information of the reference point in the real coordinate system according to the relative position between the reference point and the calibration point in the preset pattern.
The apparatus according to claim 10, wherein the real coordinate obtaining unit is further configured to:

Selecting one of the reference points as the calibration point;

Using the calibration point as a coordinate origin, selecting a straight line where a line connecting two reference points outside the calibration point and the calibration point is located as a coordinate axis, and constructing the true coordinate system;

Determine the real coordinate information of the reference point in the real coordinate system according to the relative position between the reference point and the calibration point in the preset pattern.
The apparatus according to claim 9, the conversion relationship determining module comprises:

A device transformation matrix calculation unit, configured to obtain a device transformation matrix for converting the device coordinate system of the augmented reality device into the image coordinate system;

A real transformation matrix calculation unit, configured to calculate a real transformation matrix transformed from the real coordinate system into the image coordinate system according to the image coordinate information, the real coordinate information, and a camera internal reference matrix of the augmented real device;

Correspondingly, the equipment coordinate system calibration module is configured to calibrate the equipment coordinate system according to the equipment transformation matrix and the real transformation matrix.
The device according to claim 12, wherein the device coordinate system calibration module is specifically configured to:

The device coordinate system of the first augmented reality device and the second augmented reality device is calibrated according to the following formula:

P cameraA_real -1 P cameraA_AR O ARA = P cameraB_real -1 P cameraB_AR O ARB

In the above formula, P cameraA_real represents a real transformation matrix corresponding to the first augmented reality device, P cameraA_AR represents a device transformation matrix corresponding to the first augmented reality device, and O ARA represents a point O on the first augmented reality device. Coordinates in the device coordinate system, P cameraB_real represents the real transformation matrix corresponding to the second augmented reality device, P cameraB_AR represents the device transformation matrix corresponding to the second augmented reality device, and O ARB represents the point O at the second augmentation Coordinates in the device coordinate system of the real device.
The apparatus according to claim 13, the device coordinate system calibration module is further configured to:

If there are more than two augmented reality devices for device coordinate system calibration, the device coordinate system is calibrated based on the device coordinate system of any one of the augmented reality devices.
The apparatus according to claim 12, wherein the coordinate system calibration processing unit is specifically configured to: according to the image coordinate information, the real coordinate information, and a camera internal parameter matrix of the augmented real device, use SolvePnP algorithm to calculate and obtain Describe the real transformation matrix.
The device according to claim 9, wherein the preset pattern in the image information of the preset pattern acquired by the image information acquisition module includes at least one of a two-dimensional code and a barcode.
A computer storage medium stores a computer program thereon, and when the computer program is executed, the method according to any one of claims 1 to 8 is implemented.
A coordinate system calibration system for an augmented reality device includes at least one processor and a memory for storing processor-executable instructions, and the processor implements the method according to any one of claims 1 to 8 when executing the instructions. .