TWI712004B - Coordinate system calibration method and device of augmented reality equipment - Google Patents

Abstract

This manual provides a coordinate system calibration method and device for augmented reality equipment. The method includes: using an augmented reality device to scan a preset pattern to obtain image information of the preset pattern in the augmented reality device; and acquiring a reference point in the image information in the augmented reality device The image coordinate information in the image coordinate system of the device, and the real coordinate information of the reference point in the real coordinate system; based on the image coordinate information and the real coordinate information, the real coordinate system and the real coordinate system are obtained. The conversion relationship between the image coordinate system and the device coordinate system of the augmented reality device, and the device coordinate system is calibrated according to the conversion relationship. By using the various embodiments in this specification, it is possible to calibrate the AR coordinate system without restricting the AR device to be set in the same position and the same posture. The operation is convenient and the practicability is strong, which meets the needs of the coordinate system calibration of the AR device of users in different locations. .

Description

Coordinate system calibration method and device of augmented reality equipment

This specification belongs to the field of augmented reality technology, and in particular relates to a coordinate system calibration method and device for augmented reality equipment.

With the development of science and technology, there are more and more applications of augmented reality technology, namely AR (Augmented Reality) technology. AR technology applies virtual information to the real world. The real environment and virtual objects are instantly superimposed on the same screen. Or the space exists at the same time, and people can use AR equipment to experience the combination of virtual and reality.

However, the AR coordinate system in the AR device is usually related to the posture and position of the AR device when it is initialized. Different AR devices have different positions and postures during initialization, and the AR coordinate system may be different. Different users cannot interact in the same AR space. Therefore, AR applications are usually stand-alone applications. In the prior art, the method for calibrating the coordinate system of an AR device is usually to place each device in the same position, maintain the same posture, and then initialize the device. It can be seen that this method has relatively strong limitations, inconvenient operation, and cannot realize the calibration of the AR coordinate system of users in different positions. Therefore, there is a great need for an implementation solution that can facilitate the calibration of multiple AR equipment coordinate systems.

The purpose of this specification is to provide a coordinate system calibration method and device for augmented reality equipment. The method is simple and realizes the calibration of the AR coordinate system of different AR equipment.

On the one hand, an embodiment of the present specification provides a coordinate system calibration method of an augmented reality device, which includes: scanning a preset pattern with the augmented reality device to obtain image information of the preset pattern in the augmented reality device; Obtain the image coordinate information of the reference point in the image information in the image coordinate system of the augmented reality device, and the real coordinate information of the reference point in the real coordinate system; based on the image The coordinate information, the real coordinate information, determine the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device; according to the conversion relationship, the device coordinate system Department for calibration.

Further, in another embodiment of the method, the method for acquiring the real coordinate information of the reference point in the real coordinate system includes: determining a calibration point in the image information; The relative position between the reference point and the calibration point determines the real coordinate information of the reference point in the real coordinate system.

Further, in another embodiment of the method, the determining the calibration point in the image information includes: Selecting one of the reference points as the calibration point; accordingly, the determining the real coordinate information of the reference point in the real coordinate system includes: taking the calibration point as the origin of the coordinate, and selecting the calibration point The straight line connecting the two reference points outside the fixed point and the calibration point is used as the coordinate axis to construct the real coordinate system; according to the relative position between the reference point and the calibration point in the preset pattern To determine the real coordinate information of the reference point in the real coordinate system.

Further, in another embodiment of the method, the determining the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device includes: acquiring the The device coordinate system of the augmented reality device is converted into a device conversion matrix of the image coordinate system; according to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device, Calculating the real conversion matrix that transforms the real coordinate system into the image coordinate system; correspondingly, the method for calibrating the device coordinate system according to the conversion relationship includes: according to the device conversion matrix, the The real conversion matrix is used to calibrate the coordinate system of the device.

Further, in another embodiment of the method, the calibrating the device coordinate system according to the device conversion matrix and the real conversion matrix includes: implementing the first augmented reality according to the following formula Calibration of the device coordinate system of the device and the second augmented reality device: P _{cameraA_real} ^-1 P _{cameraA_AR} O _ARA =P _{cameraB_real} ^-1 P _{cameraB_AR} O _ARB

In the above formula, P _{cameraA_real} represents the real conversion matrix corresponding to the first augmented reality device, P _{cameraA_AR} represents the device conversion matrix corresponding to the first augmented reality device, and O _ARA represents the point O in the first augmented reality device. amplification device coordinate system reality device _coordinates, P cameraB_real represents the second augmented reality device corresponding to the true transformation _matrix, P cameraB_AR augmented reality represents the second device corresponds to the device transformation matrix, O _ARB represents the coordinate of point O in the device coordinate system of the second augmented reality device.

Further, in another embodiment of the method, the method for calibrating the device coordinate system according to the conversion relationship further includes: if there are more than two augmented reality devices for performing device coordinate system calibration, then The device coordinate system is calibrated based on the device coordinate system of any one of the augmented reality devices.

Further, in another embodiment of the method, the calculation of the conversion of the real coordinate system into the real coordinate system based on the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device The real conversion matrix of the image coordinate system includes: according to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device, the real conversion matrix is calculated by using the SolvePnP algorithm .

Further, in another embodiment of the method, the preset pattern includes at least one of a two-dimensional code and a barcode.

On the other hand, this specification provides a coordinate system calibration device for augmented reality equipment, including: an image information acquisition module, which is used to scan a preset pattern with the augmented reality equipment to obtain the information described in the augmented reality equipment Image information of a preset pattern; a coordinate information acquisition module for acquiring image coordinate information of a reference point in the image information in the image coordinate system of the augmented reality device, and the reference Point on the real coordinate information in the real coordinate system; the conversion relationship determination module is used to obtain the real coordinate system, the image coordinate system, and the expansion based on the image coordinate information and the real coordinate information The conversion relationship between the device coordinate systems of the augmented reality device; the device coordinate system calibration module is used to calibrate the device coordinate system according to the conversion relationship.

Further, in another embodiment of the device, the coordinate information acquisition module includes a real coordinate acquisition unit, and the real coordinate acquisition unit is specifically configured to: determine a calibration point in the image information; The relative position between the reference point and the calibration point in the preset pattern determines the real coordinate information of the reference point in the real coordinate system.

Further, in another embodiment of the device, the real coordinate acquisition unit is further configured to: select one of the reference points as the calibration point; use the calibration point as the coordinate origin to select the calibration point The line connecting the two other reference points and the calibration point is used as the coordinate axis to construct Establishing the real coordinate system; and determining the real coordinate information of the reference point in the real coordinate system according to the relative position between the reference point and the calibration point in the preset pattern.

Further, in another embodiment of the apparatus, the conversion relationship determination module includes: a device conversion matrix calculation unit configured to obtain the device coordinate system of the augmented reality device and convert it into the image The device conversion matrix of the coordinate system; the real conversion matrix calculation unit is used to calculate the conversion of the real coordinate system into the real coordinate system according to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device The real conversion matrix of the image coordinate system; accordingly, the device coordinate system calibration module is used to calibrate the device coordinate system according to the device conversion matrix and the real conversion matrix.

Further, in another embodiment of the apparatus, the device coordinate system calibration module is specifically configured to: implement the device coordinate system of the first augmented reality device and the second augmented reality device according to the following formula Calibration: P _{cameraA_real} ^-1 P _{cameraA_AR} O _ARA =P _{cameraB_real} ^-1 P _{cameraB_AR} O _ARB

In the above formula, P _{cameraA_real} represents the real conversion matrix corresponding to the first augmented reality device, P _{cameraA_AR} represents the device conversion matrix corresponding to the first augmented reality device, and O _ARA represents the point O in the first augmented reality device. amplification device coordinate system reality device _coordinates, P cameraB_real represents the second augmented reality device corresponding to the true transformation _matrix, P cameraB_AR augmented reality represents the second device corresponds to the device transformation matrix, O _ARB represents the coordinate of point O in the device coordinate system of the second augmented reality device.

Further, in another embodiment of the device, the device coordinate system calibration module is further used for: if there are more than two augmented reality devices for performing device coordinate system calibration, use any one of them to augment reality The equipment coordinate system of the equipment is used as a reference to calibrate the equipment coordinate system.

Further, in another embodiment of the device, the coordinate system calibration processing unit is specifically configured to: according to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device, The SolvePnP algorithm is used to calculate the true conversion matrix.

Further, in another embodiment of the device, the preset pattern in the image information of the preset pattern acquired by the image information acquisition module includes at least one of a two-dimensional code and a barcode.

In another aspect, the embodiments of the present specification provide a computer storage medium with a computer program stored thereon, and when the computer program is executed, the coordinate system calibration method of the augmented reality device in the scope of the patent application is realized.

In another aspect, an embodiment of the present specification provides a coordinate system calibration system for augmented reality equipment, which includes at least one processor and a memory for storing executable instructions of the processor. The processor implements the foregoing when executing the instructions. The coordinate system calibration method of augmented reality equipment.

The coordinate system calibration method and equipment of the augmented reality equipment provided in this manual Based on the image information of the preset patterns in different AR devices, the conversion relationship between the image coordinate system, AR coordinate system, and real coordinate system of the AR device can be obtained, according to the conversion relationship between each coordinate system The AR coordinate system of different AR devices can be calibrated, so that different users can share the position and direction of virtual objects and get the same experience, making AR experience interaction between multiple users possible, and developing for subsequent AR applications. Provide a theoretical basis. The method provided by the embodiment of this description can perform AR coordinate system calibration without restricting the AR device to be set in the same position and maintaining the same posture. It is easy to operate and has strong practicability, and meets the needs of the coordinate system calibration of AR devices of users in different locations. .

S2-S8: steps

41: Image information acquisition module

42: Coordinate information acquisition module

43: Conversion relationship determination module

44: Equipment coordinate system calibration module

51: Device conversion matrix calculation unit

52: Real conversion matrix calculation unit

61: processor

62: Storage

63: Bus

In order to more clearly explain the technical solutions in the embodiments of this specification or the prior art, the following will briefly introduce the drawings that need to be used in the embodiments or the description of the prior art. Obviously, the drawings in the following description are merely the present For some of the embodiments described in the specification, for those of ordinary skill in the art, without creative labor, other schemes can be obtained from these schemes.

Fig. 1 is a schematic flow chart of the coordinate system calibration method of an augmented reality device in an embodiment provided in this specification; Fig. 2 is a schematic diagram of constructing a real coordinate system using a two-dimensional code image in an embodiment of this specification; Fig. 3 It is a schematic flow diagram of the coordinate system calibration method of the augmented reality device in another embodiment of this specification; 4 is a schematic diagram of the module structure of an embodiment of the coordinate system calibration device for augmented reality equipment provided in this specification; FIG. 5 is a schematic diagram of the structure of the device coordinate system calibration module in an embodiment of this specification; FIG. 6 is this specification A schematic diagram of the module structure of an embodiment of a coordinate system calibration system for augmented reality equipment is provided.

In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the drawings in the embodiments of this specification. Obviously, the described The embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this specification.

Augmented reality technology, that is, AR technology is increasingly used in engineering, medical, military, navigation and aviation, and more and more applications developed using AR technology, such as an augmented reality launched by Apple Environment application software development kit ARKit. The augmented reality device in the embodiments of this specification may also be called an AR device, and may include a user terminal with AR applications, such as smart phones, tablets, smart wearable devices (smart watches, virtual reality glasses, virtual Reality helmets, etc.) and other electronic equipment.

When using an AR device, you need to determine the device coordinate system in the AR device (that is, the AR coordinate system), and the AR coordinate system is the Camera in the AR device A bridge between the coordinate system (that is, the image coordinate system in the embodiment of this specification) and the real coordinate system of the real world. The AR device can realize the interaction between the virtual world and the real world, and the virtual world can be connected with the real world by using the AR coordinate system in the AR device. Under normal circumstances, the AR coordinate system in the AR device is the position of the Camera in the AR device in the real world after the AR device is successfully initialized. The direction of the Y axis of the AR coordinate system is the opposite direction of gravity, and the directions of the X axis and Z axis are the same as The posture of the AR device is related. Therefore, even for the same AR device, as long as the position of the AR device changes or the posture of the AR device is different during the startup process, the AR coordinate system after initialization will be different. Therefore, different AR devices cannot share the position and direction of virtual objects, which will cause the isolation of the AR experience, and different users cannot interact in the same AR space.

The coordinate system calibration method of the augmented reality device provided by the embodiment of this specification scans different AR devices with preset patterns. Based on the image information of the preset patterns scanned in the AR device, the AR coordinate system of the AR device can be obtained. , The conversion relationship between the real coordinate system and the image coordinate system. According to the conversion relationship between the coordinate systems, the calibration of the AR coordinate system of different AR devices can be realized. There is no need to limit the position and posture of the AR device, and it is not necessary to fix different AR devices at the same fixed point and maintain the same posture to realize the calibration of the coordinate system of the AR device. The method is simple, flexible, and has wide applicability, which meets the needs of AR device coordinate system calibration among different users.

Specifically, FIG. 1 is a schematic flow chart of the coordinate system calibration method of an augmented reality device in an embodiment provided in this specification, as shown in FIG. 1, The coordinate system calibration method of augmented reality equipment provided in the embodiments of this specification includes:

S2. Scan the preset pattern with the augmented reality device to obtain image information of the preset pattern in the augmented reality device.

Specifically, the AR device can be controlled to scan the preset pattern, and the data of the Camera in the AR device can be processed after scanning, that is, the image information of each frame in the Camera is processed to obtain the image information of the preset pattern in the AR device . When using an AR device to scan a preset pattern, different AR devices can scan the same preset pattern, or send information such as images (such as photos) of the preset pattern to different AR devices, or send the preset pattern When copying multiple copies, different AR devices scan the copied image with a preset pattern, or different AR devices may scan patterns with the same image characteristics and the same size.

The preset pattern in an embodiment of this specification may be a two-dimensional code, a barcode, or other characteristic planar patterns. Two-dimensional codes, barcodes and other flat patterns are relatively common in daily life, and it is easier to guide users to use AR devices to scan preset patterns, and it is easier for users to scan QR codes or barcodes. For example: if users A, B, and C are located in different cities, users A, B, and C can jointly agree to use the same QR code image as the preset pattern, and users A, B, and C can use their respective AR devices to scan For the agreed two-dimensional code image, the image information of the two-dimensional code image can be obtained from the AR devices of users A, B, and C.

S4. Obtain the image coordinate information of the reference point in the image information in the image coordinate system of the augmented reality device, and the reference point in the real The real coordinate information in the coordinate system.

Specifically, the reference point can be selected from the preset pattern, and the reference point can be any pixel point in the preset pattern, for example, it can be a corner point, a center point in the preset pattern, or a special feature in the preset pattern The number of reference points can be set according to actual needs, such as four, which is not specifically limited in the embodiment of this specification. By analyzing the image information of the preset pattern in the AR device, it is possible to obtain the image coordinate information of the reference point in the image information of the preset pattern in the image coordinate system of the AR device, and the reference point in the real coordinate system Real coordinate information. The image coordinate information of the reference point in the image coordinate system of the AR device can be obtained from the AR device’s own system by analyzing the image information of the preset pattern in the AR device. For example, it can be obtained through the detection barcode request of the AR device. Let the reference point in the image information of the pattern be the coordinate information in the image coordinate system of the AR device. The image coordinate system can represent the Camera coordinate system of the AR device, the real coordinate system can represent the real world coordinate system, and the specific construction method of the real coordinate system can be selected according to actual needs, and the embodiment of this specification does not specifically limit it.

The coordinate information of the reference point in the real coordinate system of the image information of the preset pattern can be determined according to the actual size of the preset image and the actual position of the reference point in the preset pattern.

In an embodiment of the present specification, the method for acquiring real coordinate information of the reference point in the real coordinate system includes: determining a calibration point in the image information; and according to the reference point and the reference point in the preset pattern The relative position between the fixed points determines the real position of the reference point in the real coordinate system Standard information.

Wherein, the calibration point may be one of the reference points or other pixel points in the preset pattern except the reference point. The calibration point can be used to determine the relative position of the real coordinate system, and can be used as the reference point of the reference point in the real coordinate system in the preset pattern. According to the relative position of the calibration point and the reference point in the preset pattern, the real coordinate information of the reference point in the real coordinate system is determined. For example: the calibration point can be used as the coordinate origin to construct the real coordinate system. For example, if the preset pattern is rectangular, square or parallelogram, the direction parallel to the side length of the preset pattern can be used as the coordinate axis to construct the real coordinate system. According to the actual size of the preset pattern and the position of the reference point in the preset pattern, the real coordinate information of the reference point is determined.

In an embodiment of this specification, the determining the calibration point in the image information includes: selecting one of the reference points as the calibration point; correspondingly, the determining that the reference point is in the real coordinate system The real coordinate information includes: taking the calibration point as the origin of the coordinate, and selecting the line connecting two reference points outside the calibration point and the calibration point as the coordinate axis to construct the real coordinate system; Determine the real coordinate information of the reference point in the real coordinate system according to the relative position between the reference point and the calibration point in the preset pattern.

Figure 2 is a schematic diagram of using a two-dimensional code image to construct a real coordinate system in an embodiment of this specification. As shown in Figure 2, an embodiment of this specification In, the four corner points of the two-dimensional code image can be selected as reference points, and the reference point in the lower left corner can be used as the calibration point. Take the reference point in the lower left corner as the origin, and use the line with the two side lengths of the calibration point in the two-dimensional code image as the coordinate axis. As shown in Figure 2, a real coordinate system can be constructed and the reference point is in the real coordinate system. The real coordinate information in. For example: if the side length of the two-dimensional code image is 1 decimeter, the coordinate information of the four reference points in the real coordinate system can be expressed as: (0, 0), (1, 0), (1, 1) , (0,1). Of course, according to actual needs, the reference point can also be selected from other positions in the QR code image, such as the center point of the QR code image, etc. The construction method of the real coordinate system can also be adjusted according to actual needs, such as: A point outside the two-dimensional code image is used as a coordinate origin to construct a real coordinate system, etc., which is not specifically limited in the embodiment of this specification.

It should be noted that the method for constructing the real coordinate system of different AR devices in the embodiments of this specification is the same, which can ensure that the real coordinate information of the reference point in different AR devices is the same.

If multiple two-dimensional code images have the same size, as shown in Figure 2, if four corner points of the two-dimensional code image are selected as reference points, different AR devices can scan two-dimensional code images of the same size. Obtain the image information of the preset pattern. At this time, the real coordinate information of the reference points in the two-dimensional code images scanned by different AR devices can be considered the same.

S6. Determine a conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device based on the image coordinate information and the real coordinate information.

Obtain the image coordinate data of the reference point in the image coordinate system of the AR device After the real coordinate information in the real coordinate system, the conversion relationship between the real coordinate system, the image coordinate system, and the AR coordinate system, that is, the device coordinate system, can be obtained by analyzing the image coordinate information and the real coordinate information of the reference point . For example, according to the real coordinate information of the reference point and the image coordinate information, how to offset and rotate the reference point can be calculated, and the real coordinate information of the reference point in the real coordinate system can be converted into the image coordinate system. Position to obtain the conversion relationship between the real coordinate system and the image coordinate system. At the same time, the AR device system can be used to obtain the conversion relationship between the AR coordinate system and the image coordinate system of the AR device. For example, the AR coordinate system and the image coordinate system of the AR device can be obtained through the view transformation matrix based on OpenGL (open graphics interface). The conversion relationship between the image coordinate systems can also be directly obtained through the ARKit API (Application Program Interface) to obtain the conversion relationship between the AR coordinate system of the AR device and the image coordinate system (for example, through The ARKit API is obtained using the view Matrix in AR Camera). ARKit can represent an augmented reality application software development kit launched by Apple, and API can represent a call interface left by the operating system to the application.

S8. Calibrate the device coordinate system according to the conversion relationship.

In an embodiment of this specification, the conversion relationship between the real coordinate system and the image coordinate system, and the conversion relationship between the AR coordinate system and the image coordinate system can be used to indirectly obtain the relationship between the AR coordinate system and the real coordinate system The conversion relationship of, realizes the calibration of the AR coordinate system of different AR equipment to the real coordinate system, and realizes the calibration of the coordinate system of different AR equipment.

For example, if the real coordinate system of an AR device is between the image coordinate system The conversion relationship of is: A×k=B, the conversion relationship between AR coordinate system and image coordinate system is: C×p=B, where A can represent the coordinate of pixel O in the real coordinate system, and B can represent The coordinates of the pixel O in the image coordinate system, C can represent the coordinates of the pixel O in the AR coordinate system, and k and p both represent conversion coefficients. The conversion relationship between AR coordinate system and real coordinate system can be expressed as: C=A×(k/p). The AR coordinate system of the AR device can be calibrated to the real coordinate system. This method can be used to calibrate the coordinate system of different AR devices to obtain the conversion relationship between the AR coordinate system and the real coordinate system of different AR devices to realize AR The calibration of the coordinate system of the device makes the AR coordinate system of different AR devices unified.

The coordinate system calibration method of the augmented reality device provided in the embodiment of this specification is based on the image information of the preset patterns in different AR devices, and the image coordinate system, the AR coordinate system, and the real coordinate system of the AR device can be obtained. According to the conversion relationship between various coordinate systems, the calibration of the AR coordinate system of different AR devices can be realized, so that different users can share the position and direction of the virtual object and get the same experience, which is the connection for subsequent AR applications Development provides a theoretical basis. The method provided by the embodiment of this description can perform AR coordinate system calibration without restricting the AR device to be set in the same position and maintaining the same posture. It is easy to operate and has strong practicability, and meets the needs of the coordinate system calibration of AR devices of users in different locations. .

On the basis of the foregoing embodiment, in an embodiment of this specification, the determining the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device includes: obtaining The device coordinate system of the augmented reality device is converted to the The device conversion matrix of the image coordinate system; according to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device, calculate the conversion of the real coordinate system to the image coordinate system Real conversion matrix; correspondingly, the method for calibrating the device coordinate system according to the conversion relationship includes: calibrating the device coordinate system according to the device conversion matrix and the real conversion matrix.

Specifically, when calculating the conversion relationship between the real coordinate system and the image coordinate system, an embodiment of this specification can use the obtained image coordinate information of the reference point, the real coordinate information, and the camera internal parameter matrix of the AR device, through Calculate the displacement, offset, rotation, etc. of the reference point to obtain the real conversion matrix between the real coordinate system and the image coordinate system. The camera internal parameter matrix can represent the transformation relationship from the camera coordinate system to the image plane coordinate system. The image plane coordinate system is expressed in pixel units, while the camera coordinate system (ie image coordinate system) is expressed in millimeters. Therefore, to complete the changing process, the linear relationship between the pixel unit of the image plane and the millimeter unit must be obtained first. The camera internal parameter matrix in the embodiment of this specification can be obtained from the parameters of the camera in the AR device.

In an embodiment of this specification, the real conversion matrix between the real coordinate system and the image coordinate system can be achieved through the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device, using SolvePnP The algorithm is calculated. SolvePnP can represent an algorithm for solving Camera rotation and translation provided in the OpenCV library. OpenCV can represent a library of programmed functions mainly for real-time computer vision.

The SolvePnP algorithm can calculate the posture of the camera of the AR device, and calculate the real conversion matrix between the real coordinate system and the image coordinate system.

At the same time, the AR device can be used to obtain the device conversion matrix between the AR coordinate system and the image coordinate system, and the real conversion matrix and the device conversion matrix can be used to calibrate the AR coordinate systems of different AR devices.

In an embodiment of this specification, the calibration of the device coordinate system according to the device conversion matrix and the real conversion matrix may include: implementing the first augmented reality device and the device according to the following formula (1) Calibration of the device coordinate system of the second augmented reality device: P _{cameraA_real} ^-1 P _{cameraA_AR} O _ARA =P _{cameraB_real} ^-1 P _{cameraB_AR} O _ARB (1)

In the above formula, P _{cameraA_real} can represent the real conversion matrix corresponding to the first augmented reality device, P _{cameraA_AR} can represent the device conversion matrix corresponding to the first augmented reality device, and O _ARA can represent that point O is at all For the coordinates in the device coordinate system of the first augmented reality device, P _{cameraB_real} may represent the real conversion matrix corresponding to the second augmented reality device, and P _{cameraB_AR} may represent the corresponding real conversion matrix of the second augmented reality device The device conversion matrix, O _ARB may represent the coordinate of point O in the device coordinate system of the second augmented reality device.

Suppose there are two AR devices A and B, and a point O in the real world is known, and the definition is as follows: the coordinate of point O in the image coordinate system of AR device A is O _cameraA , and point O is in the image of AR device B The coordinates in the image coordinate system are O _cameraB , the coordinates of the point O in the real coordinate system are O _real , the real coordinate system is transformed into the real conversion matrix of the image coordinate system of the AR device A is P _{cameraA_real} , and the real coordinate system is transformed into AR The real conversion matrix of the image coordinate system of device B is P _{cameraB_real} , the coordinate of point O in the AR coordinate system of AR device A is O _ARA , and the coordinate of point O in the AR coordinate system of AR device B is O _ARB , AR The device matrix of the AR coordinate system of the device A converted to the image coordinate system of the AR device A is P _{cameraA_AR} , and the device matrix of the AR coordinate system of the AR device B converted to the image coordinate system of the AR device B is P _{cameraB_AR} . Then the following equations can be obtained:

The above formula (1) can be obtained by transforming the above formula (2) into rows. Therefore, the real conversion matrix, AR coordinate system and image coordinate system between the real coordinate system and image coordinate system of different AR devices are calculated After the device conversion matrix between the devices, the coordinate information of point O can be calibrated using formula (1), which is simple and accurate, and realizes the coordinate system calibration of different AR devices.

On the basis of the foregoing embodiment, in an embodiment of this specification, the method for calibrating the device coordinate system further includes: if more than two augmented reality devices are used for the device coordinate system calibration, then The device coordinate system is calibrated based on the device coordinate system of any one of the augmented reality devices.

For example: if you need to calibrate the coordinate system of 3 AR devices, you can obtain the real conversion matrix between the real coordinate system and the image coordinate system corresponding to the 3 AR devices, and the difference between the AR coordinate system and the image coordinate system The device conversion matrix can be based on the AR coordinate system of any one of the AR devices to obtain the conversion relationship between the AR coordinate system of the other two AR devices and the AR coordinate system, for example, the following formula (3) can be obtained:

The AR coordinate system of AR device B and C is calibrated to the AR coordinate system of AR device A, and the coordinate system of three AR devices is calibrated.

Fig. 3 is a schematic flowchart of a method for calibrating the coordinate system of an augmented reality device in another embodiment of this specification. The following will specifically introduce the calibration process of the coordinate system of the AR device in the embodiment of this specification with reference to Fig. 3:

(1) AR device initialization. AR device startup requires initialization.

(2) Process the frame data in the camera of the AR device. The user can use the AR device to scan the preset QR code image. After scanning, the frame data of the image information in the camera of the AR device can be processed.

(3) Identify whether there is a two-dimensional code image in the image information of the AR device, if so, perform step (4), if not, return to step (2) until the two-dimensional code image is recognized.

(4) Obtain the four corner points in the two-dimensional code image in the image coordinate system Like coordinate information, and the real coordinate information of the four corner points in the two-dimensional code image in the real coordinate system.

(5) According to the image coordinate information of the two-dimensional image, the real coordinate information, and the camera internal parameter matrix of the AR device, the SolvePnP algorithm is used to calculate the camera pose of the AR device, and the real coordinate system is converted into the image coordinate system of the AR device Real conversion matrix.

(6) Obtain the AR space view transformation matrix of the AR device, that is, the device transformation matrix for converting the AR coordinate system of the AR device into the image coordinate system.

(7) Use the above formula (1) to calculate the conversion relationship of the AR coordinate system of different AR devices, and complete the calibration of the coordinate system of the AR device.

The method provided in the embodiments of this specification can realize the calibration of the AR coordinate system of different AR devices based on the image information of the preset patterns in different AR devices, so that different users can share the position and direction of the virtual object to obtain the same Experience makes it possible to interact with AR experience among multiple users, and provides a theoretical basis for subsequent development of AR applications. The method provided by the embodiment of this description can perform AR coordinate system calibration without restricting the AR device to be set in the same position and the same posture. It is easy to operate and has strong practicability, and meets the needs of the coordinate system calibration of AR devices of users in different locations. Improve the user experience.

The various embodiments of the above method in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. For related details, please refer to the partial description of the method embodiment.

Based on the coordinate system calibration method of augmented reality equipment described above, this One or more embodiments of the specification also provide a coordinate system calibration device for augmented reality equipment. The described devices may include systems (including distributed systems), software (applications), modules, components, servers, clients, etc., which use the methods described in the embodiments of this specification, combined with necessary implementation hardware devices. Based on the same innovative concept, the devices in one or more embodiments provided in the embodiments of this specification are as described in the following embodiments. Since the implementation scheme of the device to solve the problem is similar to the method, the implementation of the specific device in the embodiment of this specification can refer to the implementation of the foregoing method, and the repetition will not be repeated. As used below, the term "unit" or "module" can be a combination of software and/or hardware that can implement predetermined functions. Although the device described in the following embodiments is preferably implemented by software, the implementation of hardware or a combination of software and hardware is also possible and conceived.

Specifically, FIG. 4 is a schematic diagram of the module structure of an embodiment of the coordinate system calibration device of the augmented reality device provided in this specification. As shown in FIG. 4, the coordinate system calibration device of the augmented reality device provided in this specification Including: image information acquisition module 41, coordinate information acquisition module 42, conversion relationship determination module 43, equipment coordinate system calibration module 44, of which: image information acquisition module 41 can be used to use augmented reality The device scans the preset pattern to obtain the image information of the preset pattern in the augmented reality device; the coordinate information obtaining module 42 can be used to obtain the reference point in the image information in the augmented reality device. The image coordinate information in the image coordinate system of the device, and the real coordinate information of the reference point in the real coordinate system; the conversion relationship determination module 43 can be used to determine the image coordinates based on the image coordinate information. Information, the real coordinate information, determine the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device; the device coordinate system calibration module 44 can be used for The device coordinate system is calibrated according to the conversion relationship.

The coordinate system calibration device of the augmented reality device provided in the embodiment of this specification can obtain the image coordinate system, AR coordinate system, and real coordinate system of the AR device based on the image information of the preset patterns in different AR devices. According to the conversion relationship between various coordinate systems, the calibration of the AR coordinate system of different AR devices can be realized, so that different users can share the position and direction of the virtual object and get the same experience, which is the connection for subsequent AR applications Development provides a theoretical basis. The method provided by the embodiment of this description may not restrict the AR device to be set in the same position and the same posture to perform the AR coordinate system calibration, which is convenient to operate and has strong practicability, and meets the needs of the coordinate system calibration of AR devices of users in different locations.

On the basis of the foregoing embodiment, the coordinate information acquisition module includes a real coordinate acquisition unit, and the real coordinate acquisition unit is specifically configured to: determine the calibration point in the image information; The relative position between the reference point and the calibration point determines the real coordinate information of the reference point in the real coordinate system.

On the basis of the foregoing embodiment, the real coordinate acquiring unit is further used for: Select one of the reference points as the calibration point; take the calibration point as the origin of the coordinate, and select the line connecting the two reference points outside the calibration point and the calibration point as the coordinate axis to construct the The real coordinate system; according to the relative position between the reference point and the calibration point in the preset pattern, determine the real coordinate information of the reference point in the real coordinate system.

The coordinate system calibration device of the augmented reality equipment provided in the embodiment of the specification obtains the coordinate information of the reference point in the real coordinate system based on the actual size of the preset pattern and the relative position between the reference point and the calibration point. Since different ARs use preset patterns of the same size, the coordinate system of the AR device can be calibrated based on the real coordinate information of the reference points in the preset patterns.

FIG. 5 is a schematic structural diagram of a device coordinate system calibration module in an embodiment of the present description. As shown in FIG. 5, on the basis of the foregoing embodiment, the conversion relationship determination module 43 includes: a device conversion matrix calculation unit 51, It can be used to obtain the device conversion matrix of the device coordinate system of the augmented reality device converted into the image coordinate system; the real conversion matrix calculation unit 52 can be used to obtain the image coordinate information, the Real coordinate information, the camera internal parameter matrix of the augmented reality device, and calculate the real conversion matrix for transforming the real coordinate system into the image coordinate system; accordingly, the device coordinate system calibration module 44 can be used for According to the The device conversion matrix and the real conversion matrix are used to calibrate the device coordinate system.

In the embodiment of this specification, by using the real conversion matrix between the image coordinate system and the real coordinate system of the AR device, and the conversion matrix between the AR coordinate system and the image coordinate system, the difference between the AR coordinate systems between different AR devices can be obtained. The conversion relationship provides an accurate theoretical basis for the calibration of the coordinate system of the AR device.

On the basis of the above-described embodiment, the device coordinate system calibration module is configured to: implement a first amplification calibration device coordinate system and a second device reality augmented reality device according to the following _equation: P cameraA_real ^{- 1} P _{cameraA_AR} O _ARA =P _{cameraB_real} ^-1 P _{cameraB_AR} O _ARB

In the above formula, P _{cameraA_real} represents the real conversion matrix corresponding to the first augmented reality device, P _{cameraA_AR} represents the device conversion matrix corresponding to the first augmented reality device, and O _ARA represents the point O in the first augmented reality device. amplification device coordinate system reality device _coordinates, P cameraB_real represents the second augmented reality device corresponding to the true transformation _matrix, P cameraB_AR augmented reality represents the second device corresponds to the device transformation matrix, O _ARB represents the coordinate of point O in the device coordinate system of the second augmented reality device.

The embodiments of this specification provide the coordinate system conversion formulas of different AR devices, which can accurately calibrate the coordinate systems of each AR device.

On the basis of the foregoing embodiment, the device coordinate system calibration module is also used for: if more than two augmented reality devices perform device coordinate system calibration, then The device coordinate system is calibrated based on the device coordinate system of any one of the augmented reality devices.

On the basis of the foregoing embodiment, the coordinate system calibration processing unit is specifically configured to: use the SolvePnP algorithm according to the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device, The real conversion matrix is obtained by calculation.

In the embodiment of this specification, the SolvePnP algorithm can be used to calculate the posture of the camera in the AR device and obtain the real conversion matrix between the real coordinate system of the AR device and the image coordinate system. The method is simple and accurate.

On the basis of the above embodiment, the preset pattern in the image information of the preset pattern acquired by the image information acquiring module includes at least one of a two-dimensional code and a barcode.

In the embodiments of this specification, two-dimensional codes, barcodes, and other flat patterns that have characteristics and are relatively common in daily life are used as preset patterns, which are convenient for guiding users to scan, easy to operate, and improve user experience.

It should be noted that the above-mentioned device may also include other implementation manners according to the description of the method embodiment. For specific implementation manners, reference may be made to the description of the related method embodiments, which will not be repeated here.

In one embodiment of this specification, a computer storage medium may also be provided, on which a computer program is stored. When the computer program is executed, the video data processing method in the above-mentioned embodiment is implemented. For example, the following method may be implemented: The reality device scans the preset pattern to obtain image information of the preset pattern in the augmented reality device; Obtain the image coordinate information of the reference point in the image information in the image coordinate system of the augmented reality device, and the real coordinate information of the reference point in the real coordinate system; based on the image The coordinate information, the real coordinate information, determine the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device; according to the conversion relationship, the device coordinate system Department for calibration.

The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the attached patent application. In some cases, the actions or steps described in the scope of the patent application may be performed in a different order from the embodiment and still achieve desired results. In addition, the process depicted in the drawings does not necessarily require a specific order or sequential order of display to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

The methods or devices described in the above-mentioned embodiments provided in this specification can implement business logic through a computer program and record them on a storage medium, and the storage medium can be read and executed by a computer to achieve the effects of the solutions described in the embodiments of this specification.

The coordinate system calibration method or device of the augmented reality device provided in the embodiments of this specification can be implemented by the processor in the computer by executing the corresponding program instructions, such as using the c++ language of the windows operating system to be implemented on the PC side or the linux system , Or other such as the use of android, iOS system programming language in smart terminals, and quantum computer-based processing logic implementation. The coordinate system calibration of an augmented reality device provided in this manual In an embodiment of the barebones system, FIG. 6 is a schematic diagram of the module structure of an embodiment of the coordinate system calibration system for augmented reality equipment provided in this specification. As shown in FIG. 6, the augmented reality provided by the embodiment of this specification The coordinate system calibration system of the equipment may include a processor 61 and a memory 62 for storing executable instructions of the processor. The processor 61 and the memory 62 communicate with each other through the bus 63; the processor 61 is used for calling The program instructions in the storage 62 to execute the methods provided in the embodiments of the above-mentioned seismic data processing methods, for example, include: scanning a preset pattern with an augmented reality device to obtain the preset pattern in the augmented reality device Image information of the pattern; acquiring the image coordinate information of the reference point in the image information in the image coordinate system of the augmented reality device, and the real coordinate information of the reference point in the real coordinate system Based on the image coordinate information and the real coordinate information, determine the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device, according to the conversion The relationship calibrates the coordinate system of the device.

It should be noted that the device, computer storage medium, and system described above in the specification may also include other implementation manners according to the description of the related method embodiments. For specific implementation manners, refer to the description of the method embodiments, which will not be repeated here.

The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. Especially for hard As far as the body+program embodiment is concerned, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiment.

The embodiments of this specification are not limited to the conditions described in one or more embodiments of this specification that must comply with industry communication standards, standard computer data processing and data storage rules. Certain industry standards or implementations described in custom methods or examples with slight modifications can also achieve the same, equivalent or similar implementation effects of the foregoing examples, or predictable implementation effects after modification. Examples obtained by applying these modified or deformed data acquisition, storage, judgment, processing methods, etc., can still fall within the scope of optional implementation schemes of the embodiments of the present specification.

In the 1990s, the improvement of a technology can be clearly distinguished between hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (for method flow Improve). However, with the development of technology, the improvement of many methods and processes of today can be regarded as a direct improvement of the hardware circuit structure. Designers almost always get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by the hardware entity module. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. It is programmed by the designer to "integrate" a digital system on a PLD without requiring the chip manufacturer to design and manufacture a dedicated integrated circuit Road chip. Moreover, nowadays, instead of manually making integrated circuit chips, this programming is mostly realized by using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation must also be written in a specific programming language, which is called Hardware Description Language (HDL), and HDL is not only one, but there are many, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., Currently the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be clear that only need to logically program the method flow in the above hardware description languages and program it into an integrated circuit, the hardware circuit that implements the logic method flow can be easily obtained.

The controller can be implemented in any suitable manner. For example, the controller can be a microprocessor or a processor and a computer readable program code (such as software or firmware) that can be executed by the (micro) processor. Media, logic gates, switches, Application Specific Integrated Circuits (ASIC), programmable logic controllers and embedded microcontrollers. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D , Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the storage controller can also be implemented as part of the storage control logic. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, it is completely possible to make the controller use logic gates, switches, integrated circuits for special applications, and programmable logic by logically programming method steps. Controllers and embedded microcontrollers can realize the same function. Therefore, such a controller can be regarded as a hardware component, and the devices included in it for realizing various functions can also be regarded as a structure within the hardware component. Or even, the device for realizing various functions can be regarded as both a software module for realizing the method and a structure in a hardware component.

The systems, devices, modules or units explained in the above embodiments may be implemented by computer chips or entities, or implemented by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a vehicle-mounted human-computer interaction device, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, Tablets, wearable devices, or any combination of these devices.

Although one or more embodiments of this specification provide method operation steps as described in the embodiments or flowcharts, conventional or non-inventive means may include more or fewer operation steps. The sequence of steps listed in the embodiment is only one way of the execution sequence of the steps, and does not represent the only execution sequence. When the actual device or terminal product is executed, it can be executed sequentially or in parallel according to the method shown in the embodiment or the diagram (for example, in a parallel processor or multi-threaded processing environment, or even distributed Data processing environment). The terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, product, or device that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed Elements, or also include elements inherent to such processes, methods, products, or equipment. If there are no more restrictions, it does not exclude that there are other identical or equivalent elements in the process, method, product, or device including the elements. Words such as first and second are used to denote names, but do not denote any specific order.

For the convenience of description, when describing the above device, the functions are divided into various modules and described separately. Of course, when implementing one or more of this specification, the functions of each module can be implemented in the same one or more software and/or hardware, or the modules that implement the same function can be composed of multiple sub-modules or sub-units. The combination of realization and so on. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation. For example, multiple units or components can be combined or integrated into Another system, or some features can be ignored, or not implemented. On the other hand, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may be electrical, mechanical or other forms.

The present invention is described with reference to flowcharts and/or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to general-purpose computers, The processor of a dedicated computer, embedded processor, or other programmable data processing equipment is used to generate a machine so that the instructions executed by the processor of the computer or other programmable data processing equipment are used to implement a process or Multiple processes and/or block diagrams A device with functions specified in one block or multiple blocks.

These computer program instructions can also be stored in a computer-readable storage that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable storage produce a manufactured product including the instruction device , The instruction device realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to generate computer-implemented processing, so that the computer or other programmable equipment The instructions executed above provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.

In a typical configuration, the computing device includes one or more processors (CPU), input/output interfaces, network interfaces, and memory.

Memory may include non-permanent memory in computer readable media, random access memory (RAM) and/or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

Computer-readable media includes permanent and non-permanent, removable and non-removable media, and information storage can be realized by any method or technology. Information can So it is computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), and other types of random access memory (RAM) , Read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only CD-ROM, digital multi-function Optical discs (DVD) or other optical storage, magnetic cassettes, magnetic tape storage, graphene storage or other magnetic storage devices or any other non-transmission media can be used to store information that can be accessed by computing devices. According to the definition in this article, computer-readable media does not include transient computer-readable media (transitory media), such as modulated data signals and carrier waves.

Those skilled in the art should understand that one or more embodiments of this specification can be provided as a method, a system or a computer program product. Therefore, one or more embodiments of this specification may adopt the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, one or more embodiments of this specification can be implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. In the form of a computer program product.

One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Can also be divided One or more embodiments of this specification are practiced in a distributed computing environment. In these distributed computing environments, remote processing devices connected through a communication network perform tasks. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the part of the description of the method embodiment. In the description of this specification, descriptions with reference to terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean specific features described in conjunction with the embodiment or example , Structure, materials or features are included in at least one embodiment or example of this specification. In this specification, the schematic representations of the aforementioned terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples and the features of the different embodiments or examples described in this specification under the condition of not conflicting each other.

The above descriptions are only examples of one or more embodiments of this specification, and are not used to limit one or more embodiments of this specification. For those skilled in the art, one or more embodiments of this specification can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the patent application. Within the surrounding.

Claims (16)

A method for calibrating the coordinate system of an augmented reality device includes: scanning a preset pattern by the augmented reality device to obtain image information of the preset pattern in the augmented reality device; The image coordinate information of the reference point in the image coordinate system of the augmented reality device, and the real coordinate information of the reference point in the real coordinate system; based on the image coordinate information and the real coordinate information , Determine the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device; calibrate the device coordinate system according to the conversion relationship; wherein, the The preset pattern includes at least one of a two-dimensional code and a barcode. According to the method described in item 1 of the scope of patent application, the method for obtaining the real coordinate information of the reference point in the real coordinate system includes: determining the calibration point in the image information; according to the reference in the preset pattern The relative position between the point and the calibration point determines the real coordinate information of the reference point in the real coordinate system. According to the method described in item 2 of the scope of patent application, the determining the calibration point in the image information includes: Selecting one of the reference points as the calibration point; accordingly, the determining the real coordinate information of the reference point in the real coordinate system includes: taking the calibration point as the origin of the coordinate, and selecting the calibration point The straight line connecting the two reference points outside the fixed point and the calibration point is used as the coordinate axis to construct the real coordinate system; according to the relative position between the reference point and the calibration point in the preset pattern To determine the real coordinate information of the reference point in the real coordinate system. For the method described in item 1 of the scope of the patent application, the determining the conversion relationship between the real coordinate system, the image coordinate system, and the device coordinate system of the augmented reality device includes: obtaining the extension The device coordinate system of the augmented reality device is converted into a device conversion matrix of the image coordinate system; based on the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device, calculate The real coordinate system is transformed into the real conversion matrix of the image coordinate system; accordingly, the method for calibrating the device coordinate system according to the conversion relationship includes: according to the device conversion matrix, the real The conversion matrix is used to calibrate the coordinate system of the device. According to the method described in item 4 of the scope of patent application, the calibration of the device coordinate system according to the device conversion matrix and the real conversion matrix includes: implementing the first augmented reality device according to the following formula Calibration with the device coordinate system of the second augmented reality device: P _{cameraA_real} ^-1 P _{cameraA_AR} O _ARA = P _{cameraB_real} ^-1 P _{cameraB_AR} O _{ARB In the} above formula, P _{cameraA_real} represents the corresponding first augmented reality device Real conversion matrix, P _{cameraA_AR} represents the device conversion matrix corresponding to the first augmented reality device, O _ARA represents the coordinate of point O in the device coordinate system of the first augmented reality device, and P _{cameraB_real} represents the The real conversion matrix corresponding to the second augmented reality device, P _{cameraB_AR} represents the device conversion matrix corresponding to the second augmented reality device, O _ARB represents the point O in the device coordinate system of the second augmented reality device Coordinates in. For the method described in item 5 of the scope of patent application, the method of calibrating the device coordinate system according to the conversion relationship further includes: if more than two augmented reality devices are used for device coordinate system calibration, then The device coordinate system of any augmented reality device is used as a reference to calibrate the device coordinate system. According to the method described in item 4 of the scope of patent application, the calculation of the real coordinate system is transformed into the said image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device The real conversion matrix of the image coordinate system includes: according to the image coordinate information, the real coordinate information, the expansion The camera internal parameter matrix of the augmented reality device is calculated to obtain the real conversion matrix using the SolvePnP algorithm. A coordinate system calibration device for augmented reality equipment includes: an image information acquisition module for scanning a preset pattern with the augmented reality equipment to obtain image information of the preset pattern in the augmented reality equipment The coordinate information acquisition module is used to acquire the image coordinate information of the reference point in the image information in the image coordinate system of the augmented reality device, and the reference point in the real coordinate system Real coordinate information; a conversion relationship determination module for determining the real coordinate system, the image coordinate system, and the device coordinates of the augmented reality device based on the image coordinate information and the real coordinate information The conversion relationship between the systems; the device coordinate system calibration module is used to calibrate the device coordinate system according to the conversion relationship; wherein the image information acquisition module obtains the image of the preset pattern The preset pattern in the image information includes at least one of a two-dimensional code and a barcode. For the device described in item 8 of the scope of patent application, the coordinate information acquisition module includes a real coordinate acquisition unit, and the real coordinate acquisition unit is specifically configured to: determine the calibration point in the image information; Set the relative position between the reference point and the calibration point in the pattern, and determine the real position of the reference point in the real coordinate system Standard information. As in the device described in item 9 of the scope of patent application, the true coordinate acquisition unit is further used to: select one of the reference points as the calibration point; use the calibration point as the coordinate origin to select one of the calibration points The straight line connecting the two outer reference points and the calibration point is used as the coordinate axis to construct the real coordinate system; according to the relative position between the reference point and the calibration point in the preset pattern, determine The real coordinate information of the reference point in the real coordinate system. According to the device described in item 8 of the scope of patent application, the conversion relationship determination module includes: a device conversion matrix calculation unit for obtaining the device coordinate system of the augmented reality device and converting it into the image coordinate System of equipment conversion matrix; a real conversion matrix calculation unit for calculating the conversion of the real coordinate system to the said image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device The real conversion matrix of the image coordinate system; accordingly, the device coordinate system calibration module is used to calibrate the device coordinate system according to the device conversion matrix and the real conversion matrix. For the device described in item 11 of the scope of the patent application, the device coordinate system calibration module is specifically used to implement the calibration of the device coordinate system of the first augmented reality device and the second augmented reality device according to the following formula : P _{cameraA_real} ^-1 P _{cameraA_AR} O _ARA = P _{cameraB_real} ^-1 P _{cameraB_AR} O _{ARB In the} above formula, P _{cameraA_real} represents the real conversion matrix corresponding to the first augmented reality device, and P _{cameraA_AR} represents the first augmented reality device. The device conversion matrix corresponding to the environment device, O _ARA represents the coordinate of point O in the device coordinate system of the first augmented reality device, P _{cameraB_real} represents the real conversion matrix corresponding to the second augmented reality device, P _{cameraB_AR} represents the device conversion matrix corresponding to the second augmented reality device, and O _ARB represents the coordinate of the point O in the device coordinate system of the second augmented reality device. For the device described in item 12 of the scope of patent application, the device coordinate system calibration module is also used to: if there are more than two augmented reality devices for performing device coordinate system calibration, use any one of the augmented reality devices The equipment coordinate system of is used as the reference to calibrate the equipment coordinate system. As for the device described in item 11 of the scope of patent application, the coordinate system calibration processing unit is specifically configured to: use the image coordinate information, the real coordinate information, and the camera internal parameter matrix of the augmented reality device The SolvePnP algorithm calculates and obtains the real conversion matrix. A computer storage medium has a computer program stored thereon, and when the computer program is executed, the method described in any one of items 1 to 7 of the scope of patent application is realized. A coordinate system calibration system for augmented reality equipment includes at least one processor and a memory for storing executable instructions of the processor. When the processor executes the instructions, any of items 1 to 7 of the scope of patent application can be realized The method described in one item.

Images