CN117372531A

CN117372531A - Calibration method and device of head-mounted equipment and electronic equipment

Info

Publication number: CN117372531A
Application number: CN202210774826.8A
Authority: CN
Inventors: 郑光璞
Original assignee: Beijing Zitiao Network Technology Co Ltd
Current assignee: Beijing Zitiao Network Technology Co Ltd
Priority date: 2022-07-01
Filing date: 2022-07-01
Publication date: 2024-01-09

Abstract

The disclosure provides a calibration method, a device and electronic equipment of head-mounted equipment, which relate to data processing and are used for solving the problem that in the prior art, when a calibrator calibrates VR equipment, experience of the calibrator is relied on, so that the calibration precision of the VR equipment is lower. The method comprises the following steps: acquiring a mechanical arm script for fixing a mechanical arm of the head-mounted equipment to be calibrated; acquiring a calibration image based on a mechanical arm script; determining the actual distance between each target characteristic point in the calibration image and each target characteristic point except the target characteristic point according to the calibration parameters of the head-mounted equipment to be calibrated and the calibration image; and determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance and the actual distance between each target characteristic point and each target characteristic point except the target characteristic points.

Description

Calibration method and device of head-mounted equipment and electronic equipment

Technical Field

The disclosure relates to the technical field of data processing, and in particular relates to a calibration method and device of head-mounted equipment and electronic equipment.

Background

At present, in a calibration scenario of a head-mounted device, when the head-mounted device is shipped, the head-mounted device needs to be calibrated, for example: when the head-mounted device is a Virtual Reality (VR) device, a calibrator typically calibrates the VR device according to his own experience. Thus, different calibration personnel rely on the experience of the calibration personnel when calibrating the VR device, resulting in lower accuracy of the calibration of the VR device.

Disclosure of Invention

In view of the above, the present disclosure provides a method, an apparatus, and an electronic device for calibrating a head-mounted device, which are used for solving the problem in the prior art that when a calibrator calibrates a VR device, depending on experience of the calibrator, calibration accuracy of the VR device is low.

In order to achieve the above object, the present disclosure provides the following technical solutions:

in a first aspect, the present disclosure provides a method of calibrating a head-mounted device, comprising: acquiring a mechanical arm script for fixing a mechanical arm of the head-mounted equipment to be calibrated; acquiring a calibration image based on a mechanical arm script; determining the actual distance between each target characteristic point in the calibration image and each target characteristic point except the target characteristic point according to the calibration parameters of the head-mounted equipment to be calibrated and the calibration image; and determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance and the actual distance between each target characteristic point and each target characteristic point except the target characteristic points.

As an optional embodiment of the present disclosure, obtaining a robot arm script for fixing a robot arm of a to-be-calibrated head-mounted device includes: acquiring equipment information of the head-mounted equipment to be calibrated; and determining a mechanical arm script for fixing the mechanical arm of the head-mounted device to be calibrated according to the device information.

As an alternative embodiment of the present disclosure, the device information includes a device model; according to the device information, determining a mechanical arm script for fixing the mechanical arm of the to-be-calibrated head-mounted device comprises: inquiring a target script corresponding to the equipment information in a preset corresponding relation according to the equipment model; and determining the mechanical arm script for fixing the mechanical arm of the to-be-calibrated head-mounted device as the target script according to the target script.

As an optional embodiment of the disclosure, acquiring a calibration image based on a robot arm script includes: according to the mechanical arm script, controlling the mechanical arm to move at least one target position; and controlling the head-mounted device to be calibrated to shoot the calibration plate at each target position, and acquiring a calibration image shot by the head-mounted device to be calibrated.

As an optional embodiment of the disclosure, determining, according to calibration parameters of the headset to be calibrated and the calibration image, an actual distance between each target feature point and each target feature point other than the target feature point includes: determining world coordinates corresponding to each target feature point in the calibration image according to calibration parameters of the head-mounted equipment to be calibrated and the calibration image; the actual distance between each target feature point in the calibration image and each target feature point other than the target feature point is determined according to the world coordinates.

As an optional embodiment of the present disclosure, determining a calibration result of the to-be-calibrated head-mounted device according to a theoretical distance and an actual distance between each target feature point and each target feature point other than the target feature point includes: determining an actual difference value between each target feature point and each target feature point except the target feature point according to the theoretical distance and the actual distance between each target feature point and each target feature point except the target feature point; and determining a calibration result of the head-mounted device to be calibrated according to the actual difference value.

As an optional embodiment of the disclosure, determining a calibration result of the to-be-calibrated head-mounted device according to the actual difference value includes: under the condition that the actual difference value is smaller than or equal to a preset threshold value, determining that the calibration result of the head-mounted equipment to be calibrated is normal in precision; and under the condition that the actual difference value is larger than a preset threshold value, determining that the calibration result of the to-be-calibrated head-mounted equipment is abnormal in accuracy.

As an optional embodiment of the present disclosure, after determining a calibration result of the to-be-calibrated headset according to a theoretical distance and an actual distance between each target feature point and each target feature point other than the target feature point, the calibration method of the headset provided by the present disclosure further includes: resetting the mechanical arm.

In a second aspect, the present disclosure provides a calibration apparatus for a head-mounted device, comprising: the acquisition unit is used for acquiring a mechanical arm script for fixing the mechanical arm of the head-mounted equipment to be calibrated; the processing unit is used for acquiring a calibration image based on the mechanical arm script acquired by the acquisition unit; the processing unit is also used for determining the actual distance between each target characteristic point in the calibration image and each target characteristic point except the target characteristic point according to the calibration parameters of the head-mounted device to be calibrated and the calibration image; the processing unit is further used for determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance and the actual distance between each target feature point and each target feature point except the target feature point.

As an optional implementation manner of the present disclosure, an acquiring unit is specifically configured to acquire device information of a to-be-calibrated head-mounted device; the processing unit is specifically configured to determine, according to the device information acquired by the acquiring unit, a mechanical arm script for fixing a mechanical arm of the to-be-calibrated head-mounted device.

As an alternative embodiment of the present disclosure, the device information includes a device model; the processing unit is specifically used for inquiring the target script corresponding to the equipment information in the pre-configured corresponding relation according to the equipment model acquired by the acquisition unit; the processing unit is specifically configured to determine, according to the target script, a mechanical arm script for fixing the mechanical arm of the to-be-calibrated head-mounted device as the target script.

As an optional implementation manner of the disclosure, the processing unit is specifically configured to control the mechanical arm to move at least one target position according to the mechanical arm script acquired by the acquiring unit; the processing unit is specifically used for controlling the head-mounted device to be calibrated to shoot the calibration plate at each target position and acquiring a calibration image shot by the head-mounted device to be calibrated.

As an optional implementation manner of the disclosure, the processing unit is specifically configured to determine world coordinates corresponding to each target feature point in the calibration image according to calibration parameters of the head-mounted device to be calibrated and the calibration image; the processing unit is specifically configured to determine an actual distance between each target feature point in the calibration image and each target feature point except the target feature point according to the world coordinates.

As an optional embodiment of the disclosure, the processing unit is specifically configured to determine an actual difference value according to a theoretical distance and an actual distance between each target feature point and each target feature point except the target feature point; the processing unit is specifically configured to determine a calibration result of the head-mounted device to be calibrated according to the actual difference value.

As an optional implementation manner of the present disclosure, the processing unit is specifically configured to determine that the calibration result of the to-be-calibrated head-mounted device is normal in accuracy when the actual difference value is less than or equal to a preset threshold value; the processing unit is specifically configured to determine that the calibration result of the to-be-calibrated head-mounted device is abnormal in accuracy when the actual difference value is greater than a preset threshold value.

As an optional embodiment of the disclosure, the processing unit is further configured to reset the mechanical arm.

In a third aspect, the present disclosure provides an electronic device comprising: a memory and a processor, the memory for storing a computer program; the processor is configured to cause the electronic device to implement the method of calibrating a head mounted device as provided in the first aspect above, when executing the computer program.

In a fourth aspect, the present disclosure provides a computer readable storage medium having stored thereon a computer program which, when executed by a computing device, causes the computing device to implement a method of calibrating a head mounted device as provided in the first aspect above.

In a fifth aspect, the present disclosure provides a computer program product, characterized by causing a computer to implement the method of calibrating a head mounted device as provided in the first aspect above, when the computer program product is run on the computer.

It should be noted that the above-mentioned computer instructions may be stored in whole or in part on the first computer readable storage medium. The first computer readable storage medium may be packaged together with the processor of the calibration device of the head-mounted device, or may be packaged separately from the processor of the calibration device of the head-mounted device, which is not limited in this disclosure.

The descriptions of the second, third, fourth, and fifth aspects of the present disclosure may be referred to the detailed description of the first aspect; further, the advantageous effects described in the second aspect, the third aspect, the fourth aspect, and the fifth aspect may refer to the advantageous effect analysis of the first aspect, and are not described herein.

In the present disclosure, the names of the calibration means of the above-described head-mounted device do not constitute a limitation on the device or the functional module itself, which may appear under other names in a practical implementation. Insofar as the function of each device or function module is similar to the present disclosure, it is within the scope of the claims of the present disclosure and the equivalents thereof.

These and other aspects of the disclosure will be more readily apparent from the following description.

Compared with the prior art, the technical scheme provided by the disclosure has the following advantages:

when the head-mounted device to be calibrated is calibrated each time, the head-mounted device to be calibrated can be fixed on the mechanical arm, and the mechanical arm script of the mechanical arm is acquired, so that a calibration image can be quantitatively acquired. Then, according to calibration parameters of the head-mounted device to be calibrated and the calibration image, determining the actual distance between each target characteristic point in the calibration image and each target characteristic point except the target characteristic point; according to the theoretical distance and the actual distance between each target characteristic point and each target characteristic point except the target characteristic points, the calibration result of the to-be-calibrated head-mounted device is determined, and the accuracy of the calibration result is ensured.

In addition, when the headset device to be calibrated is a VR device, the VR device may be fixed on the mechanical arm at this time, and then a calibration image is acquired based on the mechanical arm script. Then, the calibration image is analyzed, so that a calibration result of the VR device can be obtained. The problem that the accuracy of the calibration of the VR device is low due to the fact that the experience of the calibrator is relied on when the calibrator calibrates the VR device in the prior art is solved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.

Fig. 1 is a schematic view of a scenario of a calibration method of a head-mounted device according to an embodiment of the disclosure;

FIG. 2 is a flow chart of a method for calibrating a headset according to an embodiment of the disclosure;

FIG. 3 is a second flowchart of a method for calibrating a headset according to an embodiment of the disclosure;

FIG. 4 is a third flow chart of a calibration method of a headset according to an embodiment of the disclosure;

FIG. 5 is a flowchart of a method for calibrating a headset according to an embodiment of the present disclosure;

FIG. 6 is a flowchart of a method for calibrating a headset according to an embodiment of the present disclosure;

fig. 7 is a schematic illustration of calculation of an actual position in a calibration method of a head-mounted device according to an embodiment of the disclosure;

FIG. 8 is a flowchart of a method for calibrating a headset according to an embodiment of the present disclosure;

FIG. 9 is a flowchart of a method for calibrating a headset according to an embodiment of the disclosure;

FIG. 10 is a flowchart eighth of a method for calibrating a headset according to an embodiment of the present disclosure;

fig. 11 is a schematic structural diagram of a calibration device of a head-mounted device according to an embodiment of the disclosure;

fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure;

fig. 13 is a schematic structural diagram of a computer program product of a calibration method of a head-mounted device according to an embodiment of the disclosure.

Detailed Description

In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.

It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Fig. 1 is a schematic view of a scenario of a calibration method of a head-mounted device according to an embodiment of the present disclosure, including: the calibration device comprises a calibration plate 1, a mechanical arm 2, a head-mounted device 3 to be calibrated and a main control device 4, wherein a clamp (holder) 2-1 is arranged on the mechanical arm 2, and the head-mounted device 3 to be calibrated is fixed on the clamp 2-1.

In some examples, the mechanical arm 2 and the to-be-calibrated head-mounted device 3 are both connected with the master control device 4 in a wired or wireless manner, so that the master control device 4 can control the mechanical arm 2 to move, and can control the to-be-calibrated head-mounted device 3 to shoot the calibration plate 1. Or, the mechanical arm 2 is connected with the master control device 4 in a wired or wireless manner, and the to-be-calibrated head-mounted device 3 is connected with the mechanical arm 2 in a wired or wireless manner, so that the master control device 4 can control the movement of the mechanical arm 2 and simultaneously control the to-be-calibrated head-mounted device 3 to shoot the calibration board 1.

For example, taking the connection of the mechanical arm 2 with the master control device 4 in a wired or wireless manner, the connection of the to-be-calibrated head-mounted device 3 with the mechanical arm 2 in a wired or wireless manner is illustrated, after the to-be-calibrated head-mounted device 3 is fixed on the fixture 2-1, the to-be-calibrated head-mounted device 3 needs to be powered on, so as to ensure that the to-be-calibrated head-mounted device 3 is powered on. After that, the master control device 4 obtains the mechanical arm script of the mechanical arm, and controls the mechanical arm 2 to run according to the mechanical arm script, and when the mechanical arm 2 runs to the target position, the master control device 4 controls the head-mounted device 3 to be calibrated to shoot the calibration board 1, and obtains the calibration image shot by the head-mounted device 3 to be calibrated. After acquiring the calibration image shot by the to-be-calibrated head-mounted device 3 at each target position, the main control device 4 determines the actual distance between any two target feature points in the calibration image, such as the actual distance between the target feature point A (corresponding to the calibration pattern A in the calibration plate) and the target feature point B (corresponding to the calibration pattern B in the calibration plate) in the calibration image according to the calibration parameters of the to-be-calibrated head-mounted device and the calibration image. Since the distance between the calibration pattern a and the calibration pattern B in the calibration plate 1 is known, the calibration result of the head-mounted device to be calibrated can be determined from the theoretical distance and the actual distance between any two target feature points.

Specifically, the mechanical arm 2 can move up, down, left and right and overturn, for example: nine-axis mechanical arm. The master device 4 may be an electronic device or a server, and the electronic device includes any one of a personal digital assistant (personal digital assistant, PDA) computer, a tablet computer, and a laptop computer (laptop computer). The head mounted device 3 to be calibrated may be any one of an augmented Reality (Augmented Reality, AR) device, VR device or Mixed Reality (MR) device, unmanned aerial vehicle, robot, autopilot, etc. that needs to be calibrated.

It should be noted that the above example is described taking an example in which the calibration board 1 includes only 2 target feature points, and the to-be-calibrated head-mounted device 3 includes 2 image capturing devices for capturing the current environment image. In other examples, the calibration board 1 includes 3 or more target feature points and/or the to-be-calibrated head-mounted device 3 includes 3 or more image capturing devices for acquiring the current environmental image, when determining the actual distance between any two target feature points in the calibration image, feature point matching is performed on the calibration image captured by any two image capturing devices, and then the actual distance between any two target feature points is calculated based on the calibration image of the matched feature points. And finally, determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance and the actual distance between any two target feature points.

For example, taking the main control device 4 as an example of an execution body for executing the method for calibrating the head-mounted device provided by the embodiment of the present disclosure, the method for calibrating the head-mounted device provided by the embodiment of the present disclosure is described, and a specific implementation process is as follows:

fig. 2 is a flowchart illustrating a method of calibrating a head mounted device, as shown in fig. 2, according to an exemplary embodiment, the method including the following S11-S14.

S11, acquiring a mechanical arm script for fixing the mechanical arm of the head-mounted device to be calibrated.

In some examples, the headset to be calibrated is fixed on the mechanical arm, so that the mechanical arm can be controlled to move to a specific point, the headset to be calibrated is controlled to shoot the calibration plate, and a calibration image is acquired. In this way, the actual distance between any two target feature points in the calibration image can be determined according to the calibration parameters of the head-mounted device to be calibrated and the calibration image, and the calibration result of the head-mounted device to be calibrated can be determined according to the theoretical distance and the actual distance between any two target feature points. Thus, a quantized index can be given for feeding back the calibration accuracy of the headset to be calibrated.

S12, acquiring a calibration image based on the mechanical arm script.

In some examples, different mechanical arm scripts correspond to different calibration plates, so that when the mechanical arm runs the mechanical arm scripts, the mechanical arm can move to the corresponding calibration plate, and the to-be-calibrated head-mounted device can shoot the corresponding calibration plate, so that a corresponding calibration image is obtained. Thus, the calibration image is obtained from a memory (such as a secure digital card (Secure Digital Memory Card, SD card)) of the headset to be calibrated.

S13, determining the actual distance between each target characteristic point of the calibration image and each target characteristic point except the target characteristic point according to the calibration parameters of the head-mounted device to be calibrated and the calibration image.

In some examples, in combination with the schematic scene diagram shown in fig. 1, the calibration image collected by the headset to be calibrated only includes 2 target feature points, namely, a target feature point a and a target feature point B, and only the actual distance between the target feature point a and the target feature point B needs to be calculated.

In other examples, when only 3 target feature points, namely, a target feature point a, a target feature point B and a target feature point C are included in the calibration image acquired by the headset to be calibrated, an actual distance 1 between the target feature point a and the target feature point B, an actual distance 2 between the target feature point a and the target feature point C, and an actual distance 3 between the target feature point B and the target feature point C need to be calculated. And then, determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance 1 between the target feature point A and the target feature point B, the theoretical distance 2 between the target feature point A and the target feature point C, the theoretical distance 3 between the target feature point B and the target feature point C, the actual distance 1, the actual distance 2 and the actual distance 3.

It can be seen that when the calibration image contains 3 or more target feature points, it is necessary to calculate the actual distance between each of the target feature points and each of the feature points other than the target feature point, and determine the calibration result of the head-mounted device to be calibrated based on the theoretical distance between each of the target feature points and each of the feature points other than the target feature point, and the actual distance.

Specifically, the calibration image includes at least 2 target feature points, for example, 3 target feature points, namely, a target feature point a, a target feature point B and a target feature point C, and when the selected target feature point a is the feature point to be currently calculated, each target feature point except the target feature point includes a target feature point B and a target feature point C. When the selected target feature point B is a feature point that is currently required to be calculated, then each target feature point other than the target feature point includes a target feature point a and a target feature point C. When the selected target feature point C is the feature point to be currently calculated, each target feature point except the target feature point includes a target feature point a and a target feature point B.

S14, determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance and the actual distance between each target feature point and each target feature point except the target feature points.

In some examples, in combination with the example given in S13 above, when only 2 target feature points are included in the calibration image, the calibration result of the to-be-calibrated head-mounted device is determined only according to the theoretical distance and the actual distance corresponding to the 2 target feature points. Such as: the actual difference is determined based on the absolute value of the difference between the theoretical distance (e.g., 2 meters) and the actual distance. And then, determining a calibration result according to the magnitude relation between the actual difference value and the preset threshold value. Or, in the corresponding relation between the pre-configured difference interval and the calibration result, inquiring the calibration result corresponding to the difference interval in which the actual difference value falls, and taking the calibration result corresponding to the difference interval as the calibration result of the to-be-calibrated head-mounted device.

In other examples, in combination with the example given in S13 above, when only 3 or more target feature points are included in the calibration image, for example, 3 target feature points are included in the calibration image, that is, the target feature point a, the target feature point B, and the target feature point C, respectively, an actual difference 1 between the theoretical distance 1 and the actual distance 1, an actual difference 2 between the theoretical distance 2 and the actual distance 2, and an actual difference 3 between the theoretical distance 3 and the actual distance 3 need to be calculated. And then, determining a calibration result of the to-be-calibrated head-mounted device according to the actual difference 1, the actual difference 2 and the actual difference 3. Such as: and determining a calibration result according to the magnitude relation between the actual difference 1, the actual difference 2 and the actual difference 3 and a preset threshold value. Or, in the corresponding relation between the pre-configured difference interval and the calibration result, inquiring the calibration result corresponding to the difference interval in which the tie value of the actual difference 1, the actual difference 2 and the actual difference 3 falls, and taking the calibration result corresponding to the difference interval as the calibration result of the to-be-calibrated head-mounted device.

The above example is described by taking the same calibration image of the target feature point included in the calibration plate as an example. In other examples, the calibration images of the target feature points contained in the calibration plate may be different to facilitate determination of the target feature points.

As can be seen from the foregoing, according to the calibration method of the head-mounted device provided by the embodiment of the disclosure, the head-mounted device to be calibrated is fixed on the mechanical arm, and the mechanical arm script of the mechanical arm is obtained, so that the calibration image can be quantitatively obtained. Then, according to calibration parameters of the head-mounted device to be calibrated and the calibration image, determining the actual distance between each target characteristic point in the calibration image and each target characteristic point except the target characteristic point; according to the theoretical distance and the actual distance between each target characteristic point and each target characteristic point except the target characteristic points, the calibration result of the to-be-calibrated head-mounted device is determined, and the accuracy of the calibration result is ensured.

As an alternative embodiment of the present disclosure, in connection with fig. 2, as shown in fig. 3, S11 may be specifically implemented by S110 and S111 described below.

S110, acquiring device information of the head-mounted device to be calibrated.

In some examples, the device information corresponding to the to-be-calibrated head-mounted device may be input to the master control device 4 by the user, or the master control device 4 may obtain the device information of the to-be-calibrated head-mounted device by capturing current product images of the to-be-calibrated head-mounted device at different angles and then importing the current product images into a pre-configured recognition model. The training process of the pre-configured recognition model is as follows:

acquiring a training sample image and a labeling result of the training sample image; the training sample image comprises a historical product image of the to-be-calibrated head-mounted device, and the marking result comprises device information of the to-be-calibrated head-mounted device.

The training sample image is input into a deep learning model.

And determining whether a prediction comparison result of the training sample image output by the deep learning model is matched with the labeling result or not based on the target loss function.

And when the prediction comparison result is not matched with the labeling result, repeatedly and circularly updating the network parameters of the deep learning model until the model converges to obtain a preconfigured identification model.

S111, determining a mechanical arm script for fixing the mechanical arm of the head-mounted device to be calibrated according to the device information.

Specifically, when the mechanical arm is only used for fixing the to-be-calibrated head-mounted device of one device information, the mechanical arm only corresponds to one target script. In this way, the target script can be directly acquired each time the headset to be calibrated is fixed on the mechanical arm, and the target script is used as the mechanical arm script for fixing the mechanical arm of the headset to be calibrated.

When the mechanical arm can be used for fixing two or more types of to-be-calibrated head-mounted devices with different device information, because the mechanical arm scripts corresponding to the to-be-calibrated head-mounted devices with different device information are different, the target scripts corresponding to the to-be-calibrated head-mounted devices with different device information are required to be stored in the memory of the master control device 4 in advance, so that after the to-be-calibrated head-mounted devices are fixed on the mechanical arm, the master control device 4 can search the memory for the target scripts corresponding to the device information according to the device information corresponding to the to-be-calibrated head-mounted devices, and then the target scripts are used as the mechanical arm scripts for fixing the mechanical arm of the to-be-calibrated head-mounted device.

As an alternative embodiment of the present disclosure, the device information includes a device model; referring to fig. 3, as shown in fig. 4, S111 described above may be implemented by the following S1110 and S1111.

S1110, inquiring the target script corresponding to the equipment information in the pre-configured corresponding relation according to the equipment model.

S1111, determining a mechanical arm script for fixing the mechanical arm of the to-be-calibrated head-mounted device as a target script according to the target script.

The above example is described by taking the device information including the device model number as an example. In some other examples, the device information may also be information that uniquely characterizes the headset to be calibrated, such as: the device identification code is not limited herein.

As an alternative embodiment of the present disclosure, in connection with fig. 2, as shown in fig. 5, S12 may be specifically implemented by S120 and S121 described below.

S120, controlling the mechanical arm to move at least one target position according to the mechanical arm script.

In some examples, when the to-be-calibrated head-mounted device is at the target position to shoot the calibration plate, all image acquisition devices, such as cameras, arranged on the to-be-calibrated head-mounted device can acquire the calibration patterns in the calibration plate at the same time. Wherein, a calibration pattern corresponds to a target feature point. Such as: in combination with the scene schematic diagram shown in fig. 1, the calibration board 1 contains 2 calibration patterns, namely a calibration pattern a and a calibration pattern B, when the to-be-calibrated head-mounted device is positioned at the target position to shoot the calibration board, all image acquisition devices arranged on the to-be-calibrated head-mounted device, such as a camera, can acquire the calibration pattern a and the calibration pattern B in the calibration board 1 at the same time, so that the subsequent calculation of the actual distance is facilitated.

For example, taking a scene schematic diagram given in connection with fig. 1, the to-be-calibrated head-mounted device includes 4 image acquisition devices, namely, an image acquisition device 0, an image acquisition device 1, an image acquisition device 2 and an image acquisition device 3, where the image acquisition device 0 and the image acquisition device 1 can be divided into a group, and denoted as 10; dividing the image acquisition device 2 and the image acquisition device 3 into a group, which is marked as 32;10 corresponds to a target position 1, so that the image acquisition device 0 and the image acquisition device 1 can acquire the calibration pattern a and the calibration pattern B in the calibration plate 1 at the same time. Similarly, 32 corresponds to a target position 1, so that the image acquisition device 2 and the image acquisition device 3 can both acquire the calibration pattern a and the calibration pattern B in the calibration plate 1 at the same time.

S121, controlling the head-mounted device to be calibrated to shoot a calibration plate at each target position, and acquiring a calibration image shot by the head-mounted device to be calibrated.

In some examples, when there is a difference between the image format of the calibration image captured by the head-mounted device to be calibrated and the image format recognizable by the master device 4, the image format of the calibration image captured by the head-mounted device to be calibrated needs to be converted, so that the master device 4 can directly recognize the calibration image.

Specifically, when the head-mounted device to be calibrated shoots the calibration plate, the calibration plate can be shot for a plurality of times (for example, 30 times), so that a plurality of calibration images can be obtained. And then, selecting an optimal calibration image from the plurality of calibration images for identification, so as to avoid the problem of inaccurate calibration results caused by fuzzy shot calibration images.

As an alternative embodiment of the present disclosure, in connection with fig. 2, as shown in fig. 6, the above S13 may be specifically implemented by the following S130 and S131.

S130, according to calibration parameters of the head-mounted device to be calibrated and the calibration image, determining world coordinates corresponding to each target feature point in the calibration image.

Specifically, the calibration parameters include a camera internal parameter and a camera external parameter, and the process of determining world coordinates corresponding to each target feature point in the calibration image according to the calibration parameters of the head-mounted device to be calibrated and the calibration image is as follows:

in combination with the example given in S120 and the schematic diagram given in fig. 7, it is assumed that the pixel coordinate corresponding to the target feature point a in the calibration image captured by cam0 is the pixel coordinate a, and the three-dimensional coordinate of the target feature point a under the camera coordinate system of cam0 is a-cam0 according to the camera internal reference of cam0 and the pixel coordinate a.

And according to camera external parameters between cam0 and cam1, converting the target feature point A into a corresponding three-dimensional coordinate A-cam1 under a camera coordinate system of cam1.

Assuming that the pixel coordinate corresponding to the target feature point A in the calibration image shot by the cam1 is the pixel coordinate A ', then determining the three-dimensional coordinate of the target feature point A under the camera coordinate system of the cam1 as A ' -cam1 by the camera internal reference and the pixel coordinate A ' of the cam1.

The optical center line of the point A-cam0 and cam0 forms a light ray L1, and the optical center line of the point A' -cam1 and cam1 forms a light ray L2. Thereafter, point_a is determined by the intersection of the light ray L1 and the light ray L2. Similarly, point_b can be obtained.

And then, calculating the Euclidean distance between the world coordinates corresponding to the point_A and the world coordinates corresponding to the point_B, and taking the Euclidean distance as the actual distance between the target feature Point A and the target feature Point B in the calibration image shot by cam 0.

It can be seen that only how the actual distance between the target feature point a and the target feature point B in the calibration image taken by cam0 is calculated is given here. The calculation method for calculating the actual distance between the target feature point a and the target feature point B in the calibration image shot by cam1, the actual distance between the target feature point a and the target feature point B in the calibration image shot by cam2, and the actual distance between the target feature point a and the target feature point B in the calibration image shot by cam3 are the same as the calculation method for calculating the actual distance between the target feature point a and the target feature point B in the calibration image shot by cam0, and will not be described again here.

It should be noted that: in calculating the intersection point of the light ray L1 and the light ray L2, it is necessary to ensure that the point a-cam0 and the point a' -cam1 are under the same camera coordinate system, such as: here, the cam1 coordinate system is turned to perform the processing of the intersection of the light ray L1 and the light ray L2. Therefore, the origin of L1 is the position of cam0 in the camera coordinate system of cam1, i.e. cam0 you consider it as a point, and its coordinates in cam1 are T0, T1, T2; the origin of L2 is the cam 1's own origin, as it is now under the cam 1's camera coordinate system.

S131, according to world coordinates, determining the actual distance between each target feature point in the calibration image and each target feature point except the target feature point.

As an alternative embodiment of the present disclosure, in connection with fig. 2, as shown in fig. 8, the above S14 may be specifically implemented by the following S140 and S141.

S140, determining the actual difference value between each target feature point and each target feature point except the target feature point according to the theoretical distance and the actual distance between each target feature point and each target feature point except the target feature point.

S141, determining a calibration result of the head-mounted device to be calibrated according to the actual difference value.

As an alternative embodiment of the present disclosure, in connection with fig. 8, as shown in fig. 9, the above S141 may be specifically implemented by the following S1410 and S1411.

S1410, determining that the calibration result of the to-be-calibrated head-mounted device is normal in accuracy under the condition that the actual difference value is smaller than or equal to a preset threshold value.

S1411, determining that the calibration result of the to-be-calibrated head-mounted device is abnormal in accuracy under the condition that the actual difference value is larger than a preset threshold value.

It can be seen that when the head-mounted device to be calibrated is calibrated through the calibration device of the head-mounted device provided by the embodiment of the disclosure, the quantization index of the positioning system precision before leaving the factory of the head-mounted device to be calibrated can be calculated, the calibration result of the head-mounted device to be calibrated can be intuitively measured, the shipment quality is improved, and the repair cost is reduced.

As an alternative implementation of the present disclosure, in conjunction with fig. 2, as shown in fig. 10, the calibration method of the head-mounted device provided by the embodiment of the present disclosure needs to execute S15 after S14 is executed.

S15, resetting the mechanical arm.

In some examples, S15 needs to be performed after S11-S14 described above are performed in order to be able to continue using the robot arm next time. Therefore, after the calibration result of the to-be-calibrated head-mounted device is obtained each time, the mechanical arm can be reset, so that the next calibration of the to-be-calibrated head-mounted device is facilitated.

The foregoing description of the solution provided by the embodiments of the present invention has been mainly presented in terms of a method. To achieve the above functions, it includes corresponding hardware structures and/or software modules that perform the respective functions. Those of skill in the art will readily appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as hardware or combinations of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

The embodiment of the invention can divide the functional modules of the calibration device of the head-mounted device according to the method example, for example, each functional module can be divided corresponding to each function, and two or more functions can be integrated in one processing module. The integrated modules may be implemented in hardware or in software functional modules. It should be noted that, in the embodiment of the present invention, the division of the modules is schematic, which is merely a logic function division, and other division manners may be implemented in actual implementation.

As shown in fig. 11, an embodiment of the present invention provides a schematic structural diagram of a calibration device 10 of a head-mounted device. The calibration device 10 of the head-mounted apparatus comprises an acquisition unit 101 and a processing unit 102.

The acquisition unit is used for acquiring a mechanical arm script for fixing the mechanical arm of the head-mounted equipment to be calibrated; the processing unit is used for acquiring a calibration image based on the mechanical arm script acquired by the acquisition unit; the processing unit is also used for determining the actual distance between each target characteristic point in the calibration image and each target characteristic point except the target characteristic point according to the calibration parameters of the head-mounted device to be calibrated and the calibration image; the processing unit is further used for determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance and the actual distance between each target feature point and each target feature point except the target feature point.

As an optional implementation manner of the disclosure, the processing unit is specifically configured to determine world coordinates of each target feature point in the calibration image according to calibration parameters of the headset to be calibrated and the calibration image; the processing unit is specifically configured to determine an actual distance between each target feature point in the calibration image and each target feature point except the target feature point according to the world coordinates.

All relevant contents of each step related to the above method embodiment may be cited to the functional descriptions of the corresponding functional modules, and their effects are not described herein.

Of course, the calibration device 10 of the head-mounted device provided in the embodiment of the present invention includes, but is not limited to, the above module, for example, the calibration device 10 of the head-mounted device may further include a storage unit 103. The memory unit 103 may be used for storing program code of the calibration means 10 of the head-mounted device, and may also be used for storing data generated during operation of the calibration means 10 of the head-mounted device, such as data in a write request or the like.

Fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, as shown in fig. 12, the electronic device may include: at least one processor 51, a memory 52, a communication interface 53 and a communication bus 54.

The following describes the respective constituent elements of the electronic device in detail with reference to fig. 12:

the processor 51 is a control center of the electronic device, and may be one processor or a collective term of a plurality of processing elements. For example, processor 51 is a central processing unit (Central Processing Unit, CPU), but may also be an integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits configured to implement embodiments of the present invention, such as: one or more DSPs, or one or more field programmable gate arrays (Field Programmable Gate Array, FPGAs).

In a particular implementation, processor 51 may include one or more CPUs, such as CPU0 and CPU1 shown in FIG. 12, as an example. Also, as one example, the electronic device 10 may include multiple processors, such as the processor 51 and the processor 56 shown in fig. 12. Each of these processors may be a Single-core processor (Single-CPU) or a Multi-core processor (Multi-CPU). A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (e.g., computer program instructions).

The Memory 52 may be, but is not limited to, a Read-Only Memory (ROM) or other type of static storage device that can store static information and instructions, a random access Memory (Random Access Memory, RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a compact disc (Compact Disc Read-Only Memory, CD-ROM) or other optical disk storage, optical disk storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory 52 may be stand alone and be coupled to the processor 51 via a communication bus 54. Memory 52 may also be integrated with processor 51.

In a specific implementation, the memory 52 is used to store data in the present invention and to execute software programs of the present invention. The processor 51 may perform various functions of the air conditioner by running or executing a software program stored in the memory 52 and calling data stored in the memory 52.

The communication interface 53 uses any transceiver-like means for communicating with other devices or communication networks, such as a radio access network (Radio Access Network, RAN), a wireless local area network (Wireless Local Area Networks, WLAN), a terminal, a cloud, etc. The communication interface 53 may include an acquisition unit to implement an acquisition function.

The communication bus 54 may be an industry standard architecture (Industry Standard Architecture, ISA) bus, an external device interconnect (Peripheral Component Interconnect, PCI) bus, or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in fig. 12, but not only one bus or one type of bus.

As an example, in connection with fig. 11, the acquisition unit 101 in the calibration apparatus 10 of the head-mounted device performs the same function as the communication interface 53 in fig. 12, the processing unit 102 of the calibration apparatus 10 of the head-mounted device performs the same function as the processor 51 in fig. 12, and the memory unit 103 of the calibration apparatus 10 of the head-mounted device performs the same function as the memory 52 in fig. 12.

Another embodiment of the present invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a computing device, causes the computing device to implement the method shown in the above-mentioned method embodiment.

In some embodiments, the disclosed methods may be implemented as computer program instructions encoded on a computer-readable storage medium in a machine-readable format or encoded on other non-transitory media or articles of manufacture.

Fig. 13 schematically illustrates a conceptual partial view of a computer program product provided by an embodiment of the invention, the computer program product comprising a computer program for executing a computer process on a computing device.

In one embodiment, a computer program product is provided using signal bearing medium 410. The signal bearing medium 410 may include one or more program instructions that when executed by one or more processors may provide the functionality or portions of the functionality described above with respect to fig. 2. Thus, for example, referring to the embodiment shown in FIG. 2, one or more features of S11-S14 may be carried by one or more instructions associated with signal bearing medium 410. Further, the program instructions in fig. 13 also describe example instructions.

In some examples, signal bearing medium 410 may comprise a computer readable medium 411 such as, but not limited to, a hard disk drive, compact Disk (CD), digital Video Disk (DVD), digital tape, memory, read-only memory (ROM), or random access memory (random access memory, RAM), among others.

In some implementations, the signal bearing medium 410 may include a computer recordable medium 412 such as, but not limited to, memory, read/write (R/W) CD, R/W DVD, and the like.

In some implementations, the signal bearing medium 410 may include a communication medium 413 such as, but not limited to, a digital and/or analog communication medium (e.g., fiber optic cable, waveguide, wired communications link, wireless communications link, etc.).

The signal bearing medium 410 may be conveyed by a communication medium 413 in wireless form (e.g., a wireless communication medium conforming to the IEEE 802.41 standard or other transmission protocol). The one or more program instructions may be, for example, computer-executable instructions or logic-implemented instructions.

In some examples, a calibration apparatus 10, such as the head mounted device described with respect to fig. 11, may be configured to provide various operations, functions, or actions in response to program instructions through one or more of the computer readable medium 411, the computer recordable medium 412, and/or the communication medium 413.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be implemented by different functional modules according to needs, i.e. the internal structure of the apparatus is divided into different functional modules to implement all or part of the functions described above.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical functional division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another apparatus, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and the parts displayed as units may be one physical unit or a plurality of physical units, may be located in one place, or may be distributed in a plurality of different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a readable storage medium. Based on such understanding, the technical solution of the embodiments of the present invention may be essentially or a part contributing to the prior art or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, including several instructions for causing a device (may be a single-chip microcomputer, a chip or the like) or a processor (processor) to perform all or part of the steps of the method described in the embodiments of the present invention. And the aforementioned storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, etc.

The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of calibrating a head-mounted device, comprising:

acquiring a mechanical arm script for fixing the mechanical arm of the head-mounted equipment to be calibrated;

acquiring a calibration image based on the mechanical arm script;

determining the actual distance between each target characteristic point in the calibration image and each target characteristic point except the target characteristic point according to the calibration parameters of the head-mounted equipment to be calibrated and the calibration image;

and determining a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance between each target feature point and each target feature point except the target feature points and the actual distance.

2. The method for calibrating a head-mounted device according to claim 1, wherein the acquiring the robot arm script for fixing the robot arm of the head-mounted device to be calibrated includes:

acquiring equipment information of the to-be-calibrated head-mounted equipment;

and determining a mechanical arm script for fixing the mechanical arm of the head-mounted device to be calibrated according to the device information.

3. The method of calibrating a head-mounted device according to claim 2, wherein the device information comprises a device model;

the determining, according to the device information, a mechanical arm script for fixing the mechanical arm of the to-be-calibrated head-mounted device includes:

inquiring a target script corresponding to the equipment information in a pre-configured corresponding relation according to the equipment model;

and determining the mechanical arm script for fixing the mechanical arm of the to-be-calibrated head-mounted device as the target script according to the target script.

4. The method of calibrating a head-mounted device according to claim 1, wherein the acquiring a calibration image based on the robot arm script comprises:

according to the mechanical arm script, controlling the mechanical arm to move at least one target position;

And controlling the head-mounted device to be calibrated to shoot a calibration plate at each target position, and acquiring a calibration image shot by the head-mounted device to be calibrated.

5. The method for calibrating a headset according to claim 1, wherein determining an actual distance between each target feature point in the calibration image and each target feature point other than the target feature point according to the calibration parameters of the headset to be calibrated and the calibration image comprises:

determining world coordinates corresponding to each target feature point in the calibration image according to the calibration parameters of the head-mounted equipment to be calibrated and the calibration image;

and determining the actual distance between each target feature point in the calibration image and each target feature point except the target feature point according to the world coordinates.

6. The method for calibrating a head-mounted device according to claim 1, wherein the determining the calibration result of the head-mounted device to be calibrated according to the theoretical distance between each target feature point and each target feature point other than the target feature point and the actual distance comprises:

Determining an actual difference value between each target feature point and each target feature point except the target feature point according to the theoretical distance and the actual distance between each target feature point and each target feature point except the target feature point;

and determining a calibration result of the to-be-calibrated head-mounted device according to the actual difference value.

7. The method for calibrating a head-mounted device according to claim 6, wherein the determining the calibration result of the head-mounted device to be calibrated according to the actual difference value comprises:

under the condition that the actual difference value is smaller than or equal to a preset threshold value, determining that the calibration result of the to-be-calibrated head-mounted equipment is normal in precision;

and under the condition that the actual difference value is larger than the preset threshold value, determining that the calibration result of the to-be-calibrated head-mounted equipment is abnormal in precision.

8. The method for calibrating a head-mounted device according to any one of claims 1 to 7, wherein after determining the calibration result of the head-mounted device to be calibrated according to the theoretical distance between each target feature point and each target feature point other than the target feature point and the actual distance, the method further comprises:

Resetting the mechanical arm.

9. A calibration apparatus for a head-mounted device, comprising:

the acquisition unit is used for acquiring a mechanical arm script of the mechanical arm for fixing the head-mounted equipment to be calibrated;

the processing unit is used for acquiring a calibration image based on the mechanical arm script acquired by the acquisition unit;

the processing unit is further configured to determine an actual distance between each target feature point in the calibration image and each target feature point except the target feature point according to the calibration parameters of the headset to be calibrated and the calibration image;

the processing unit is further configured to determine a calibration result of the to-be-calibrated head-mounted device according to the theoretical distance between each target feature point and each target feature point except the target feature point and the actual distance.

10. An electronic device, comprising: a memory and a processor, the memory for storing a computer program; the processor is configured to cause the electronic device to implement the method of calibrating a head-mounted device of any of claims 1-8 when executing a computer program.

11. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a computing device, causes the computing device to implement the method of calibrating a head mounted device according to any of claims 1-8.

12. A computer program product, characterized in that the computer program product, when run on a computer, causes the computer to implement a method of calibrating a head-mounted device according to any of claims 1-8.