CN113329179B - Shooting alignment method, device, equipment and storage medium - Google Patents

Abstract

The application discloses a shooting alignment method, a shooting alignment device, shooting alignment equipment and a storage medium, and belongs to the field of shooting. The shooting alignment method comprises the following steps: acquiring a first picture and a pre-stored reference picture, wherein the first picture is a picture shot by first electronic equipment in a first position controlled by a mechanical arm; acquiring first camera external reference information when a reference picture is shot and acquiring second camera external reference information when the first picture is shot; determining motion control parameters of the mechanical arm according to the first camera external reference information and the second camera external reference information; and controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic equipment moves from the first position to the second position along with the mechanical arm, so that the first electronic equipment shoots a second picture matched with the composition of the reference picture in the second position.

Description

Shooting alignment method, device, equipment and storage medium

technical field

The present application belongs to the field of photographing, and in particular relates to a photographing alignment method, device, equipment and storage medium.

Background technique

When testing the shooting effect of the electronic device, it is often necessary to perform alignment, that is, to move the electronic device to a designated position for shooting.

Usually, the electronic device is fixed at the end of the jig of the mechanical arm, and the electronic device is moved to a designated position by controlling the movement of the mechanical arm to realize automated photo testing. Electronic devices have certain requirements for composition when shooting objects. Therefore, in the process of automated photo testing, a better shooting alignment method is required to ensure that the composition requirements are met for shooting.

However, there is no effective shooting alignment scheme in the case of multiple scenes and multiple cameras.

Contents of the invention

The purpose of the embodiments of the present application is to provide a shooting alignment method, device, device and storage medium, which can solve the problem that there is currently no shooting alignment solution suitable for multiple scenes and multiple cameras.

In the first aspect, the embodiment of the present application provides a shooting alignment method, which is applied to a control device, and the method includes:

Acquiring a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device controlled by the mechanical arm in the first position, and the reference picture is a picture obtained by shooting a target scene;

determining first camera extrinsic information when taking the reference picture, and determining second camera extrinsic information when taking the first picture;

determining motion control parameters of the robotic arm according to the first camera extrinsic information and the second camera extrinsic information;

controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the first electronic device A second picture matching the composition of the reference picture is photographed at the second pose.

In the second aspect, the embodiment of the present application provides a shooting alignment device, which is applied to a control device, and the device includes:

An acquisition module, configured to acquire a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device under the control of the mechanical arm in the first position, and the reference picture is a picture taken of the target scene get the picture;

A first determining module, configured to determine the first camera extrinsic information when taking the reference picture, and determine the second camera extrinsic information when taking the first picture;

A first determining module, configured to determine motion control parameters of the robotic arm according to the first camera extrinsic parameter information and the second camera extrinsic parameter information;

A control module, configured to control the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the The first electronic device photographs a second picture matching the composition of the reference picture in the second pose.

In a third aspect, the embodiment of the present application provides a control device, which includes a processor, a memory, and a program or instruction stored in the memory and operable on the processor, and the program or instruction is controlled by The processor implements the steps of the method described in the first aspect when executed.

In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or an instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented .

In the fifth aspect, the embodiment of the present application provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions, so as to implement the first aspect the method described.

In the embodiment of the present application, in the process of testing the shooting effect of the first electronic device, the first picture taken by the first electronic device controlled by the mechanical arm in the first position and the pre-stored reference picture are acquired, refer to The picture is a picture obtained by shooting the target scene; then, the first camera extrinsic information when taking the reference picture and the second camera extrinsic information when taking the first picture are obtained, and according to the first camera extrinsic information and The extrinsic information of the second camera determines the motion control parameters; then, the mechanical arm is controlled to move according to the motion control parameters. When the mechanical arm completes the movement according to the motion control parameters, the first electronic device moves from the first pose to the second pose with the movement of the robotic arm, and the first electronic device can take pictures in the second pose and the reference The composition of the picture matches the picture. In this way, if it is necessary to test the shooting effect of the electronic device in multiple scenes, the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. In addition, the embodiment of the present application can be applied to the shooting effect test of different cameras, and specifically can be adapted to the test of wide-angle, x1, x2, x5 and other camera lenses. In this way, the requirements of multiple scenes and multiple cameras are met.

Description of drawings

FIG. 1 is a schematic diagram of an embodiment of a shooting alignment system provided by the present application.

FIG. 2 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application.

Fig. 3 is a schematic diagram of an embodiment of an alignment mark provided by the present application.

Fig. 4 is a schematic diagram of an embodiment of a reference picture provided by the present application.

Fig. 5 is a schematic diagram of an embodiment of a checkerboard provided by the present application.

FIG. 6 shows a schematic diagram of the mapping relationship between world coordinates and image coordinates provided by the present application.

FIG. 7 shows a schematic diagram of the relationship among pixel coordinates, internal reference matrix, scale system, external reference matrix and world coordinates provided by the present application.

FIG. 8 shows a schematic diagram of constructing a scale space provided by the present application.

FIG. 9 shows a schematic diagram of feature point positioning provided by the present application.

FIG. 10 shows a schematic diagram of the main direction assignment of feature points provided by the present application.

FIG. 11 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application.

FIG. 12 is a schematic structural diagram of an embodiment of a shooting alignment device provided by the present application.

Fig. 13 is a schematic structural diagram of an embodiment of a control device provided by the present application.

Fig. 14 is a schematic structural diagram of another embodiment of a control device provided by the present application.

Detailed ways

The following will clearly describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, but not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in this application belong to the protection scope of this application.

The terms "first", "second" and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific sequence or sequence. It should be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the application can be practiced in sequences other than those illustrated or described herein, and that references to "first," "second," etc. distinguish Objects are generally of one type, and the number of objects is not limited. For example, there may be one or more first objects. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

Before describing the photographing alignment method provided in this application, the nouns involved in this application will be described first.

World coordinate system (world coordinate system): The coordinate system of the three-dimensional world defined by the user is introduced to describe the position of the target object in the real world.

Pixel coordinate system (pixel coordinate system): Introduced to describe the coordinates of the image point of the object on the digital image (photo) after imaging, it is the coordinate system where the information we actually read from the camera is located. The unit is a (number of pixels).

Image coordinate system (image coordinate system): Introduced to describe the projection and transmission relationship of objects from the camera coordinate system to the image coordinate system during the imaging process, so as to facilitate further obtaining the coordinates in the pixel coordinate system.

Camera coordinate system (camera coordinate system): The coordinate system established on the camera is defined to describe the position of the object from the perspective of the camera, as an intermediate link between the world coordinate system and the image/pixel coordinate system.

The following describes the embodiments of the present application in detail through specific embodiments and application scenarios in conjunction with the accompanying drawings.

FIG. 1 is a schematic diagram of an embodiment of a shooting alignment system provided by the present application. As shown in FIG. 1 , the shooting alignment system includes a control device 01 , a mechanical arm 02 and an electronic device 03 . The control device 01 may be a computer device, such as a mobile phone, a computer, or a console. The electronic device 03 may be any device with a shooting function, such as a mobile phone, a video camera, a camera, and the like.

In order to realize the alignment shooting of the electronic device 03 , the electronic device 03 is installed at the end of the mechanical arm 02 first, and the communication connection between the control device 01 , the mechanical arm 02 and the electronic device 03 is established.

Then, the electronic device 03 takes a first picture, and sends the taken first picture to the control device 01 . Wherein, the electronic device 03 can take the first picture under the control of the control device 01 , or can take the first picture under the control of the staff. After the control device 01 receives the first picture sent by the electronic device 03 , it determines the motion control parameters of the mechanical arm 02 according to the first picture, and controls the mechanical arm 02 to move according to the motion control parameters, so as to achieve alignment shooting of the electronic device 03 .

Wherein, control software can be installed on the control device 01, and the movement of the mechanical arm 02 can be controlled by the software.

Based on the above shooting alignment system, the present application provides a shooting alignment method, which can be applied to the above-mentioned control device 01 . FIG. 2 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application.

As shown in Figure 2, the shooting alignment method includes:

S102. Acquire a first picture and a pre-stored reference picture. The first picture is a picture taken by the first electronic device under the control of a mechanical arm in a first position, and the reference picture is a picture obtained by taking a target scene. For example, the target scene is a scene that the first electronic device needs to shoot when it wants to test the shooting effect of the first electronic device.

Wherein, the first pose is the current pose of the first electronic device, and correspondingly, the first picture is a picture taken by the first electronic device in the current pose. The reference picture may be a picture of an ideal composition of the test scene required by the first electronic device. In order to align the first electronic device and capture the composition effect of the reference picture, S104 is executed.

As an example, S102 may specifically include: when the first electronic device establishes a communication connection with the control device, receiving a first picture sent by the first electronic device; and acquiring a parameter picture stored in a predetermined location.

Shooting alignment methods also include:

S104. Acquire the extrinsic information of the first camera when the reference picture is taken, and acquire the extrinsic information of the second camera when the first picture is taken.

Among them, the first camera extrinsic information represents the transformation relationship from the world coordinates to the first coordinates, and the second camera extrinsic information represents the transformation relation from the world coordinates to the second coordinates. The coordinates in the world coordinate system, the first coordinates are the coordinates of multiple target points on the reference picture, and the second coordinates are the coordinates of multiple target points on the first picture.

As an example, the multiple target points on the reference picture may be corner points on the reference picture, for example, the multiple target points are four corner points on the reference picture. The first coordinates and the second coordinates may be pixel coordinates. Taking the first coordinate as an example, the first coordinate includes the number of pixels from the target point on the reference picture to the coordinate origin on the horizontal axis, and the number of pixels from the target point on the reference picture to the coordinate origin on the vertical axis.

Shooting alignment methods also include:

S106. Determine motion control parameters of the robotic arm according to the first camera extrinsic parameter information and the second camera extrinsic parameter information;

S108, controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the first electronic device takes a picture in the second pose that is consistent with the reference picture The composition matches the second picture.

The shooting alignment in the embodiment of the present application is completed based on the pre-stored reference picture and the first picture actually taken. Objects and scenes in the reference picture can be replaced, and there is no fixed requirement for fixed scenes. Therefore, if it is necessary to test the shooting effect of the first electronic device in multiple scenes (such as close-up shooting and long-distance shooting), the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. It is possible to quickly switch between different scenarios for testing. In addition, the embodiment of the present application can be applied to the shooting effect test of different cameras. For example, when only the monocular camera to be tested is used without external auxiliary shooting and positioning equipment, this application can be adapted to wide-angle, x1, x2, Complicated test scenarios for multiple camera lenses such as x5. In this way, the requirements of multiple scenes and multiple cameras are met. Moreover, the embodiment of the present application is aimed at unfixed objects to be photographed, and is applicable to photographing pose alignment of any object, and the photographed pictures can satisfy the expected composition of the reference picture to a large extent.

In one or more embodiments of the present application, the reference picture includes at least two positioning markers. As an example, the positioning mark may be the positioning mark as shown in FIG. 3 , and the positioning mark is a circular black and white mark.

During the process of taking the reference picture, the positioning mark can be placed within the shooting range of the object to be photographed. In this way, the captured reference picture includes not only the object to be photographed, but also the positioning mark. For example, the reference picture is the picture shown in Figure 4. In Figure 4, two positioning markers are placed on both sides of the disc. mark.

Prior to S104, the shooting alignment method may also include:

Obtain a first distance between the two locators on the reference picture, where the first distance may be the distance between the center points of the two locators;

Determine the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual space distance between the two positioning marks;

According to the preset camera extrinsic parameter calculation algorithm, the world coordinates, the first coordinates, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device are calculated to obtain the first camera extrinsic information. Wherein, the first electronic device may be a camera or a terminal with a camera function (such as a mobile phone or a computer).

As an example, obtaining the first distance between the two locators on the reference picture may specifically include: detecting the two locators on the reference picture; determining the center point of each locator on the reference image through image recognition; calculating the two A first distance between the center points of each positioning mark on the reference image, where the first distance may be a Euclidean distance.

The two locators on the reference picture can be detected by using a target detection model. The target detection model can be a YOLO (You Only Look Once) neural network model. Specifically, the target detection model can be a convolutional neural network model yolov4-Tiny. The target detection model can be deployed on the device side to detect and recognize targets in various complex scenarios.

In addition to obtaining the first distance between the two positioning marks on the reference picture, it is also necessary to obtain the actual spatial distance of the two positioning marks. The actual spatial distance between the two positioning marks can be obtained by manual measurement by the tester, or by sensor measurement.

After the first distance and the actual spatial distance of the two locators are obtained, the world coordinates of the multiple target points in the world coordinate system can be determined according to the first distance and the actual spatial distance of the two locators.

As an example, determining the world coordinates of multiple target points on the reference picture in the world coordinate system according to the first distance and the actual space distance between the two positioning marks may include:

Obtaining first relative position information of multiple target points on the reference picture;

According to the first ratio between the first distance and the actual space distance and the first relative position information, determine the second relative position information of multiple target points in the actual space, wherein the first relative position information and the second relative position information The ratio between is consistent with the first ratio;

The world coordinates of the multiple target points on the reference picture in the world coordinate system are determined according to the second relative position information.

The following uses multiple target points as the four corner points on the reference image as an example to illustrate how to determine the world coordinates.

Continue to refer to the reference picture shown in FIG. 4 to obtain the first relative position information between the four corner points (ie corner point A, corner point B, corner point C and corner point D) on the reference picture, the first relative position The information is the relative position information of each corner point relative to other corner points. For example, the first relative position information includes the position of corner B at the 100th pixel in the right direction of corner A, the position of corner C at the 70th pixel in the upward direction of corner B, and the position of corner D It is located at the 100th pixel in the leftward direction of corner point C, and the corner point A is located at the 70th pixel in the downward direction of corner point D.

Assume that the first distance between the two positioning marks on the reference picture is 50 pixels, the actual space distance between the two positioning marks is 50 cm, and corner point B is located at the 100th point to the right of corner point A on the reference picture The position of the pixel, therefore, in real space, corner point B is located 100 centimeters to the right of corner point A. Then, take the center point of the reference picture as the coordinate origin to establish a world coordinate system, and according to the relative positions of corner point B and corner point A in the actual space, the world coordinates of corner point B and corner point A in the world coordinate system can be determined . Similarly, the relative positions of the corner point C and the corner point D in the reference picture in the real space, and the world coordinates of the corner point C and the corner point D in the world coordinate system can be determined.

The above is an example of determining the world coordinates by taking multiple target points as the four corner points on the reference picture as an example. Wherein, when the reference picture is taken, the positioning mark can be placed at any position of the scene desired to be taken, and the world coordinates of the target point on the taken reference picture can be accurately calculated.

The world coordinates of multiple target points are used to determine the first camera extrinsic information, and the determination of the first camera extrinsic information not only needs to use the world coordinates of multiple target points, but also requires the use of a second electronic device that takes reference pictures internal reference information. The internal reference information can be obtained by calibrating the camera that takes the reference picture.

As an example, Zhang Zhengyou's camera calibration method can be used to calibrate the camera that takes the reference picture. Among them, the opencv camera calibration interface can be called to complete the calibration of the camera internal parameters.

Specifically, the camera is first used to take pictures of the checkerboard to obtain a photo including the checkerboard. The number of photos depends on whether the internal reference information converges within the specified error range. Generally, at least four photos are taken, one is taken straight and three are tilted. The pattern of the checkerboard is shown in Figure 5. Moreover, it is not allowed to take pictures of the entire checkerboard, and sub-pixel corner detection can be performed on the pictures taken, which increases the robustness.

After the photo including the checkerboard is taken, the QR code in the marked white grid in the photo is recognized, and the position information of the corresponding white grid on the chessboard is stored in the QR code in the white grid. Specifically, a serial number is stored in the two-dimensional code in the white grid, and the serial number corresponds to the position information of the rows and columns of the white grid on the chessboard.

After determining the position information of the white grid on the chessboard, based on the position information of the white grid on the chessboard, the position of the chessboard can be determined, and then the black and white corner points on the chessboard are detected, and according to the detection results, it is determined that each corner point is in the photo The pixel coordinates on and the world coordinates of each corner point.

Based on the pixel coordinates of each corner point on the photo and the world coordinates of each corner point, internal reference information of the second electronic device can be determined.

How to determine the internal reference information of the second electronic device will be described below with reference to FIG. 6 .

在2D空间中，点在像素坐标系中的坐标m＝[u，v]^T，其齐次坐标

FIG. 6 shows a schematic diagram of a mapping relationship between world coordinates and image coordinates provided by the present application. As shown in Figure 6, in 3D space, the world coordinate M=[X,Y,Z] ^T of a point in the world coordinate system, its homogeneous coordinate

In 2D space, the coordinate m=[u, v] ^T of a point in the pixel coordinate system, its homogeneous coordinate

Assuming that the checkerboard is located at Z=0, and the i-th column of the rotation matrix R is defined as r _i , then:

Therefore, the mapping from the world coordinate system to the image coordinate system is:

Based on this, the relationship among pixel coordinates, internal parameter matrix, scale system s, external parameter matrix and world coordinates is shown in Figure 7.

In the process of determining the internal reference information of the second electronic device, it is necessary to map the points in one picture to the corresponding points in another picture. The homography transformation is the current transformation of the points under the aligned sub-coordinates. The homography matrix H is highly restrictive. , is a point-to-point one-to-one correspondence. H is a 3*3 matrix with one element as a homogeneous coordinate, therefore, H has 8 degrees of freedom. Now there are 8 degrees of freedom to be solved, so four corresponding points are needed, that is to say, the homography matrix H from the image coordinate system to the world coordinate system can be obtained through four points, and the homography matrix H includes the internal reference matrix and extrinsic matrix. The internal reference matrix included in the homography matrix H is the internal reference information of the second electronic device.

Among them, the internal reference matrix specifically includes the physical size and focal length of a pixel, the distortion factor of the physical coordinates of the image, and the vertical and horizontal offsets u and v of the image origin relative to the optical center imaging point. The external parameter matrix includes the rotation R and translation T matrices from the world coordinate system to the camera coordinate system.

In addition, because the real lens is not an ideal perspective imaging, but has different degrees of distortion. Theoretically, lens distortion includes radial distortion and tangential distortion. Therefore, in the process of calibrating the camera, the camera distortion coefficient of the second electronic device can also be obtained, and the camera distortion coefficient includes radial camera distortion coefficients k1, k2, k3, ~, and tangential camera distortion coefficient p1 of the camera , p2, ~. Since the tangential distortion of the camera has less influence, the radial distortion is usually considered, and in the process of solving the radial distortion, the first two coefficients k1 and k2 of the dominant binary Taylor series expansion are mainly considered.

The above has explained how to determine the world coordinates of multiple target points, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device, and the first coordinates of multiple target points on the reference picture can be based on the reference picture to get. Then, the world coordinates, the first coordinates, and the internal parameter information of the second electronic device that took the reference picture can be calculated according to the preset camera external parameter calculation algorithm to obtain the first camera external parameter information. Among them, the camera extrinsic parameter calculation algorithm can solve for N-point perspective pose (solve Perspective-N-Point, solvePNP).

When using solvePNP to solve the first camera extrinsic information, the camera model based on solvePNP solution can include the relationship shown in Figure 7, based on the relationship shown in Figure 7, the pixel coordinates of multiple target points on the reference picture (i.e. The first coordinate), the internal parameter matrix of the camera that took the reference picture, the scale coefficient, and the world coordinates of multiple target points are calculated, and the obtained external parameter matrix is the external parameter information of the first camera.

In one or more embodiments of the present application, before S104, the shooting alignment method may further include:

performing feature detection processing on the reference picture and the first picture to obtain a conversion matrix from the reference picture to the first picture;

Perform perspective transformation according to the transformation matrix and the first coordinate to obtain the second coordinate;

The world coordinates, the second coordinates, the internal parameter information of the first electronic device, and the camera distortion coefficient of the first electronic device are calculated according to the preset camera external parameter calculation algorithm to obtain the second camera external parameter information.

As an example, the transformation matrix from the reference picture to the first picture may be obtained by using a Speeded Up Robust Features (SURF) algorithm to perform feature matching on the reference picture and the first picture.

How to obtain the conversion matrix H is exemplified below.

1. Construct a Hessian matrix (Hessian) for each picture in the reference picture and the first picture, and generate key points on the picture for feature extraction.

The Hessian matrix of the picture f(x, y) is as follows:

Before constructing the Hessian matrix, it is necessary to perform Gaussian filtering on the picture, and the filtered Hessian matrix is expressed as:

Among them, when the discriminant of the Hessian matrix takes a local maximum value, it is judged that the current point is brighter or darker than other points in the surrounding area, thereby determining the position of the key point.

2. Construct scale space

In SURF, the size of the image between different groups is the same, the difference is that the template size of the box filter used between different groups gradually increases, and the same size filter is used between different layers in the same group, just filtering The blur coefficient of the filter increases gradually, as shown in Figure 8.

3. Feature point positioning

Referring to Figure 9, each pixel processed by the Hessian matrix is compared with 26 points in the two-dimensional image space and scale space neighborhood, and the key points are initially located, and then the key points with weaker energy and errors are filtered out. Locate the key points and filter out the final stable feature points.

4. Main direction assignment of feature points

In SURF, the Haar-like features (Haar-like features, referred to as harr) wavelet features in the circular neighborhood of statistical feature points are used. That is, in the circular neighborhood of feature points, the sum of the horizontal and vertical harr wavelet features of all points in the 60-degree fan is counted, and then the fan is rotated at an interval of 0.2 radians and the harr wavelet eigenvalues in the area are counted again, and finally the The direction of the sector with the largest value is taken as the main direction of the feature point. The schematic diagram of the process is shown in Figure 10.

5. Generate feature point descriptors

Take a 4*4 rectangular area block around the feature point, but the direction of the obtained rectangular area is along the main direction of the feature point. Each sub-region counts the haar wavelet features of the horizontal and vertical directions of 25 pixels.

6. Feature matching

The matching degree is determined by calculating the Euclidean distance between two feature points. The shorter the Euclidean distance, the better the matching degree of the two feature points. In SURF, the judgment of the Hessian matrix trace is also added. If the matrix traces of two feature points have the same sign, it means that the two features have contrast changes in the same direction. If they are different, it means that the two feature points have the same contrast. The direction of contrast change is opposite, even if the Euclidean distance is 0, the page is directly excluded. Therefore, according to the Euclidean distance between the feature points in the reference picture and the feature points in the first picture, a conversion matrix from the reference picture to the first picture is generated.

After the transformation matrix from the reference picture to the first picture is obtained, perspective transformation is performed according to the transformation matrix and the first coordinates to obtain the second coordinates of the multiple target points on the first picture. In this way, through perspective transformation, the position of the target point on the reference picture on the first picture is determined.

After obtaining the world coordinates of multiple target points, the second coordinates of multiple target points on the first picture, the internal parameter information of the first electronic device, and the camera distortion coefficient, these data can be calculated to obtain the external parameter information of the second camera . Among them, the way of calculating the extrinsic information of the second camera is similar to the way of calculating the extrinsic information of the first camera, but how to calculate the extrinsic information of the first camera has been explained, and how to calculate the extrinsic information of the second camera will not be discussed here Do similar repetitions.

Wherein, the internal reference information and the camera distortion coefficient of the first electronic device may be obtained by calibrating the first electronic device. The specific calibration method is similar to the above-mentioned calibration method for the camera that captures the reference picture, and will not be repeated here.

In one or more embodiments of the present application, both the first camera extrinsic parameter information and the second camera extrinsic parameter information are expressed in the form of rotation vectors. In order to obtain the motion control parameters of the manipulator, it is necessary to perform a rotation matrix transformation on the first camera extrinsic parameter information and the second camera extrinsic parameter information.

Based on this, S106 may specifically include:

converting the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain a first rotation matrix corresponding to the first camera extrinsic information and a second rotation matrix corresponding to the second camera extrinsic information;

According to the first rotation matrix and the second rotation matrix, the motion control parameters of the mechanical arm are determined.

Among them, the rotation vector can be converted into a rotation matrix according to the Rodriguez formula, and the Rodriguez formula is as follows:

R＝cosθI+(1-cosθ)nn ^T +sinθn^ (5)

Among them, I is the identity matrix, n is the unit vector of the rotation vector, and θ is the modulus length of the rotation vector.

As an example, the motion control parameters include the offset information of the robotic arm and the attitude adjustment angle; according to the first rotation matrix and the second rotation matrix, determining the motion control parameters of the robotic arm may specifically include:

The first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and the first rotation matrix is subtracted from the second rotation matrix to obtain offset information.

Wherein, the first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the camera rotation matrix from the first picture to the reference picture, and the camera rotation matrix is determined as the attitude adjustment Euler angle (ie attitude adjustment angle) . The first rotation matrix is subtracted from the second rotation matrix to obtain the position offset value of the mechanical arm.

After obtaining the above-mentioned motion control parameters of the mechanical arm, if the end of the clamp of the mechanical arm clamps the first electronic device, then when the clamp of the mechanical arm is controlled to move according to the motion control parameters, the electronic device and the first electronic device also Moves with the movement of the gripper. When the gripper of the mechanical arm completes the motion, the first electronic device moves to the second posture. In the second pose, the first electronic device may take a second picture that matches the composition of the reference picture.

It should be noted that the fixture calibration is in the control part of the robotic arm. In the embodiment of this application, the end of the fixture defaults to the calibrated user coordinate system, which can be directly controlled by giving the robotic arm a user coordinate system offset value.

In the embodiment of the present application, by converting the first camera extrinsic information and the second camera extrinsic information in the form of rotation vectors into corresponding rotation matrices, it is easy to obtain the motion control parameters of the robotic arm and realize precise control of shooting alignment .

The embodiment of the present application will be further described below with reference to FIG. 11 .

FIG. 11 is a schematic flow chart of an embodiment of a shooting alignment method provided by the present application. As shown in Figure 11, the shooting alignment method includes:

S202, use the checkerboard to calibrate the internal reference information of the second electronic device that captures the reference picture and the camera distortion coefficient of the second electronic device;

S204. Obtain a first distance between two locators on the reference picture on the reference picture;

S206. Determine the world coordinates of the four corner points on the reference picture according to the first distance and the actual space distance between the two locators;

S208, according to the internal reference information of the second electronic device that took the reference picture, the camera distortion coefficient of the second electronic device, the world coordinates of the four corner points, and the first coordinates of the four corner points on the reference picture, determine the location of the reference picture The first camera extrinsic information of the camera;

S210. When the first electronic device on the mechanical arm captures the first picture, perform feature detection processing on the reference picture and the first picture to obtain a conversion matrix from the reference picture to the first picture;

S212. Perform perspective transformation according to the transformation matrix and the first coordinates of the four corner points on the reference picture to obtain second coordinates of the four corner points on the first picture;

S214. Determine the second position of the first electronic device according to the internal reference information of the first electronic device, the camera distortion coefficient of the first electronic device, the world coordinates of the four corner points, and the second coordinates of the four corner points on the first picture. Camera extrinsic information;

S216, converting the first camera extrinsic information represented by the rotation vector into a first rotation matrix based on the world coordinate system, and converting the second camera extrinsic information represented by the rotation vector into a second rotation matrix based on the world coordinate system;

S218. Multiply the first rotation matrix by the transpose matrix of the second rotation matrix to obtain a target rotation matrix from the first picture to the reference picture;

S220. Determine the target rotation matrix as the position offset value of the clamp end of the mechanical arm, and subtract the first rotation matrix from the second rotation matrix to obtain the attitude adjustment Euler angles of the mechanical arm end.

Corresponding to the photographing alignment method provided in the present application, the present application also provides a photographing alignment device. FIG. 12 is a schematic structural diagram of an embodiment of the photographing alignment device provided in the present application. As shown in Figure 12, the photographing alignment device 300 includes:

The first obtaining module 302 is configured to obtain a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device controlled by the mechanical arm in the first position, and the reference picture is obtained by shooting the target scene picture of;

The second acquiring module 304 is configured to acquire the first camera extrinsic information when taking the reference picture, and acquire the second camera extrinsic information when taking the first picture;

The first determining module 306 is configured to determine motion control parameters of the robotic arm according to the first camera extrinsic parameter information and the second camera extrinsic parameter information;

The control module 308 is configured to control the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the first electronic device takes pictures in the second pose A second picture matching the composition of the reference picture is generated.

The shooting alignment in the embodiment of the present application is completed based on the pre-stored reference picture and the first picture actually taken. Objects and scenes in the reference picture can be replaced, and there is no fixed requirement for fixed scenes. Therefore, if it is necessary to test the shooting effect of the first electronic device in multiple scenes (such as close-up shooting and long-distance shooting), the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. It is possible to quickly switch between different scenarios for testing. In addition, the embodiment of the present application may be applicable to shooting effect tests of different cameras. In this way, the requirements of multiple scenes and multiple cameras are met. Moreover, the embodiment of the present application is aimed at unfixed objects to be photographed, and is applicable to photographing pose alignment of any object, and the photographed pictures can satisfy the expected composition of the reference picture to a large extent.

In one or more embodiments of the present application, the first camera extrinsic information represents the transformation relationship from the world coordinate to the first coordinate, and the second camera extrinsic information represents the transformation relation from the world coordinate to the second coordinate, wherein the world coordinate are the coordinates of multiple target points on the reference picture in the world coordinate system, the first coordinates are the coordinates of the multiple target points on the reference picture, and the second coordinates are the coordinates of the multiple target points on the first picture.

In one or more embodiments of the present application, at least two positioning marks are included in the reference picture; the shooting alignment device 300 may also include:

The third obtaining module is used to obtain the first distance between the two positioning markers on the reference picture;

The second determination module is used to determine the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual spatial distance of the two positioning marks;

The first calculation module is used to calculate the world coordinates, the first coordinates, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device according to the preset camera external parameter calculation algorithm, to obtain the second electronic device. A camera extrinsic information.

In one or more embodiments of the present application, the second determination module may include:

A first acquiring unit, configured to acquire first relative position information of a plurality of target points on a reference picture;

The first determining unit is configured to determine second relative position information of multiple target points in real space according to the first ratio between the first distance and the actual space distance and the first relative position information, wherein the first relative position The ratio between the information and the second relative position information is consistent with the first ratio;

The second determining unit is configured to determine the world coordinates according to the second relative position information.

In one or more embodiments of the present application, the shooting alignment device 300 may also include:

A processing module, configured to perform feature detection processing on the reference picture and the first picture, to obtain a conversion matrix from the reference picture to the first picture;

The perspective transformation module is used to perform perspective transformation according to the conversion matrix and the first coordinates to obtain the second coordinates;

The second calculation module is configured to calculate the world coordinates, the second coordinates, the internal parameter information of the first electronic device, and the camera distortion coefficient of the first electronic device according to a preset camera external parameter calculation algorithm to obtain the second camera external parameters information.

In one or more embodiments of the present application, both the first camera extrinsic information and the second camera extrinsic information are expressed in the form of rotation vectors; the first determining module 306 includes:

The conversion unit is configured to convert the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain the first rotation matrix corresponding to the first camera extrinsic information and the second rotation matrix corresponding to the second camera extrinsic information. rotation matrix;

The third determining unit is configured to determine motion control parameters of the mechanical arm according to the first rotation matrix and the second rotation matrix.

In one or more embodiments of the present application, the motion control parameters include the offset information of the robotic arm and the attitude adjustment angle; the third determining unit is used for:

The first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and the first rotation matrix is subtracted from the second rotation matrix to obtain offset information.

It should be noted that, the photographing alignment method provided in the embodiment of the present application may be executed by a photographing alignment device, or a control module in the photographing alignment device for executing the photographing alignment method. In the embodiment of the present application, the photographing alignment device provided by the embodiment of the present application is described by taking the photographing alignment device executing the photographing alignment method as an example.

The photographing alignment device in the embodiment of the present application may be a device, or may be a component, an integrated circuit, or a chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a vehicle electronic device, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant). , PDA), etc., non-mobile electronic equipment can be server, network attached storage (NetworkAttached Storage, NAS), personal computer (personal computer, PC), television (television, TV), teller machine or self-service machine, etc., the embodiment of the present application Not specifically limited.

The shooting alignment device in the embodiment of the present application may be a device with an operating system. The operating system may be an Android (Android) operating system, an ios operating system, or other possible operating systems, which are not specifically limited in this embodiment of the present application.

The photographing alignment device provided in the embodiment of the present application can realize various processes realized by the method embodiment in FIG. 2 or FIG. 11 , and details are not repeated here to avoid repetition.

The present application also provides an electronic device, including a processor, a memory, and a program or instruction stored on the memory and operable on the processor. When the program or instruction is executed by the processor, the shooting alignment of any of the above-mentioned embodiments can be realized. method steps.

As shown in FIG. 13, the embodiment of the present application also provides a control device 400, including a processor 401, a memory 402, and a program or instruction stored in the memory 402 and operable on the processor 401. The program or instruction is processed When executed by the device 401, each process of the above embodiment of the photographing alignment method can be realized, and the same technical effect can be achieved. To avoid repetition, details are not repeated here.

Fig. 13 is a schematic structural diagram of another embodiment of a control device provided by the present application.

The control device 500 includes but is not limited to: a radio frequency unit 501, a network module 502, an audio output unit 503, an input unit 504, a sensor 505, a display unit 506, a user input unit 507, an interface unit 508, a memory 509, and a processor 510, etc. .

Those skilled in the art can understand that the control device 500 can also include a power supply (such as a battery) for supplying power to various components, and the power supply can be logically connected to the processor 510 through the power management system, so that the management of charging, discharging, and function can be realized through the power management system. Consumption management and other functions. The device structure shown in FIG. 14 does not constitute a limitation on the control device. The control device may include more or less components than shown in the figure, or combine certain components, or arrange different components, which will not be repeated here.

Wherein, the processor 510 is used for:

Acquiring a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device controlled by the mechanical arm in the first position;

Obtaining the extrinsic information of the first camera when taking the reference picture, and acquiring the extrinsic information of the second camera when taking the first picture;

Determining motion control parameters of the robotic arm according to the external parameter information of the first camera and the external parameter information of the second camera;

Controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the robotic arm, so that the first electronic device takes a composition with the reference picture in the second pose Match the second picture.

The shooting alignment in the embodiment of the present application is completed based on the pre-stored reference picture and the first picture actually taken. Objects and scenes in the reference picture can be replaced, and there is no fixed requirement for fixed scenes. Therefore, if it is necessary to test the shooting effect of the first electronic device in multiple scenes (such as close-up shooting and long-distance shooting), the reference pictures taken for each scene can be provided to complete the shooting effect test of the corresponding scene. It is possible to quickly switch between different scenarios for testing. In addition, the embodiment of the present application may be applicable to shooting effect tests of different cameras. In this way, the requirements of multiple scenes and multiple cameras are met. Moreover, the embodiment of the present application is aimed at unfixed objects to be photographed, and is applicable to photographing pose alignment of any object, and the photographed pictures can satisfy the expected composition of the reference picture to a large extent.

In one or more embodiments of the present application, the first camera extrinsic information represents the transformation relationship from the world coordinate to the first coordinate, and the second camera extrinsic information represents the transformation relation from the world coordinate to the second coordinate, wherein the world coordinate are the coordinates of multiple target points on the reference picture in the world coordinate system, the first coordinates are the coordinates of the multiple target points on the reference picture, and the second coordinates are the coordinates of the multiple target points on the first picture.

In one or more embodiments of the present application, the processor 510 is specifically used to:

Obtain the first distance between the two positioning markers on the reference picture;

Determine the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual space distance between the two positioning marks;

According to the preset camera extrinsic parameter calculation algorithm, the world coordinates, the first coordinates, the internal reference information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device are calculated to obtain the first camera extrinsic information.

In one or more embodiments of the present application, the processor 510 is specifically used to:

Obtaining first relative position information of multiple target points on the reference picture;

According to the first ratio between the first distance and the actual space distance and the first relative position information, determine the second relative position information of multiple target points in the actual space, wherein the first relative position information and the second relative position information The ratio between is consistent with the first ratio;

The world coordinates are determined according to the second relative position information.

In one or more embodiments of the present application, the processor 510 is specifically used to:

performing feature detection processing on the reference picture and the first picture to obtain a transformation matrix from the reference picture to the first picture;

Perform perspective transformation according to the transformation matrix and the first coordinate to obtain the second coordinate;

The world coordinates, the second coordinates, the internal parameter information of the first electronic device, and the camera distortion coefficient of the first electronic device are calculated according to the preset camera external parameter calculation algorithm to obtain the second camera external parameter information.

In one or more embodiments of the present application, both the first camera extrinsic parameter information and the second camera extrinsic parameter information are represented in the form of rotation vectors; the processor 510 is specifically configured to:

converting the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain a first rotation matrix corresponding to the first camera extrinsic information and a second rotation matrix corresponding to the second camera extrinsic information;

According to the first rotation matrix and the second rotation matrix, the motion control parameters of the mechanical arm are determined.

In one or more embodiments of the present application, the processor 510 is specifically used to:

The first rotation matrix is multiplied by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and the first rotation matrix is subtracted from the second rotation matrix to obtain offset information.

It should be understood that, in the embodiment of the present application, the input unit 504 may include a graphics processor (Graphics Processing Unit, GPU) 5041 and a microphone 5042, and the graphics processor 5041 is compatible with the image capture device (such as Camera) to process the image data of still pictures or videos. The display unit 506 may include a display panel 5061, and the display panel 5061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 507 includes a touch panel 5071 and other input devices 5072 . The touch panel 5071 is also called a touch screen. The touch panel 5071 may include two parts, a touch detection device and a touch controller. Other input devices 5072 may include, but are not limited to, physical keyboards, function keys (such as volume control keys, switch keys, etc.), trackballs, mice, and joysticks, which will not be repeated here. Memory 509 may be used to store software programs as well as various data, including but not limited to application programs and operating systems. The processor 510 may integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces, and application programs, and the modem processor mainly processes wireless communications. It can be understood that the foregoing modem processor may not be integrated into the processor 510 .

The embodiment of the present application also provides a readable storage medium, the readable storage medium stores a program or an instruction, and when the program or instruction is executed by the processor, the various processes of the above-mentioned embodiment of the shooting alignment method are realized, and can achieve The same technical effects are not repeated here to avoid repetition.

Wherein, the processor is the processor in the control device described in the above embodiments. The readable storage medium includes computer readable storage medium, such as computer read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, etc.

The embodiment of the present application further provides a chip, the chip includes a processor and a communication interface, the communication interface is coupled to the processor, and the processor is used to run programs or instructions to realize the above-mentioned embodiment of the shooting alignment method Each process, and can achieve the same technical effect, in order to avoid repetition, will not repeat them here.

It should be understood that the chips mentioned in the embodiments of the present application may also be called system-on-chip, system-on-chip, system-on-a-chip, or system-on-a-chip.

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

The embodiments of the present application have been described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementations. The above-mentioned specific implementations are only illustrative and not restrictive. Those of ordinary skill in the art will Under the inspiration of this application, without departing from the purpose of this application and the scope of protection of the claims, many forms can also be made, all of which belong to the protection of this application.

Claims (11)

1. A shooting alignment method applied to control equipment, characterized in that the method comprises: Acquiring a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device controlled by the mechanical arm in the first position, and the reference picture is a picture obtained by shooting a target scene; acquiring first camera extrinsic information when taking the reference picture, and acquiring second camera extrinsic information when taking the first picture; determining motion control parameters of the robotic arm according to the first camera extrinsic information and the second camera extrinsic information; controlling the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the first electronic device taking a second picture matching the composition of the reference picture at the second pose; The first camera extrinsic parameter information represents a transformation relationship from world coordinates to first coordinates, and the second camera extrinsic information represents a transformation relation from world coordinates to second coordinates, wherein the world coordinates are the reference The coordinates of multiple target points on the picture in the world coordinate system, the first coordinates are the coordinates of multiple target points on the reference picture, and the second coordinates are the coordinates of multiple target points on the reference picture. The coordinates on the first picture; The reference picture includes at least two positioning marks; before acquiring the extrinsic information of the first camera when taking the reference picture, the method further includes: Acquiring a first distance between the two locators on the reference picture; determining the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual spatial distance between the two positioning marks; According to a preset camera external parameter calculation algorithm, the world coordinates, the first coordinates, the internal parameter information of the second electronic device that took the reference picture, and the camera distortion coefficient of the second electronic device are calculated to obtain The extrinsic information of the first camera. 2. The method according to claim 1, wherein, according to the first distance and the actual space distance between the two positioning marks, it is determined that a plurality of target points on the reference picture are in the world coordinate system The world coordinates of , including: Acquiring first relative position information of multiple target points on the reference picture; According to the first ratio between the first distance and the actual space distance and the first relative position information, determine the second relative position information of a plurality of the target points in the actual space, wherein the first a ratio between the relative position information and the second relative position information is consistent with the first ratio; The world coordinates are determined according to the second relative position information. 3. The method according to claim 1, wherein, before the acquisition of the second camera extrinsic information when taking the first picture, the method further comprises: performing feature detection processing on the reference picture and the first picture to obtain a transformation matrix from the reference picture to the first picture; performing perspective transformation according to the transformation matrix and the first coordinates to obtain the second coordinates; Calculate the world coordinates, the second coordinates, the internal parameter information of the first electronic device, and the camera distortion coefficient of the first electronic device according to a preset camera external parameter calculation algorithm to obtain the second Camera extrinsic information. 4. The method according to claim 1, characterized in that, both the first camera extrinsic information and the second camera extrinsic information are expressed in the form of rotation vectors; The determining the motion control parameters of the robotic arm according to the first camera extrinsic information and the second camera extrinsic information includes: Converting the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain a first rotation matrix corresponding to the first camera extrinsic information, and a first rotation matrix corresponding to the second camera extrinsic information The second rotation matrix of ; According to the first rotation matrix and the second rotation matrix, a motion control parameter of the mechanical arm is determined. 5. The method according to claim 4, wherein the motion control parameters include offset information and attitude adjustment angles of the mechanical arm; the first rotation matrix and the second rotation matrix , to determine the motion control parameters of the mechanical arm, including: multiplying the first rotation matrix by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and subtracting the first rotation matrix from the second rotation matrix to obtain the offset information. 6. A shooting alignment device, applied to control equipment, characterized in that the device includes: The first acquisition module is used to acquire a first picture and a pre-stored reference picture, the first picture is a picture taken by the first electronic device under the control of the mechanical arm in the first position, and the reference picture is a picture of the target scene The pictures obtained by taking pictures; A second acquiring module, configured to acquire the first camera extrinsic information when taking the reference picture, and acquire the second camera extrinsic information when taking the first picture; A first determining module, configured to determine motion control parameters of the robotic arm according to the first camera extrinsic parameter information and the second camera extrinsic parameter information; A control module, configured to control the mechanical arm to move according to the motion control parameters, wherein the first electronic device moves from the first pose to the second pose along with the mechanical arm, so that the The first electronic device captures a second picture matching the composition of the reference picture in the second pose; The first camera extrinsic parameter information represents a transformation relationship from world coordinates to first coordinates, and the second camera extrinsic information represents a transformation relation from world coordinates to second coordinates, wherein the world coordinates are the reference The coordinates of multiple target points on the picture in the world coordinate system, the first coordinates are the coordinates of multiple target points on the reference picture, and the second coordinates are the coordinates of multiple target points on the reference picture. The coordinates on the first picture; The reference picture includes at least two positioning marks; the device also includes: A third acquiring module, configured to acquire a first distance between two locators on the reference picture; The second determination module is configured to determine the world coordinates of multiple target points in the world coordinate system according to the first distance and the actual space distance between the two positioning marks; The first calculation module is configured to calculate the world coordinates, the first coordinates, the internal parameter information of the second electronic device that took the reference picture, and the internal parameter information of the second electronic device according to a preset camera external parameter calculation algorithm. The camera distortion coefficient is calculated to obtain the extrinsic information of the first camera. 7. The device according to claim 6, wherein the second determining module comprises: A first acquiring unit, configured to acquire first relative position information of a plurality of target points on the reference picture; A first determining unit, configured to determine a second relative position of a plurality of target points in real space according to a first ratio between the first distance and the real space distance and the first relative position information information, wherein the ratio between the first relative position information and the second relative position information is consistent with the first ratio; The second determining unit is configured to determine the world coordinates according to the second relative position information. 8. The device according to claim 6, further comprising: A processing module, configured to perform feature detection processing on the reference picture and the first picture to obtain a transformation matrix from the reference picture to the first picture; A perspective transformation module, configured to perform perspective transformation according to the transformation matrix and the first coordinates to obtain the second coordinates; The second calculation module is configured to perform calculation on the world coordinates, the second coordinates, the internal reference information of the first electronic device, and the camera distortion coefficient of the first electronic device according to a preset camera external parameter calculation algorithm. Calculate to obtain the extrinsic information of the second camera. 9. The device according to claim 6, wherein the first camera extrinsic information and the second camera extrinsic information are both expressed in the form of rotation vectors; The first determination module includes: a conversion unit, configured to convert the first camera extrinsic information and the second camera extrinsic information into rotation matrices respectively, to obtain a first rotation matrix corresponding to the first camera extrinsic information, and the second The second rotation matrix corresponding to the camera extrinsic information; A third determining unit, configured to determine motion control parameters of the mechanical arm according to the first rotation matrix and the second rotation matrix. 10. The device according to claim 9, wherein the motion control parameters include offset information and attitude adjustment angles of the robotic arm; the third determining unit is used for: multiplying the first rotation matrix by the transpose matrix of the second rotation matrix to obtain the attitude adjustment angle, and subtracting the first rotation matrix from the second rotation matrix to obtain the offset information. 11. A control device, characterized in that it includes a processor, a memory, and a program or instruction stored on the memory and operable on the processor, and the program or instruction is implemented when executed by the processor. The step of the method for photographing alignment as described in any one of claims 1-5.

Images