KR20220094250A - System and method for estimating the position of object in image - Google Patents

Abstract

The present invention relates to a system and method for estimating a location of an object in an image. The system for estimating a location of an object in an image according to the present invention comprises: three or more reference location units placed in different specific absolute coordinates; an image acquisition unit acquiring a two-dimensional image including all the reference location units and an object to be estimated; a relationship calculation unit calculating a relationship between the location of the reference location unit on the two-dimensional image and an actual location of the reference location unit; and a location estimation unit calculating the absolute coordinates of the object to be estimated through the relationship. Accordingly, it is possible to more easily estimate the location of the object in the image.

Description

System and method for estimating the position of object in image

The present invention relates to a system and method for estimating the position of an object in an image, and more particularly, to a system and method for estimating the position of an object in an image capable of more easily estimating the position of the object in the image.

The robot uses a manipulator to perform work, and the movement characteristics of the manipulator are determined according to the geometrical relationship between links and joints. Kinematics is a mathematical expression of this geometric relationship, and most manipulators can move the robot hand in the direction to perform a task by using the kinematic characteristics.

Since the robot is usually equipped with a camera and can determine the operation of the manipulator by checking the position of the work object, in order to operate the manipulator accurately, the position of the work object must be accurately grasped through the image captured by the camera.

In addition, the robot's coordinate system and the camera's coordinate system must be calibrated so that they correspond to each other so that the manipulator can perform an accurate operation on the work object whose position has been identified through the camera.

As one of the prior art related to a method of performing calibration between a robot and a camera, there is a 'robot hand and a method and apparatus for calibration of a camera attached to the robot hand' of registration number 10-1465652.

In the prior art, the transformation relation between the robot hand coordinate system and the camera coordinate system is calculated using the rotation transformation relation between the robot hand coordinate system and the camera coordinate system and the translation transformation relation between the robot hand coordinate system and the camera coordinate system.

This calibration method between the robot and the camera has a problem in that it is not easy to perform the calibration work because the movement of the robot and the transformation relation of the camera coordinate system must be calculated accordingly, which may be a hindrance factor in facilitating the operation of the robot.

KR 10-1465652 B1

Accordingly, an object of the present invention is to solve the problems of the related art, and to provide a system and method for estimating the position of an object in an image, which facilitates the operation of a robot by estimating the position of the object in the image.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

The above object is, according to the present invention, three or more reference position parts placed on different specific absolute coordinates; an image acquisition unit configured to acquire a two-dimensional image including both the reference position unit and the estimated target object; a relationship calculator configured to calculate a relationship between the position of the reference position on the two-dimensional image and the actual position of the reference position; and a position estimator for calculating the absolute coordinates of the estimation target object through the relationship;

All of the reference position units may be located on the same plane.

The reference position portion may be located on different corners of a rectangle.

The system for estimating the position of an object in an image according to the present invention may further include an image conversion unit that converts the two-dimensional image so that the reference position part is located on different corners of one rectangle on the two-dimensional image.

The reference position unit may include a QR code image including information about the absolute coordinates of the reference position unit.

The system for estimating the position of an object in an image according to the present invention may further include an image acquisition unit position calculator configured to calculate a position on absolute coordinates of the image acquisition unit through the relationship.

According to another embodiment of the present invention, a reference position unit arrangement step of arranging three or more reference position units on different specific absolute coordinates; an image acquisition step of acquiring a two-dimensional image including both the reference position part and the estimated target object; a relationship calculation step of calculating a relationship between the position of the reference position on the two-dimensional image and the actual position of the reference position; and a position estimation step of calculating the absolute coordinates of the estimation target object through the relationship; is provided.

In the step of arranging the reference position portion, the reference position portion may be arranged on different corners of a rectangle.

The method for estimating the position of an object in an image according to the present invention, which is performed after the image acquisition step, is an image conversion step of converting a two-dimensional image so that the reference position part is located on different corners of a rectangle on the two-dimensional image may further include.

The method for estimating the position of an object in an image according to the present invention is performed after the relation calculating step, and the image obtaining unit position calculation for calculating the position of the image obtaining unit for obtaining the two-dimensional image in the image obtaining step through the relation It may include further steps.

According to the system for estimating the position of an object in an image according to the present invention, the absolute coordinates of the actual estimated object can be easily calculated by photographing the reference position units placed on specific absolute coordinates and the estimated object together.

And there is no need to calibrate the camera as an image acquisition unit, nor is it necessary to calibrate the camera and the manipulator of the robot.

When the reference position units are located on the same plane or on different corners of a rectangle, it is possible to more easily calculate the absolute coordinates of the estimated target object.

When the system for estimating the position of an object in an image according to the present invention further includes an image conversion unit, it is possible for a person to intuitively confirm the actual position of the estimation target object only by checking the two-dimensional image.

When the reference position unit has a QR code image, it is possible to easily identify the absolute coordinates of each reference position unit.

1 is a schematic configuration diagram of a system for estimating the position of an object in an image according to the present invention;
2 is an explanatory view of the operation of the system for estimating the position of an object in an image according to the present invention;
3 is an explanatory view of an image conversion unit constituting a system for estimating the position of an object in an image according to the present invention;
4 is an explanatory diagram of an image acquisition unit position calculation unit constituting a system for estimating the position of an object in an image according to the present invention;
5 is a flowchart of a method for estimating the position of an object in an image according to the present invention.

Hereinafter, specific embodiments of the present invention will be described in detail with reference to the drawings.

1 is a schematic configuration diagram of a system 1 for estimating an object in an image according to the present invention, and FIG. 2 is an explanatory diagram illustrating an operation of the system 1 for estimating an object in an image according to the present invention is shown.

The system 1 for estimating the position of an object in an image according to the present invention includes a reference position unit 10 , an image acquisition unit 20 , a relationship calculation unit 30 , and a position estimation unit 40 .

Three or more reference position units 10 are formed, and each is placed at a different specific position. That is, each of the reference position units 10 is placed on specific absolute coordinates whose coordinates are known. For example, as shown in FIG. 2 , the reference position portion 10 is A(X1, Y1, Z1), B(X2, Y2, Z2), C(X3, Y3, Z3) and D(X4, Y4, Z4) may be located.

The image acquisition unit 20 acquires a two-dimensional image I including all three reference position units 10 and the estimated target object B. The image acquisition unit 20 may be, for example, a digital camera. In the two-dimensional image (I), the reference position part 10 of the point A is a (x1, y1) point, the reference position part 10 of the point B is the b(x2, y2) point, the reference position part of the point C ( 10) may be located at a point c(x3, y3), and the reference location unit 10 of the point D may be located at a point d(x4, y4).

The relationship calculating unit 30 calculates a relationship between the position of the reference position unit 10 ′ on the two-dimensional image I and the actual position of the reference position unit 10 . The relationship includes a ratio relationship indicating a change in size that occurs when the actual reference position portions 10 become the two-dimensional image I, and a ratio relationship that occurs when the actual reference position portions 10 become the two-dimensional image I There may be an angular relationship representing a change in angle. The ratio relationship may be calculated, for example, by comparing the distance between point A and point B and the distance between point a and point b, and comparing the distance between point A and point C and the distance between point a and point c. And the angular relationship is, for example, comparing the angle of the line segment connecting the point A and B and the angle of the line segment connecting the point a and b, the angle of the line segment connecting the point A and C, and the angle connecting the point a and c It can be calculated by comparing the angles of the line segments.

The position estimation unit 40 calculates the absolute coordinates of the estimation target object B through the relationship calculated by the relation calculation unit 30 . For example, the position estimator 40 obtains the coordinate system of the two-dimensional image I converted by photographing through the ratio relationship and the angle relationship, and then the estimation target object B' on the coordinate system of the two-dimensional image I By calculating the x-axis distance (dx) and y-axis distance (dy) of the point o(xo, yo) where the have.

According to the system for estimating the position of the object in the image according to the present invention (1), the actual estimation target object (B) is obtained by photographing the estimation target object (B) with the reference position units (10) placed on specific absolute coordinates. It is possible to easily calculate absolute coordinates.

Since the absolute coordinates on the reference position unit 10 are known, the absolute coordinates of the actual estimated target object B can be calculated even if the camera (image acquisition unit) is not calibrated. The same is true in case

In addition, when the manipulator is photographed together with the reference position parts 10 in the two-dimensional image I while working with the manipulator of the robot, the absolute coordinates of the manipulator can also be calculated, so the operation can be performed without calibration of the camera and manipulator. .

All of the reference position units 10 may be located on the same plane.

In this case, since it is easy to calculate the distance or angle between the actual reference position units 10 , the position of the reference position unit 10 ′ on the two-dimensional image I in the relationship calculator 30 and the actual reference position unit 10 ) It is possible to easily calculate the positional relationship, and thus, the absolute coordinates of the estimation target object B can be easily calculated by the position estimator 40 .

When all of the reference position units 10 are located on the same plane, it may be preferably applied to, for example, estimating the position of the estimation target on one shelf.

When all of the reference position units 10 are located on the same plane, the reference position units 10 may be located on different corners of a rectangle R.

In this case, since one reference position unit 10 and another reference position unit 10 adjacent to each other on the absolute coordinates differ only in the value of any one of the X axis, Y axis, and Z axis, the distance between the reference position units 10 or The angle calculation is easier, and the coordinate system of the two-dimensional image I can be easily obtained because the line connecting the adjacent reference position portions 10 is formed parallel to two axes of the X-axis, Y-axis, and Z-axis. Accordingly, the position estimator 40 may more easily calculate the absolute coordinates of the estimation target object B.

When the reference position unit 10 is positioned on different corners of a rectangle R, the system 1 for estimating the position of an object in an image according to the present invention may further include an image conversion unit 50 .

As shown in FIG. 3 , the image conversion unit 50 sets the two-dimensional image I on the two-dimensional image I so that the reference position part 10' is positioned on different corners of a rectangle R'. ) to convert The rectangle R' formed by the reference position portions 10' on the two-dimensional image I should be formed in the same proportion as the rectangle R formed by the actual reference position portions 10.

In this case, it is possible for a person to intuitively check the actual position of the estimated target object only by checking the two-dimensional image I. Accordingly, it is possible to easily perform an operation on the estimated target object B when, for example, a person remotely controls the robot.

The reference position unit 10 may include a QR code image 11 .

The QR code image 11 may include information about the absolute coordinates of the reference position unit 10 . Accordingly, it is possible to identify and identify the absolute coordinates of each reference position unit 10 only by photographing the QR code image 11 .

For example, when the robot performs a task while moving along several shelves, it is possible to easily identify which shelf the robot is performing the task on through the QR code image 11 .

And if the size of the work space can be freely changed by adjusting the arrangement position and number of the QR code images 11, and the QR code image 11 can be taken regardless of the type or location of the camera, the estimated target object (B) You can find the absolute coordinates of

The exact position of the reference position unit 10 may be the center of the QR code image 11 .

The system 1 for estimating the position of an object in an image according to the present invention may further include an image acquisition unit 20 and a position calculation unit 60 . 4 is an explanatory diagram of the image acquisition unit 20, the position calculation unit 60 is shown.

The image acquisition unit 20 position calculation unit 60 determines the absolute coordinates of the image acquisition unit 20 through the relationship between the position of the reference position unit 10 ′ on the two-dimensional image I and the actual position of the reference position unit 10 . Calculate the position on the

If the X-axis and Y-axis of the absolute coordinate system are aligned parallel to the x-axis and y-axis on the two-dimensional image (I), respectively, and the Z-axis of the absolute coordinate system is aligned parallel to the optical axis (Ao) of the camera, which is the image acquisition unit 20, the reference position The position of the image acquisition unit 20 may be calculated based on the absolute coordinates of the unit 10 and the coordinates on the two-dimensional image I.

More specifically, one reference position portion 10 A(X1, Y1, Z1) is projected on a point a(x1, y1) on the two-dimensional image I, with a focal length f as z1 and a two-dimensional image I ), the position of a becomes (x1, y1, z1) if the point a on Since the ratio of Z1 is the same and the ratio of y1 and f of point a on the two-dimensional image I is the same as the ratio of Y1 and Z1 of point A on the absolute coordinate system, the following relationship can be established.

,

,

Since the values of X1, Y1, and Z1 can be obtained through the absolute coordinates of the reference position unit 10, and the values of x1, y1 can be obtained through the two-dimensional image I, the value of z1 is calculated through the relationship It is possible to calculate the position of the image acquisition unit 20 .

When the position of the image acquisition unit 20 is calculated, the operation can be performed while more easily moving the robot equipped with the image acquisition unit 20 with respect to the estimation target object (B).

The system 1 for estimating the position of an object in an image according to the present invention may further include a manipulator, a storage unit, a control unit, and the like, in addition to the above-described configuration.

The manipulator (not shown) may perform a physical operation on the estimation target object B based on the absolute coordinates of the estimation target object B calculated by the position estimator 40, and a storage unit (not shown) in the absolute coordinates of the reference position unit 10 , the two-dimensional image I acquired by the image acquisition unit 20 , the relationship calculated by the relationship calculation unit 30 , and the estimated target object calculated by the position estimation unit 40 . The absolute coordinates of (B), a program for calculating a relationship, a program for calculating absolute coordinates, and the like may be stored. And the controller (not shown) controls the operation of the image acquisition unit 20, the relationship calculation unit 30, the position estimation unit 40, and the storage unit to calculate the absolute coordinates of the estimation target object B, and the calculated The operation of the manipulator may be controlled based on the absolute coordinates of the estimation target object B.

Hereinafter, a method for estimating the position of an object in an image according to the present invention will be described. While describing the method for estimating the position of an object in an image according to the present invention, detailed descriptions of the parts mentioned in the description of the system 1 for estimating the position of an object in an image of the present invention may be omitted.

5 is a flowchart of a method for estimating the position of an object in an image according to the present invention.

The method for estimating the position of an object in an image according to the present invention includes a reference position unit arrangement step (S10), an image acquisition step (S20), a relationship calculation step (S30), and a location estimation step (S40).

In the reference position unit arrangement step (S10), three or more reference position units 10 are arranged on different specific absolute coordinates. The reference location units 10 are preferably arranged on the same plane so that the relationship calculation step S30 and the location estimation step S40 to be described below can be easily performed.

In the image acquisition step (S20), a two-dimensional image (I) including both the reference position unit (10) and the estimated target object (B) is acquired using a camera.

In the relation calculation step (S30), the relationship between the position of the reference position unit 10 on the two-dimensional image I and the position of the actual reference position unit 10 is calculated. Depending on the distance or angle between the camera and the reference location part 10, the distance between the reference location parts 10' on the two-dimensional image I or the angle of the lines connecting the adjacent reference location parts 10 is the actual reference location part (10) may be different. Accordingly, the relationship includes a ratio relationship due to the distance between the actual reference location parts 10 and the distance difference between the reference location parts 10' on the two-dimensional image I, and the actual reference location parts 10 An angular relationship representing a change in angle that occurs while the two-dimensional image I is formed may be included.

In the position estimation step (S40), the absolute coordinates of the estimated target object (B) are calculated through the relationship calculated in the relationship calculation step (S30). More specifically, by obtaining the coordinate system of the two-dimensional image I through the ratio relationship and the angular relationship, and confirming the position of the estimated object B' on the coordinate system of the two-dimensional image I, the absolute coordinates can be calculated.

In the reference position part arrangement step ( S10 ), the reference position part 10 may be arranged on different corners of one rectangle R. In this case, since it is easy to calculate the distance or angle between the reference position units 10 , it is possible to more easily calculate the absolute coordinates of the estimation target object B .

When the reference position unit 10 is arranged on different corners of a rectangle (R) in the reference position unit arrangement step (S10), the method for estimating the position of an object in an image according to the present invention may further include an image conversion step. can

The image conversion step is performed after the image acquisition step (S20), and the two-dimensional image (I) so that the reference position part (10') is located on different corners of a rectangle (R') on the two-dimensional image (I) can be converted

In this case, since the actual position of the estimated target object can be intuitively confirmed through the two-dimensional image I, it is possible to easily perform a task when a person remotely controls the robot.

The method for estimating the position of an object in an image according to the present invention may further include calculating the position of the image acquisition unit.

The image acquisition unit position calculation step is performed after the relationship calculation step (S30), and the image acquisition unit ( 20) is calculated.

The position of the image acquisition unit 20 is set in a state where the X and Y axes of the absolute coordinate system are aligned parallel to the x and y axes of the two-dimensional image I, respectively, and the Z axis of the absolute coordinate system is aligned parallel to the optical axis of the camera. It can be calculated through the absolute coordinates of the position unit 10 and the coordinates of the reference position unit 10 ′ on the two-dimensional image I.

When the position of the image acquisition unit 20 is calculated, the robot can perform the operation while accurately moving with respect to the estimated target object B.

The system 1 and method for estimating the position of an object in an image according to the present invention may be implemented through one or more hardware, or may be implemented through one or more software. Alternatively, it may be implemented through hardware and software.

The scope of the present invention is not limited to the above-described embodiments, but may be implemented in various types of embodiments within the scope of the appended claims. Without departing from the gist of the present invention claimed in the claims, it is considered to be within the scope of the claims of the present invention to various extents that can be modified by any person skilled in the art to which the invention pertains.

1: System for estimating the position of an object in the image
10: reference position part 11: QR code image
20: image acquisition unit 30: relationship calculation unit
40: position estimation unit 50: image conversion unit
60: position calculation unit

Claims (10)

three or more reference position units placed on different specific absolute coordinates;
an image acquisition unit configured to acquire a two-dimensional image including both the reference position unit and the estimated target object;
a relationship calculator configured to calculate a relationship between the position of the reference position on the two-dimensional image and the actual position of the reference position; and
and a position estimator for calculating the absolute coordinates of the estimation target object through the relationship.
According to claim 1,
The system for estimating the position of an object in an image, characterized in that all of the reference position units are located on the same plane.
3. The method of claim 2,
The system for estimating the position of an object in an image, wherein the reference position part is located on different corners of a rectangle.
4. The method of claim 3,
The system for estimating the position of an object in an image, further comprising an image converter that converts the two-dimensional image so that the reference position part is located on different corners of a rectangle on the two-dimensional image.
According to claim 1,
The system for estimating the position of an object in an image, wherein the reference position unit includes a QR code image including information on the absolute coordinates of the reference position unit.
According to claim 1,
The system for estimating the position of an object in an image, further comprising an image acquisition unit position calculation unit for calculating the position on the absolute coordinates of the image acquisition unit through the relationship.
a reference position unit arrangement step of arranging three or more reference position units on different specific absolute coordinates;
an image acquisition step of acquiring a two-dimensional image including both the reference position part and the estimated target object;
a relationship calculation step of calculating a relationship between the position of the reference position on the two-dimensional image and the actual position of the reference position; and
and a position estimation step of calculating the absolute coordinates of the estimation target object through the relationship.
8. The method of claim 7,
In the arranging of the reference position unit, the reference position unit is disposed on different corners of a rectangle.
9. The method of claim 8,
As proceeds after the image acquisition step,
The method for estimating the position of an object in an image, further comprising an image conversion step of transforming the two-dimensional image so that the reference position part is located on different corners of a rectangle on the two-dimensional image.
8. The method of claim 7,
As proceeding after the relationship calculation step,
The method for estimating the position of an object in an image, further comprising an image acquisition unit position calculation step of calculating a position of the image acquisition unit acquiring the two-dimensional image in the image acquisition step through the relationship.

Images