KR20210057964A - Method and system for determining location information of moving object with photography apparatus - Google Patents

Abstract

Disclosed are a method and system for determining location information of a moving object with a photographing apparatus. The present invention includes the steps of: photographing a reference body using the photographing apparatus; detecting at least four coordinates from the reference body in the photographed image; and determining the location information of the moving object based on the detected coordinates. The method and system for determining the position information of a moving object with a photographing apparatus according to the present invention can have an effect of determining the exact location information of the moving object without a sensor.

Description

METHOD AND SYSTEM FOR DETERMINING LOCATION INFORMATION OF MOVING OBJECT WITH PHOTOGRAPHY APPARATUS}

The present invention relates to a method and system for determining position information of a moving object equipped with photographing equipment, and more particularly, to a method and system for determining position information of a moving object using at least four coordinates.

Unmanned aerial vehicles are in the spotlight from various aspects, such as military and industrial, because they can easily collect information from the ground and air at high altitudes without risk of exposure to others without the risk of users.

Today's unmanned aerial vehicles have a very high degree of freedom to measure their position, speed, and posture, create an optimal path for a given mission, and fly accordingly, diagnose and respond to failures on their own. Recently, satellite navigation systems and sensors. Civilian drones equipped with cameras have been developed, and the scope of use is expanding to fields such as transportation of goods, traffic control, and security.

In order to control the unmanned aerial vehicle as described above, it is essential to know the location information, and in the related art, it has been common to know the location information of the unmanned aerial vehicle by further separately providing a sensor such as GPS.

However, since such a GPS sensor cannot stably receive a GPS signal indoors and has a large error range outdoors, it is difficult to accurately determine the location information of a moving object 20 such as an unmanned aerial vehicle.

In particular, in the case of unmanned aerial vehicles, the number of cases requiring landing at an accurate and precise location is increasing, and in this case, there is a limitation with only the conventional GPS sensor.

Registration Patent No. 10-1483059 (Name of invention: Unmanned Vehicle Control System)

The present invention was conceived to solve the above problems, and the method and system for determining the location information of a moving object equipped with photographing equipment of the present invention uses only photographing equipment without having an additional sensor such as a GPS sensor. It aims to determine the exact location information of the moving object.

In order to solve the above problems, the present invention includes the steps of photographing a reference body using a photographing device; Detecting at least 4 coordinates from a reference body in the captured image; And determining location information of the moving object based on the detected coordinates.

The reference body may include at least four preset coordinates.

The at least four preset coordinates may be transmitted to the communication unit of the moving object when the moving object moves within a communication radius of the reference object.

In this case, the at least four coordinates are set based on a world coordinate system, and the detecting may convert the at least four coordinates into a camera coordinate system using an internal parameter.

The detecting may include checking whether at least four coordinates are correctly detected; And re-detecting the at least four coordinates when the coordinates are not correctly detected as a result of the confirmation.

In the detecting step, the at least four coordinates may be detected in order to calculate four unknowns of the following equation.

은

를 기준으로

만큼 회전한 회전행렬,

은

를 기준으로 회전한 각도,

는

의 x성분,

는

의 y성분,

는

의 z성분)(only,

silver

Based on

Rotation matrix rotated by

silver

The angle rotated based on,

Is

X component of,

Is

Y component of,

Is

Z component of)

In addition, the present invention in order to solve the above-described problem, the reference body including at least four coordinates; A moving body including a photographing device for photographing the reference body; And a processor that detects at least four coordinates based on the captured image and determines location information of the moving object based on the detected coordinates.

The reference body includes at least four corners, and the at least four coordinates may correspond to the corners.

The reference body may include at least four preset coordinates.

The moving object may include a communication unit configured to receive the at least four coordinate information when the moving object moves within a communication radius of the reference body.

As described above, the method and system for determining the location information of a moving object equipped with a photographing device of the present invention has the effect of determining the exact location information of the moving object using only the photographing equipment without having an additional sensor such as a GPS sensor. have.

1 is a diagram showing a system for determining location information of a moving object equipped with photographing equipment according to a first embodiment of the present invention.
2 is a diagram showing a method of determining location information of a moving object equipped with photographing equipment according to a second embodiment of the present invention.
3 is a diagram showing a geometric structure of a rotation vector for determining position information of a moving object according to the present invention.
4 is a diagram showing a coordinate system in space for determining location information of a moving object according to the present invention.
5 is a diagram showing the correspondence of coordinate systems in space for determining position information of a moving object according to the present invention.
6 is a diagram showing in detail a method of determining location information of a moving object equipped with photographing equipment according to a second embodiment of the present invention.
7 is a photograph showing a reference body of 0.47m × 0.47m used in Experimental Example 1 according to the present invention.
8 is a photograph showing the detection of pixel coordinates at the corners of the reference body used in Experimental Example 1 according to the present invention.
9 is a diagram showing pixel coordinates on all images detected in Experimental Example 1 according to the present invention.
10 is a diagram showing positions of various cameras in Experimental Example 2 according to the present invention.
11 is a diagram showing a rotation vector in Experimental Example 2 according to the present invention.
12 is a diagram showing a rotation vector in Experimental Example 2 according to the present invention.
13 is a diagram illustrating detection of pixel coordinates using an image in Experimental Example 2 according to the present invention.
14 is a diagram illustrating detection of pixel coordinates using an image in Experimental Example 2 according to the present invention.

In the present invention, various transformations may be applied and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to be limited to a specific embodiment of the present invention, it is to be understood to include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, when it is determined that a detailed description of a related known technology may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Terms such as first and second may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another component.

The terms used in the present invention are used only to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof does not preclude in advance.

Therefore, the spirit of the present invention is limited to the described embodiments and should not be defined, and all things that are equivalent or equivalent to the claims as well as the claims to be described later fall within the scope of the spirit of the present invention. .

Hereinafter, a system for determining location information of a moving object equipped with photographing equipment according to a first embodiment of the present invention will be described in detail as follows.

1 is a diagram showing a system for determining location information of a moving object equipped with photographing equipment according to an embodiment of the present invention.

Referring to FIG. 1, a system for determining location information of a moving body 20 having a photographing device 21 according to an embodiment of the present invention includes a reference body 10, a moving body 20, and a processor 22. Includes.

According to FIG. 1, the reference body 10 includes at least four coordinates. The coordinates included in the reference body 10 have a preset coordinate value. These coordinate values may be set with the center of the reference body 10 as the origin (0, 0).

According to FIG. 1, the reference body 10 is a quadrangular shape, and each corner is set as a first coordinate 11, a second coordinate 12, a third coordinate 13, and a fourth coordinate 14, respectively. Corners and coordinates can be associated. In addition, each of the coordinate values has a preset value.

However, the reference body 10 is not limited to a shape of a square, and may be a polygon having n corners. In some cases, it may be circular. In addition, the position of each coordinate does not have to be located only at the corner of the polygon.

The moving body 20 may include a photographing equipment 21.

The moving body 20 may be an unmanned aerial vehicle, preferably a drone.

In addition, the moving body 20 may include a processor 22 and a communication unit (not shown). As the moving body 20 moves, the reference body 10 is photographed through the photographing equipment 21. In this case, the processor 22 detects the coordinates of the reference body 10 based on the captured image or image. The processor 22 determines the location information of the moving object 20 based on the detected coordinates.

In addition, each of the preset coordinate values of the reference body 10 may be values already known to the moving body 20. That is, the moving object 20 may include a storage unit (not shown) capable of storing data, and each preset coordinate value may be stored in the storage unit (not shown).

The storage unit (not shown) may be selected from one or more of a hard disk drive (HDD), a solid state drive (SSD), a DRAM memory, an SRAM memory, a FRAM memory, and a flash memory. In addition, the reference body 10 may include a communication means (not shown). The communication means (not shown) of the reference body 10 may form a communication radius of a certain range. Therefore, when the moving body 20 moves inside the communication radius, the communication means (not shown) of the reference body 10 is a predetermined value of the reference body 10 through the communication unit (not shown) of the moving body 20. Each coordinate value (coordinate information) can be transmitted.

A method for the processor 22 to detect at least four coordinates based on the photographing equipment 21 included in the moving body 20 may be according to a second preferred embodiment of the present invention, which will be described below.

The processor 22 may determine the location information of the moving object 20 based on the detected at least four coordinates, and a method of determining the location information may follow a second preferred embodiment of the present invention to be described below. However, the location information may include the current location and inclination of the moving body 20.

Hereinafter, a detailed description will be given of a method of determining location information of a moving object equipped with photographing equipment according to a second preferred embodiment of the present invention.

In addition, it is preferable that the subject performing the method of determining the location information of the moving object equipped with the photographing equipment is a system that determines the position information of the moving object equipped with the photographing equipment.

For reference, descriptions of configurations that have the same or similar characteristics to those of the first embodiment of the present invention will be omitted and only different points will be described.

2 is a diagram showing a method of determining location information of a moving object equipped with photographing equipment according to a second embodiment of the present invention.

According to Figure 2, the present invention is a step of photographing the reference body 10 using the photographing equipment 21 (S100), the step of detecting at least four coordinates from the reference body 10 in the captured image (S200) , And determining location information of the moving body 20 based on the detected coordinates (S300).

In the step (S100) of photographing the reference body 10 using the photographing equipment 21, the photographing equipment 21 included in the moving body 20 may be used. The photographing equipment 21 may mean any means or equipment capable of photographing an image or a video. The photographing equipment 21 in the present invention may mainly mean a camera. In addition, the expression "video" used in the present invention should be interpreted as including an image such as a photograph.

In the step of detecting at least four coordinates from the reference body 10 in the captured image (S200), the processor 22 of the moving body 20 may analyze the image and detect at least four coordinates. Further, in the step S300 of determining the location information of the moving body 20 based on the detected coordinates, the location information of the moving body 20 is determined based on at least four coordinate values. At this time, at least four coordinates may be included in the reference body 10, and information on these coordinates may be previously stored in the mobile body 20 or transmitted through communication.

As described above, the reason why at least four coordinate values are required in the present invention is as follows.

In order to grasp the location information of the moving body 20, a rotation matrix may be used. The rotation matrix is divided into Roll, Pitch, and Yaw according to the largely rotating reference axis. Here, it is said that a rotation matrix is created by applying in the order of Roll, Pitch, and Yaw. I assume. However, Roll, Pitch, and Yaw refer to expressions representing an angle that means the posture of the moving body 20.

When the elevator of the moving body 20 is manipulated, the front (rider) of the moving body 20 moves up and down, and this movement is called a pitch. When the auxiliary wing (aileron) on the main wing is manipulated, the moving body 20 is inclined to the left or right as a whole, and this motion is called a roll. When the rudder is used, the direction of flight is changed as the nose of the moving body 20 moves left and right, and this movement is called yaw. These expressions can also be used for aircraft, drones, etc.

순으로 회전행렬이 적용된다고 하면 아래 [수학식 1]을 이용하여 [수학식 2]와 같이 정리할 수 있다.

Assuming that the rotation matrix is applied in order, it can be summarized as [Equation 2] using [Equation 1] below.

은 x축을 회전 기준 축으로 설정한 회전행렬,

은 y축을 회전 기준 축으로 설정한 회전행렬,

은 z축을 회전 기준 축으로 설정한 회전행렬,

은 x축을 회전 기준 축으로 회전한 각도,

은 y축을 회전 기준 축으로 회전한 각도,

은 z축을 회전 기준 축으로 회전한 각도)(only,

Is the rotation matrix with the x-axis set as the rotation reference axis,

Is the rotation matrix with the y-axis set as the rotation base axis,

Is the rotation matrix with the z-axis as the rotation reference axis,

Is the angle rotated around the x-axis as the rotation reference axis,

Is the angle rotated by the y-axis to the rotation base axis,

Is the angle rotated by the z-axis as the rotation reference axis)

은 x축을 회전 기준 축으로 설정한 회전행렬,

은 y축을 회전 기준 축으로 설정한 회전행렬,

은 z축을 회전 기준 축으로 설정한 회전행렬,

은 x축을 회전 기준 축으로 회전한 각도,

은 y축을 회전 기준 축으로 회전한 각도,

은 z축을 회전 기준 축으로 회전한 각도)(only,

Is the rotation matrix with the x-axis set as the rotation reference axis,

Is the rotation matrix with the y-axis set as the rotation base axis,

Is the rotation matrix with the z-axis as the rotation reference axis,

Is the angle rotated around the x-axis as the rotation reference axis,

Is the angle rotated by the y-axis to the rotation base axis,

Is the angle rotated by the z-axis as the rotation reference axis)

3 is a diagram showing a geometric structure of a rotation vector for determining position information of a moving object according to the present invention.

은 아래 [수학식 4]와 같이 나타낼 수 있다.According to Rodrigues' rotation fomula, an arbitrary rotation matrix can be expressed as [Equation 3] below as the rotation angle of the coordinate system with respect to the rotation reference vector and the reference vector. Also, the rotated vector

Can be expressed as [Equation 4] below.

은

를 기준으로

만큼 회전한 회전행렬,

은

를 기준으로 회전한 각도,

는 항등행렬,

는 회전기준 벡터,

는

의 교대행렬)(only,

silver

Based on

Rotation matrix rotated by

silver

The angle rotated based on,

Is the identity matrix,

Is the rotational reference vector,

Is

Shift matrix)

은 회전을 취하려 하는 벡터,

은

의

에 대해 평행한 성분,

은

의

에 대해 수직한 성분)(only,

Is the vector that is about to take a rotation,

silver

of

Components parallel to,

silver

of

Component perpendicular to)

If the geometric structure of the rotation vector according to FIG. 3 is used, the following [Equation 5] can be obtained. According to [Equation 5], the entire rotation matrix can be obtained by knowing only four values for the rotation reference vector and the rotation angle.

That is, since there are only four unknowns in the following [Equation 5], if at least four coordinates are known, the unknown number of [Equation 5] below can be found using the equation.

은

를 기준으로

만큼 회전한 회전행렬,

은

를 기준으로 회전한 각도,

는

의 x성분,

는

의 y성분,

는

의 z성분)(only,

silver

Based on

Rotation matrix rotated by

silver

The angle rotated based on,

Is

X component of,

Is

Y component of,

Is

Z component of)

4 is a diagram showing a coordinate system in space for determining location information of a moving object according to the present invention.

In the present invention, since it is necessary to find at least four coordinates with the photographing equipment 21 included in the moving object 20, the coordinates (plane coordinates) of the reference body 10 present in the image photographed by the photographing equipment 21 are It is necessary to convert the coordinates (world coordinates) of the reference body 10 existing in the actual three-dimensional space.

That is, the at least four coordinates are set based on a world coordinate system, and the detecting may convert the at least four coordinates into a camera coordinate system using an internal parameter.

로 표시된 축으로 이루어진 좌표계가 월드 좌표계이다. 또한,

로 표시된 축으로 이루어진 좌표계가 카메라 좌표계이다. 또한,

로 표시된 축으로 이루어진 좌표계가 픽셀 좌표계이다. 예를 들어, 좌표값 중 적어도 한 점의 좌표가 월드 좌표계라면 해당 좌표를 월드 좌표라고 하며, 이는 다른 좌표계를 사용한 경우에도 마찬가지이다. According to Figure 4,

The coordinate system consisting of the axes indicated by is the world coordinate system. Also,

The coordinate system composed of the axes indicated by is the camera coordinate system. Also,

The coordinate system composed of the axes indicated by is the pixel coordinate system. For example, if the coordinates of at least one of the coordinate values is a world coordinate system, the corresponding coordinates are called world coordinates, and this is the same even when a different coordinate system is used.

However, since world coordinates, camera coordinates, and pixel coordinates can be converted to each other in some cases, the scope of the present invention cannot be regarded as being limited to a specific coordinate system.

와 월드 좌표

는 아래 [수학식 6]으로 표현된다. 이때,

은 동차 좌표계상의 회전 변환과 이동 변환을 나타낸 것이며,

는 4차원 동차 좌표계를 3차원 동차 좌표계로 변환해주는 행렬이고,

는 촬영장비(21) 내부의 파라미터를 모델링한 행렬이다. Plane coordinates

And world coordinates

Is expressed by the following [Equation 6]. At this time,

Represents the rotation transformation and movement transformation in the homogeneous coordinate system,

Is a matrix that converts a 4D homogeneous coordinate system to a 3D homogeneous coordinate system,

Is a matrix modeling parameters inside the photographing equipment 21.

As such, linear transformations that transform the coordinate system include Scale, Rotation, and Translation transformations. Since the commutative law does not hold between matrices, in general, when dealing with linear transformations, scale, rotation, and translation It is common to use movement sequentially (TRS).

,

,

)(At this time,

,

,

)

는 상수,

는 픽셀 좌표계상의 x좌표,

는 픽셀 좌표계상의 y좌표,

는 카메라 초점거리,

내지

은 회전벡터 R의 성분,

내지

는 이동벡터의 성분,

는 월드좌표계 상의 타겟의 x좌표,

는 월드좌표계 상의 타겟의 y좌표,

는 월드좌표계 상의 타겟의 z좌표,

은 회전행렬,

는 이동벡터)(only,

Is a constant,

Is the x coordinate in the pixel coordinate system,

Is the y coordinate in the pixel coordinate system,

Is the camera focal length,

To

Is the component of the rotation vector R,

To

Is the component of the motion vector,

Is the x coordinate of the target in the world coordinate system,

Is the y coordinate of the target in the world coordinate system,

Is the z-coordinate of the target in the world coordinate system,

Is the rotation matrix,

Is the moving vector)

축을 순서로 롤(Roll), 피치(Pitch), 요(Yaw)만큼 회전한다고 약속한다. 이때, 점을 회전시키는 것이 아니라 좌표계의 축을 회전시키는 것이므로 이동각에 (-)부호를 붙여주어야 한다. [수학식 1]의 행렬들의 회전각에 (-)부호를 붙여 정리하면 [수학식 7]과 같이 회전행렬이 표현된다. 위의 정보들을 종합하여 TRS행렬을 구하여 보면 상기 [수학식 6]의

과 같이 표현할 수 있다.Since the scale does not differ between the world coordinate system and the camera coordinate system, the scale matrix becomes a unit matrix. Next, in the case of the rotation matrix

It promises to rotate the axis in order by Roll, Pitch, and Yaw. At this time, it does not rotate the point, but rotates the axis of the coordinate system, so a (-) sign must be added to the moving angle. If the rotation angle of the matrices of [Equation 1] is summarized by adding a (-) sign, the rotation matrix is expressed as in [Equation 7]. If the TRS matrix is obtained by synthesizing the above information, the above [Equation 6]

It can be expressed as

는 z축을 회전 기준 축으로 설정한 회전행렬,

는 y축을 회전 기준 축으로 설정한 회전행렬,

는 x축을 회전 기준 축으로 설정한 회전행렬,

는 은 z축을 회전 기준 축으로 회전한 각도,

는 y축을 회전 기준 축으로 회전한 각도,

는 x축을 회전 기준 축으로 회전한 각도)(only,

Is the rotation matrix with the z-axis as the rotation reference axis,

Is the rotation matrix with the y-axis set as the rotation reference axis,

Is the rotation matrix with the x-axis set as the rotation base axis,

Is the angle at which the z-axis is rotated about the rotation reference axis,

Is the angle at which the y-axis is rotated about the rotation reference axis,

Is the angle at which the x-axis is rotated to the axis of rotation)

5 is a diagram showing the correspondence of coordinate systems in space for determining position information of a moving object according to the present invention.

According to FIG. 5, since a camera coordinate system, a normal coordinate system, a pixel coordinate system, and a world coordinate system are shown, each coordinate system can be easily understood.

As described above, the present invention needs to find out the coordinate system using the photographing equipment 21 provided in the moving object 20, and the coordinates of the reference body 10 set based on the three-dimensional world coordinate system are converted to the camera coordinate system to enable recognition. There is a need to do it. In order to meet this need, the present invention uses the above coordinate system transformation.

The present invention seeks to obtain the position and posture of the camera when the correspondence between the point of the reference body 10 on the world coordinate system and the point in the camera image is known. In order to accurately grasp the camera's position and posture, you need to know two conditions. First, it is necessary to know the focal length of the camera and the size of one pixel on the image sensor. Second, it is necessary to know the 3D coordinates of the four points in the world coordinate system and the 2D coordinates in the image corresponding thereto.

As previously discussed, at least 4 pieces of information are required to express the rotation matrix. Therefore, three elements of the movement matrix and four elements of the rotation matrix must be obtained. Since you can get 2 equations per coordinate, if you know the coordinates corresponding to the image of 4 coordinates in the world coordinate system, you can get 8 equations, which is enough to solve the equation.

In general, coordinates on an image are expressed in pixels, so you need to multiply the size of the pixel and input the coordinates in [m]. Since the process of solving the equation is very complex, it is calculated using the processor 22 included in the moving body 20 of the present invention.

는 회전변환,

는 스케일 변환,

는 이동변환)(only,

Is the rotational transformation,

Is the scale transform,

Is shift transformation)

는 회전변환,

는 스케일 변환,

는 이동변환,

내지

은

내지

은 회전벡터

의 성분,

내지

는 이동벡터의 각 성분,

는 월드좌표계 상의 타겟의 x좌표,

는 월드좌표계 상의 타겟의 y좌표,

는 월드좌표계 상의 타겟의 z좌표

은 회전행렬,

는 이동벡터)(only,

Is the rotational transformation,

Is the scale transform,

Is the shift transformation,

To

silver

To

Is a rotation vector

Ingredients of,

To

Is each component of the motion vector,

Is the x coordinate of the target in the world coordinate system,

Is the y coordinate of the target in the world coordinate system,

Is the z coordinate of the target in the world coordinate system

Is the rotation matrix,

Is the moving vector)

은 회전변환(

)의 역행렬,

는

행렬의 i행 j열 성분,

는 x축을 회전 기준 축으로 회전한 각도,

는 y축을 회전 기준 축으로 회전한 각도,

는 z축을 회전 기준 축으로 회전한 각도)(only,

Is a rotational transformation (

) Inverse matrix,

Is

The components in row i and j of the matrix,

Is the angle at which the x-axis is rotated about the rotation reference axis,

Is the angle at which the y-axis is rotated about the rotation reference axis,

Is the angle rotated by the z-axis as the rotation reference axis)

In addition, the step of determining the location information of the moving body 20 (S300), calculates an experimental value based on the coordinates detected using [Equation 9] and [Equation 10], and at this time, the calculated experimental value and By comparing the stored theoretical values, it is possible to determine the location information of the moving body 20. That is, if the calculated experimental value and the previously stored theoretical value are the same, it may be determined that the moving object 20 is located at a desired position. Conversely, if there is a difference between the calculated experimental value and the previously stored theoretical value, it may be determined that the moving object 20 is not in a desired position.

6 is a diagram showing in detail a method of determining location information of a moving object equipped with photographing equipment according to a second embodiment of the present invention.

Referring to FIG. 6, an algorithm capable of calculating the position and posture of the photographing device 21 of the moving object 20 with respect to the world coordinate system set in the reference body 10 using the above equations is shown.

According to Fig. 6, the algorithm includes: ① extracting information on four points on the world coordinate system, ② detecting the coordinates of the corresponding pixel, ③ confirming whether the correct pixel coordinates are detected, and ④ substituting the coordinate pairs. It may include the step of detecting the internal parameters of the photographing equipment 21, ⑤ outputting a result, and the like.

In step ① above, four points (world coordinates) of an object in real space that know the size and posture are input. In step ②, the image including the four points in step ① is obtained, and the relative coordinates (pixel coordinates) of the four points with respect to the origin of the pixel coordinate system are obtained. In step ③, it is checked whether the pixel coordinates of the four points in the world coordinate system are correctly detected on the photographed image, and if the four coordinates are not correctly detected, it is detected again. This process usually uses a feature point detection technique. In step ④, internal parameters of the photographing equipment 21 that can be substituted for the coordinate pairs obtained in step ③ are detected. Here, the internal parameters mean [Equation 6], which is a matrix connecting the world coordinate system and the pixel coordinates. In step ⑤, values for the position and posture of the camera are obtained from the internal factors obtained in step ④. These four processes are described below using one experimental example.

[Experimental Example 1]

Step ①: Enter four points on the world coordinate system

For convenience, set the world coordinate system and enter the coordinates on the set coordinate system at four points on the object. In the example, the center of the rectangle is set as the origin. Using the four coordinates in the world coordinate system of the four vertices of the square tile as the origin of the world coordinate system (-0.235, -0.235, 0), (-0.235, 0.235, 0), (0.235, -0.235, 0) Set as, (0.235, 0.235, 0). The four points of the square-shaped landing point are standardized so that information can be exchanged using short-range communication when you already know or come near the landing point. The square-shaped landing point used at this time is the role of the reference body 10 and is shown in FIG. 7.

② Step: Detect the corresponding pixel coordinates

8 is a photograph showing the detection of pixel coordinates at the corners of the reference body 10 used in the experimental example according to the present invention, and FIG. 9 is a view showing the pixel coordinates on all images detected in the experimental example according to the present invention. to be

According to FIG. 8, it is possible to find the coordinates of the lower left corner and find the pixel coordinates corresponding thereto. In this way, you can find all the pixel coordinates corresponding to the coordinates of each corner.

According to FIG. 9, it can be seen that the coordinates of four points set in the corresponding object on the image are detected, and the pixel coordinates of (-0.235, -0.235, 0) become (382, 254). have. Similarly, we get the pixel coordinates of all corners as shown in the right figure.

③ Step: Check whether the pixel coordinates are detected correctly

Check whether the four pixel coordinates on the image plane corresponding to the four coordinates in the world coordinate system have been correctly detected, and if they are erroneously detected or not, try detection again.

④ Step: Coordinate pair assignment internal parameter detection

It is necessary to convert the pixel coordinates to coordinates based on the center point of the image, convert them to coordinates on the camera coordinate system, and multiply the coordinates by the size of the pixels. Table 1 shows the conversion of the pixel coordinates to the camera coordinate system.

Pixel coordinates Image center point reference Camera coordinate system Origin of the camera coordinate system (360, 480) (0,0) (0,0) 3rd quadrant corner coordinates (378, 258) (18, 222) (18, -222) 2 quadrant corner coordinates (377, 21) (17, 459) (17, -459) Quadrant corner coordinates (616, 259) (256, 221) (256, -221) First quadrant corner coordinates (611, 26) (251, 454) (251, -454)

The image center point reference coordinate is a coordinate obtained again by setting the origin of the camera coordinate system as (0,0) using the following [Equation 11]. Since the camera coordinate system is inverted on the y-axis with respect to the world coordinate system, a (-) sign is added to the value of the y component in the image center reference coordinate as shown in [Equation 12]. The obtained pixel coordinates in the camera coordinate system are input by multiplying the pixel size. The world coordinate system and the camera coordinate system can be set flexibly, so if you set the coordinate system differently from the example, you can perform an operation that suits the situation. Through this process, coordinates can be obtained as shown in [Table 1], and internal parameters are obtained using the obtained coordinates. The position and posture are obtained by using [Equation 8], [Equation 9], and [Equation 10] for the obtained internal parameters.

는 이미지의 원점을 픽셀 좌표계의 중심으로 설정한 좌표계 상의 x좌표,

는 이미지의 원점을 픽셀 좌표계의 중심으로 설정한 좌표계 상의 y좌표,

는 픽셀좌표계 상의 x좌표,

는 이미지 너비를 차지하는 픽셀 수를 반으로 나눈 값,

는 픽셀좌표계 상의 y좌표,

는 이미지 높이를 차지하는 픽셀 수를 반으로 나눈 값)(only,

Is the x coordinate on the coordinate system where the origin of the image is set as the center of the pixel coordinate system,

Is the y coordinate on the coordinate system where the origin of the image is set as the center of the pixel coordinate system,

Is the x coordinate in the pixel coordinate system,

Is the number of pixels occupying the width of the image divided by half,

Is the y coordinate in the pixel coordinate system,

Is the number of pixels occupying the image height divided by half)

는 이미지의 원점을 픽셀 좌표계의 중심으로 설정한 좌표계 상의 x좌표,

는 이미지의 원점을 픽셀 좌표계의 중심으로 설정한 좌표계 상의 y좌표,

는 픽셀 좌표계 상의 x좌표,

는 픽셀 좌표계 상의 y좌표)(only,

Is the x coordinate on the coordinate system where the origin of the image is set as the center of the pixel coordinate system,

Is the y coordinate on the coordinate system where the origin of the image is set as the center of the pixel coordinate system,

Is the x coordinate in the pixel coordinate system,

Is the y coordinate in the pixel coordinate system)

Step ⑤: Print the result

Theoretical value Experimental value error Translation Vector Rotation Vector Translation Vector Rotation Vector Translation Vector Rotation Vector -0.235

(3.141

(3.141 -0.23458386 3.14469834 0.00042 0.003698 -0.685 0 -0.72302979 0.00636603 0.03803 0.006366 1.250 0 1.20343671 0.0487402 0.04656 0.04874

[Experimental Example 2]

10 is a diagram showing positions of various cameras in Experimental Example 2 according to the present invention.

According to FIG. 10, based on the same experimental steps as in Experimental Example 1, using a 3.93mm focal length camera, a 0.47m×0.47m square was placed on the floor, and the center of the square was set as the origin, and then photographed from eight positions The camera's position was tested, pan 45°, and -45° were taken to test the camera angle.

Theoretical value (position of the camera) Experimental value error ①(-0.235, -0.685,1.250) (-0.2345,-0.7230,1.2034) (0.0004,0.0380,0.0465) ②(0.235, -0.685, 1.250) (0.1867, -0.7202, 1.2048) (0.0482,0.0352,0.0451) ③(0.685, -0.235, 1.250) (0.7195, -0.2416, 1.1945) (0.4845,0.4433,0.0554) ④(0.685, 0.235, 1.250) (0.6803, -0.2039, 1.2036) (0.0155,0.4389,0.0464) ⑤(-0.235,0.685,1.250) (-0.2327,0.6854,1.1945) (0.0023,0.0004,0.0555) ⑥(0.235, 0.685, 1.250) (0.2326, 0.6827, 1.1936) (0.0024,0.0023,0.0564) ⑦(-0.685, -0.235, 1.250) (-0.7034, -0.2364, 1.2001) (0.0184,0.0014,0.0499) ⑧(-0.685, 0.235, 1.250) (-0.6929, 0.2480, 1.220) (0.0079,0.013,0.03)

11 and 12 are diagrams showing a rotation vector in Experimental Example 2 according to the present invention.

Referring to FIG. 11, a rotation vector based on a camera at a certain position is shown, and the rotation vector is as follows [Equation 13].

는 회전기준 벡터,

는 y방향 단위 벡터)(only,

Is the rotational reference vector,

Is the y-direction unit vector)

According to FIG. 12, a rotation vector based on a camera at another predetermined position is shown, and the rotation vector is as follows [Equation 14].

는 회전기준 벡터,

는 y방향 단위 벡터)(only,

Is the rotational reference vector,

Is the y-direction unit vector)

13 and 14 are diagrams illustrating detection of pixel coordinates using an image in Experimental Example 2 according to the present invention.

Referring to FIG. 13, it is shown that pixel coordinates are detected based on the upper right corner of the landing point of the moving body 20 (ie, the reference body 10). In addition, according to FIG. 14, it is shown that pixel coordinates are detected based on the lower left corner of the landing point of the moving body 20 (ie, the reference body 10).

When the result values for pixel coordinates are output through this process, they are shown in [Table 4] and [Table 5] below.

Theoretical value Experimental value error Rotation Vector Rotation Vector Rotation Vector 0 0.0406 0.0406

0.7097 0.0752 0 -0.0271 -0.0271

Theoretical value Experimental value error Rotation Vector Rotation Vector Rotation Vector 0 0.0355 0.0355

-0.7425 0.07355 0 -0.0345 -0.0345

On the other hand, the method of determining the location information of the moving object 20 equipped with the photographing device 21 according to the present invention may be stored in a recording medium that can be read by a server computer. Such recording media include all types of recording media in which programs and data are stored so that they can be read by a computer system. Examples include ROM (Read Only Memory), RAM (Random Access Memory), CD (Compact Disk), DVD (Digital Video Disk)-ROM, magnetic tape, floppy disk, optical data storage device, etc. This includes implementations in the form of (for example, transmission over the Internet). Further, such a recording medium is distributed over a computer system connected by a network, so that computer-readable codes can be stored and executed in a distributed manner.

So far, the present invention has been looked at around its preferred embodiments. Those of ordinary skill in the art to which the present invention pertains will be able to understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the above description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

10: Reference body (or landing point)
11: first coordinate
12: second coordinate
13: 3rd coordinate
14: fourth coordinate
20: mobile (or drone)
21: photographing equipment (or camera)
22: processor

Claims (10)

Photographing a reference body using photographing equipment;
Detecting at least 4 coordinates from a reference body in the captured image; And
Determining the position information of the moving object based on the detected coordinates; Method for determining the position information of the moving object having a photographing device comprising a. The method of claim 1,
The reference body,
A method of determining the location information of a moving object equipped with photographing equipment that includes at least four preset coordinates. The method of claim 2,
The at least four preset coordinates,
When the moving object moves inside the communication radius of the reference object, it is transmitted to the communication unit of the moving object. The method of claim 1,
The at least four coordinates are set based on the world coordinate system,
The detecting step,
To convert the at least four coordinates into a camera coordinate system using an internal parameter, a method of determining the location information of a moving object equipped with a photographing equipment. The method of claim 1,
The detecting step,
Checking whether the at least four coordinates are correctly detected; And
If the at least four coordinates are not correctly detected as a result of the confirmation, the step of re-detecting the at least four coordinates; comprising, a method of determining the location information of the moving object equipped with the photographing equipment. The method of claim 1,
The detecting step,
The method of determining the location information of a moving object equipped with photographing equipment to detect the at least four coordinates in order to calculate the four unknowns of the following equation.

(only,

silver

Based on

Rotation matrix rotated by

silver

The angle rotated based on,

Is

X component of,

Is

Y component of,

Is

Z component of) A reference body including at least four coordinates;
A moving body including a photographing device for photographing the reference body; And
A system that detects at least four coordinates based on the captured image and determines location information of the moving object based on the detected coordinates. The method of claim 7,
The reference body comprises at least 4 edges,
The at least four coordinates are,
The system for determining the location information of the moving object having the photographing equipment that corresponds to the corner. The method of claim 7,
The reference body,
A system for determining the location information of a moving object equipped with photographing equipment that includes at least four preset coordinates. The method of claim 7,
The moving body,
A system for determining the location information of a moving object having a photographing device, comprising; a communication unit for receiving the at least four coordinate information when the moving object moves within a communication radius of the reference object.

Images