KR102045436B1

KR102045436B1 - Method and apparatus for recognizing curling sheet

Info

Publication number: KR102045436B1
Application number: KR1020180045596A
Authority: KR
Inventors: 설상훈; 홍명표; 윤성욱; 김가영
Original assignee: 고려대학교 산학협력단
Priority date: 2017-09-15
Filing date: 2018-04-19
Publication date: 2019-11-15
Also published as: KR20190033416A

Abstract

The present invention relates to a method and apparatus for recognizing a curling sheet, wherein the method for recognizing a curling sheet includes an internal parameter of a first camera, an internal parameter of a second camera, an external parameter of the first camera, and an external parameter of the second camera. Calculating a relational expression of the image through pre-calibration, acquiring a culling sheet image of the near area using a first camera and a culling sheet image of the far area using a second camera, and obtaining world coordinates from the culling sheet image of the near area. Extracting four known feature points, estimating a first pose value by estimating a three-dimensional pose of the first camera using the four feature points, and using the first pose value and the relational expression, Estimating a second pose value through three-dimensional pose estimation of a second camera, and using the first pose value and the second pose value W comprises the steps of calculating the coordinates of any point on the curling sheet image of a curling sheet images and the far region of the near zone. According to the present invention, the first pose value of the first camera is estimated using a pattern fixed to the curling sheet, and the first pose value is photographed through the second camera using a relationship between the external parameters of the two cameras. By calculating the coordinates of the curling sheet, the coordinates of the curling sheet can be calculated without a separate landmark, and the coordinates of the curling sheet photographed through the second camera photographing a long distance can be easily calculated.

Description

Recognition method and apparatus for curling sheet {METHOD AND APPARATUS FOR RECOGNIZING CURLING SHEET}

The present invention relates to a method and apparatus for recognizing a curling sheet for calculating image coordinates of an arbitrary point of a curling sheet photographed by a camera.

A curling race is a winter sport in which two teams decide to win or lose based on how accurately they place the stone on a rectangular ice rink called a curling sheet and place it on a given target. A curling match is a game in which both teams alternately throw a curling stone and put it in a house. The team closest to the tee in the middle of the house wins, and the number of curling stones closer than the opponent's curling stone closest to the tee. You get as many points. Competitors pitch the curling stone in the proper direction towards the house, brush with a broom and adjust the course and speed of the curling stone. The curling field is a sport that secures the path of stone by removing the pebble through brushing because there are many fine ice grains called pebble because it is scattered and frozen before the game starts.

Curling seats used in curling races are 4.2 meters wide and 42.07 meters invariable, with a length that is very long compared to the width. For this reason, according to the position of the camera, the farther distance from the camera among the images of the entire curling sheet is taken with a very small pixel value, which causes a problem in that the recognition rate is lowered. In addition, when photographing a curling sheet by installing a top-view camera, there is a problem in that the vertical shape of the rectangular curling sheet may be distorted in a curved shape, thereby photographing the correct coordinates of the curling stone. There is a difficult problem.

The present invention provides a method and apparatus for recognizing a curling sheet for determining an arbitrary coordinate value of a curling sheet by extracting four feature points from a pattern of a curling sheet image photographed at a distance and estimating a posture value using the feature points. There is a purpose.

In order to achieve the above object, a method of recognizing a curling sheet according to an embodiment of the present invention includes an internal parameter of a first camera, an internal parameter of a second camera, an external parameter of the first camera, and an external of the second camera. Calculating relations of parameters through pre-calibration, acquiring a culling sheet image of a near area using a first camera and a culling sheet image of a far area using a second camera, and world coordinates from the culling sheet image of the near area Extracting four feature points known to the user; estimating a first pose value through three-dimensional pose estimation of the first camera using the four feature points; using the first pose value and the relational expression, Estimating a second pose value by estimating a three-dimensional pose of the second camera, and using the first pose value and the second pose value Computing the curling sheet coordinates of any point of the curling sheet image of the near area and the curling sheet image of the far area.

The extracting of the four feature points may include extracting a house, two side lines, and a hogline pattern from the curling sheet image of the near area, and a first vanishing point, which is an intersection point between the two side lines, and two contacts of the house. A first contact pair, a line connecting the first contact pair, a second vanishing point, which is an intersection point of the hogline, and a second contact pair, which are two contacts of the house, may be extracted as four feature points.

In the extracting of the four feature points, the first contact pair corresponds to the intersection of the tee line of the curling sheet and the house, and the second contact pair corresponds to the intersection of the centerline of the curling sheet and the house. The feature may be used to assign world coordinates of the four feature points.

The extracting of the four feature points may include: in the curling sheet image of the near area, when the connection line of the first contact pair is parallel to the hogline and the second vanishing point does not exist, the parallel line of the hogline and the house Two contacts in contact with and extracted with the second pair of contacts may be used as the four feature points.

The extracting of the four feature points may include distinguishing a first ellipse and a second ellipse among ellipses in the house when the curling sheet image of the near area is not included. The specific point on the connection line and the connection line of the first vanishing point and the intersection point of the first ellipse and the second ellipse are respectively obtained, and the specific point where the third vanishing point and the fourth vanishing point, which are intersection points of the tangents of the respective intersection points, are coincident with each other. The two points that meet the second ellipse among the connection points of the first vanishing point and the specific point may be used as four feature points.

In the extracting of the four feature points, when the curling sheet image of the near region does not include the hogline, the ellipse in the house is divided into a first ellipse and a second ellipse in the curling sheet image of the near region. Obtaining the center of the house that is the intersection of the straight line connecting the center of the first ellipse and the center of the second ellipse and the connecting line of the first contact pair, and the straight line connecting the center of the house and the first vanishing point is Two points that meet the second ellipse may be used as four feature points by replacing the second contact pair.

Here, the first camera and the second camera is fixedly arranged in a fixed physical relationship including an angle and a distance, and the curling sheet image of the near area is the first camera is the house and the two sidelines It may be an image photographed to include.

An apparatus for recognizing a curling sheet according to another exemplary embodiment of the present invention includes a first camera set to photograph a curling sheet image of a short range region including two side lines and a house of a curling sheet, and a short region and a partial region of the curling sheet. A second camera set to capture an overlapping culling sheet image of a far-field region, an internal parameter of the first camera, an internal parameter of the second camera, an external parameter of the first camera, and an external parameter of the second camera Calculate the relational expression of the camera through pre-calibration, obtain the curling sheet image of the near area through the first camera and the curling sheet image of the far area through the second camera, and know the world coordinates from the curling sheet image of the near area. Extracted by the four feature points, and using the four feature points of the first camera. Estimating a first pose value through 3D pose estimation, using the first pose value and the relational expression, estimating a second pose value through 3D pose estimation of the second camera, and obtaining the first pose value. And a controller configured to calculate curling sheet coordinates for an arbitrary point of the curling sheet image of the near area and the curling sheet image of the far area by using the second posture value.

According to the present invention, it is possible to improve the accuracy by performing curling sheet recognition using a device equipped with two movable cameras, and to easily move the device when performing a game on another curling seat.

In addition, by estimating the attitude values of the two cameras using a pattern fixed to the curling sheet, and calculating the coordinates of the curling sheet in the image by using the same, the coordinates of the curling sheet can be calculated without a separate landmark.

In addition, the present invention can capture the near and far areas of the curling sheet using two cameras, respectively, and combine them into one curling sheet through image processing to increase the curling sheet recognition rate and minimize distortion due to the camera position. .

1 is a flowchart of a method of recognizing a curling sheet according to an exemplary embodiment of the present invention.
2 is a schematic configuration diagram of an apparatus for recognizing a curling sheet according to an embodiment of the present invention.
3 is an external view of an apparatus for recognizing a curling sheet according to an exemplary embodiment of the present invention.
4 to 5 are diagrams illustrating an installation position of an apparatus for recognizing a curling sheet and an area of an image photographed by two cameras, according to an exemplary embodiment.
FIG. 6 is a diagram for defining a coordinate system of an image photographed by an apparatus for recognizing a curling sheet according to an exemplary embodiment.
7 to 8 are diagrams for explaining four feature points according to the method of recognizing a curling sheet according to an embodiment of the present invention.
9 to 12 are diagrams for explaining a method of extracting four feature points according to a recognition method of a curling sheet according to an embodiment of the present invention.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be described in detail with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes any of a plurality of related description items or a combination of a plurality of related description items.

When a component is said to be "connected" or "connected" to another component, it may be directly connected to or connected to the other component, but it should be understood that there may be other components in between. something to do. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Throughout the specification and claims, when a part includes a certain component, it means that it may further include other components, except to exclude other components unless specifically stated otherwise.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart of a method of recognizing a curling sheet according to an exemplary embodiment of the present invention.

Referring to FIG. 1, a method of recognizing a curling sheet according to an exemplary embodiment of the present invention may include performing pre-calibration (S110), image acquisition (S120), posture value estimation (S130), and coordinate recognition of a curling sheet (S140). It can be performed as.

First, the pre-calibration S110 is a process of calculating a relationship between an internal parameter of the first camera, an internal parameter of the second camera, an external parameter of the first camera, and an external parameter of the second camera. Pre-calibration need not be performed in the curling arena, but may be performed in a separate laboratory or the like. In this case, the first camera and the second camera may be pre-calibrated in a state in which the first camera and the second camera are fixedly disposed apart from each other.

Internal parameters determine the internal factors such as the distance and angle between the lens and the image sensor. External parameters determine the position of a point in a three-dimensional world on a two-dimensional image. It consists of a matrix. Hereinafter, obtaining an external parameter is called three-dimensional attitude estimation, and specific external parameters are called first and second attitude values.

Specifically, the internal parameters of the first camera and the second camera are calculated by using the chess board and the MATLAB CALIBRATION TOOL, and the external parameters of the first camera are estimated through three-dimensional pose estimation using an arbitrary image that knows the three-dimensional world coordinates. And a relationship between the external parameters of the second camera can be calculated. When experimenting with the chessboard and the MATLAB CALIBRATION TOOL, the external parameters of the first camera and the external parameters of the second camera can be calculated, but can be changed when the device is moved or the camera's shooting position, angle, etc. are adjusted. Only the relational expression is calculated.

The pre-calibration process may include calculating a first relational expression of two-dimensional image coordinates and world coordinates of an arbitrary point photographed by the first camera and the second camera, an internal parameter of the first camera, and an internal parameter of the second camera, respectively. Calculating a second relational expression of two-dimensional image coordinates and world coordinates, and using the first relational expression and the second relational expression, except for the internal parameters of the first camera and the internal parameters of the second camera, It may be configured to calculate a third relation that is a relation between the external parameters of the two cameras. At this time, the arbitrary point represents a feature point that knows the two-dimensional image coordinates and the world coordinates, and for the experiment, a point that knows both the two-dimensional image coordinates and the world coordinates can be selected.

Hereinafter, the equation of each process will be described in detail.

Three-dimensional world coordinates can be represented by Equation 1 using two-dimensional image coordinates, internal parameters, and external parameters.

Here, X1 and X2 are two-dimensional image coordinates taken by the first and second cameras, W1 and W2 are three-dimensional world coordinates taken by the first and second cameras, and λ1 and λ2 are sizes of the first and second cameras. Coefficients, K1 and K2 are internal parameters of the first and second cameras, R1 and R2 are rotation matrices of the external parameters of the first and second cameras, and t1 and t2 are parallel shift matrices of the external parameters of the first and second cameras. Indicates.

The two-dimensional image coordinates can be converted to camera coordinates using Equation 2. In addition, the internal parameters may be removed and represented as shown in [Equation 3].

Here, C1 and C2 represent camera coordinates of the first and second cameras.

If both the internal and external parameters of the first and second cameras are known, the two-dimensional image coordinates may be converted into coordinates of the two-dimensional curling sheet through the three-dimensional camera coordinate system. That is, after acquiring an image of the curling sheet, the coordinates of an arbitrary point on the curling sheet may be recognized by moving to the left end of the two-dimensional curling sheet.

As described above, by using the internal and external parameters of the first and second cameras, the two-dimensional image coordinates, the three-dimensional world coordinates, and the three-dimensional camera coordinates, between the external parameters of the first camera and the external parameters of the second camera, The relation can be derived as shown in [Equation 4]. In this case, the relational expression between the rotation matrix of the external parameter and the relational expression between the parallel movement matrix may be derived.

Since the relationship between the external parameters can be applied equally even if the shooting environment is changed, the relationship obtained through the pre-calibration can be applied even after the device is installed in the curling stadium.

After the pre-calibration is performed, the photographing position and the photographing angle of the camera may be adjusted to include the area of the curling sheet to be photographed after the curling sheet recognition apparatus is installed in the curling stadium.

Next, the curling sheet image of the near area may be acquired through the first camera and the curling sheet image of the far area may be obtained through the second camera (S120). The process of acquiring the images may be performed simultaneously or sequentially according to the control signal. The culling sheet image of the near area photographed by the first camera and the culling sheet image of the far area photographed by the second camera may be photographed to include a hogline region proximate to the first camera and the second camera. . In this case, the first camera may include a photographing area to necessarily include a house pattern, and may or may not include the hogline itself. In addition, according to how the photographing area of the first camera is set in the second camera, the photographing area may be set so that some areas overlap.

Next, the posture value estimation may be extracted as four feature points that know the world coordinates from the curling sheet image of the near area (S130).

Specifically, the four feature points are points of knowing two-dimensional image coordinates and three-dimensional world coordinates, and can be extracted by recognizing a specific pattern (eg, house, hogline, side line, etc.) of the curling sheet. When the near-field image includes the hogline, a first vanishing point, which is an intersection obtained by extending two sidelines of the image, and a first contact point of the house, and a second vanishing point, which is an intersection point of the connection line and the hogline of the first contact pair The second contact pair, which is the two contacts of the and house, can be extracted as four feature points. The first and second contact pairs may be utilized as world coordinates by using the intersection of the house and the tee line and the intersection of the house and the center line in a curling sheet manufactured to a standardized size. In addition, when the second line disappears in parallel with the connecting line of the first contact pair when the hogrine included in the near image is not present, four feature points are extracted by extracting the two contact points contacting the parallel line and the house as the second contact pair. Can be used as The actual house is circular, but the house included in the image is taken as an oval. In addition, when the near-field image does not include the hogline, the ellipse in the house is divided into a first ellipse and a second ellipse, and the connection line of the specific point and the first vanishing point, the first ellipse, and the first vane point 2 After finding the intersections of the ellipses, obtain a specific point where the third vanishing point and the fourth vanishing point, which are the intersection points of the tangents of each intersection point, match the second ellipse between the first vanishing point and the connection line of the specific point. It can be used as four feature points instead of a pair of contacts. A detailed method of calculating the four feature points will be described in detail with reference to FIGS. 9 to 11.

The first posture value is estimated through the posture estimation, and the second posture value is estimated through the three-dimensional posture estimation of the second camera using the relational expression between the first posture value, the external parameter of the first camera, and the external parameter of the second camera. Can be estimated. The near-field curling sheet image photographed by the first camera is easy to extract the feature point because the pattern of the house is clearly captured, whereas the far-field curling sheet image photographed by the second camera is unclear the pattern of the house, so the second camera of the second camera Posture value estimation can be difficult. Therefore, the second pose value can be estimated using the first pose value of the first camera and the relational expression.

Lastly, in the coordinate recognition of the curling sheet (S140), coordinates of an arbitrary point of the curling sheet image of the near area and the curling sheet image of the far area may be calculated using the first posture value and the second posture value.

Specifically, the 2D image coordinates photographed by the second camera with respect to any point on the 3D world coordinates may be calculated by Equation 5 below.

2 is a schematic configuration diagram of an apparatus for recognizing a curling sheet according to an embodiment of the present invention, and FIG. 3 is an external view of an apparatus for recognizing a curling sheet according to an embodiment of the present invention.

2, the apparatus 100 for recognizing a curling sheet according to an exemplary embodiment may include a first camera 110, a second camera 120, and a controller 130.

The first camera 110 may be set to capture two sidelines of the curling sheet and a curling sheet image of a near area including the first hogline from the first backline. In this case, the first camera 110 may be set so that the photographing area is set to include only a short-range house, and the first backline, the first hogline, or the like may be included in the short-range area or may be set to be excluded.

The second camera 120 may be set so that the photographing area of the first camera 110 overlaps with a part of the photographing area and includes a far-field area. For example, the second camera 120 may be set to photograph a curling sheet image of a far region including a side line of the curling sheet and a first backline to a second backline. This setting is only an example, and the first camera 110 is set to photograph a short range including a house, and the second camera 120 is configured to capture a short range and a range of the overlapping short range and include a long range. Is set.

The controller 130 precalibrates a relational expression between an internal parameter of the first camera 110, an internal parameter of the second camera 120, an external parameter of the first camera 110, and an external parameter of the second camera 120. It can be calculated through In addition, the controller 130 obtains a culling sheet image of the near area through the first camera 110 and the culling sheet image of the far area through the second camera 120, and obtains world coordinates from the culling sheet image of the near area. The camera coordinates can be extracted to four known feature points. The controller 130 estimates the first pose value through the three-dimensional pose estimation of the first camera 110 by using the four feature points, and uses the first pose value and the relational expression to calculate the three poses of the second camera 120. The second attitude value may be estimated through the dimensional attitude estimation. Using the first posture value and the second posture value, coordinates of an arbitrary point of the curling sheet image of the near region and the curling sheet image of the far region may be calculated using Equation 5.

Referring to FIG. 3, the apparatus 100 for recognizing a curling sheet according to an exemplary embodiment may include a first camera 110 and a second camera 120 spaced apart by a fixed distance.

In the drawing, although the angle between the first camera 110 and the second camera 120 is fixed, each camera may be rotated in up, down, left, and right directions to change and set the shooting angle.

4 to 10 illustrate a first camera photographing an area including two sidelines, a first backline, a house, and a first hogline, and a second camera photographing two sidelines. 2 An example of photographing an area including a backline will be described. 10 illustrates an example in which a photographing area is set such that the first camera does not include the first hogline. In each of the following drawings, the first and second cameras may be photographed only by one embodiment. As described above, the first camera may be configured to include at least a house, and the second camera may be configured to include the first camera. What is necessary is just to set a photography area | region so that a some area | region may overlap with a photography area | region.

4 to 5 are diagrams illustrating an installation position of an apparatus for recognizing a curling sheet and an area of an image photographed by two cameras, according to an exemplary embodiment.

4 to 5, an apparatus for recognizing a curling sheet according to an exemplary embodiment may be disposed behind a first backline of the curling sheet.

The first camera 110 of the recognizing apparatus of the curling sheet may capture a short range region from the first backline to the first hogline, which is surrounded by both sidelines. In addition, the second camera 120 may photograph a far-field area from the first hogline surrounded by both sidelines to the second backline.

FIG. 6 is a diagram for defining a coordinate system of an image photographed by an apparatus for recognizing a curling sheet according to an exemplary embodiment.

Referring to FIG. 6, an image photographed by the apparatus for recognizing a curling sheet according to an embodiment of the present invention may photograph a curling sheet and a curling stone of a 3D world coordinate system, and convert the curling sheet into a 2D image coordinate system. The camera coordinate system can be used in this process.

Specifically, when the three-dimensional world is photographed with a camera, it is changed into a two-dimensional image. At this time, as shown in [Equation 1], the position of a point that forms a point of the three-dimensional world on the two-dimensional image by using an internal parameter and an external parameter. Can be converted to At this time, in order to obtain the physical relationship between the two cameras, that is, the relationship between the external parameters, it is necessary to move the world coordinate system to the camera coordinate system by removing the internal parameter.

7 to 8 are diagrams for explaining four feature points according to the method of recognizing a curling sheet according to an embodiment of the present invention.

7 to 8, the apparatus for recognizing a curling sheet according to an exemplary embodiment of the present invention extracts a house, two side lines, and a hogline pattern from a curling sheet image of a near area, and a vanishing point extending the side lines. The first contact pair, which is two contacts of the tee line of the and house, the line parallel to the hogline, and the two contacts of the house tee line may be extracted as four feature points. The first posture value may be estimated through three-dimensional posture estimation of the first camera using four feature points selected from the near image.

In detail, the curling sheet image of the near area photographed using the first camera includes both sidelines of the elliptical house and the width of which is photographed differently according to the distance. Two-dimensional image coordinates can be extracted from the photographed image and converted into three-dimensional world coordinates as shown in FIG. 8. In other words, the elliptical house may appear circular in the real world, and both sidelines having a variable width may be represented as sidelines having a constant width.

These four feature points represent two points (first and second points) where the house and two sidelines parallel to the line and two points (third and fourth points) where the house and centerline meet (Fig. 3). 8).

9 to 12 are diagrams for explaining a method of extracting four feature points according to a recognition method of a curling sheet according to an embodiment of the present invention.

A pair where the first line, the second point, the center line l3 and the house meet, which is a pair of contact points between the tee lines l2 and l4 parallel to the two side lines l1 and l5 of the curling sheet. The third point and the fourth point, which are points of, become four feature points. The above four feature points are extracted from the image of the curling sheet by using the first and second vanishing points, using the first vanishing point and the hogline, and using the first, third and fourth vanishing points. There is this.

Referring to FIG. 9, four feature points may be extracted using a first vanishing point and a second vanishing point in a curling sheet recognition method according to an exemplary embodiment of the present invention.

In detail, a house, two side lines, and a hogline pattern may be extracted from a curling sheet image photographing a near area. At this time, the house is composed of an elliptical first ellipse, a second ellipse, the side line may be photographed to meet at the first vanishing point.

A first vanishing point, which is an intersection point of two side lines, and a first contact pair (first point and second point), which are two contacts of a house, may be extracted. Next, a second vanishing point, which is an intersection point of the line connecting the first pair of contacts (first point and second point) and the hogline, is obtained, and a second pair of contacts (third point, Fourth point) can be extracted. The first contact pair (first and second points) extracted using the first vanishing point and the second contact pair (third and fourth points) extracted using the second vanishing point can be extracted as four feature points. have.

Referring to FIG. 10, in the method of recognizing a curling sheet according to an exemplary embodiment of the present disclosure, when the connecting line and the arc of the first contact pair are parallel to each other and the second vanishing point does not exist, the parallel line between the first vanishing point and the arc is removed. Four feature points can be extracted.

A first vanishing point, which is an intersection point of two side lines, and a first contact pair (first point and second point), which are two contacts of a house, may be extracted. Next, obtain a second vanishing point, which is an intersection point of the line with the first contact pair (first point and the second point), and the second vanishing point. If the second vanishing point does not exist, the two parallel lines of the arc line and the house contact each other. The second contact pair (third point and fourth point) that is the contact point can be extracted. The first contact pair (first and second points) extracted using the first vanishing point and the second contact pair (third and fourth points) extracted using the parallel line of the hogline can be extracted as four feature points. Can be.

Referring to FIG. 11, in the method of recognizing a curling sheet according to an embodiment of the present invention, when the hograin is not included in the near area, four feature points are extracted using the first vanishing point, the third vanishing point, and the fourth vanishing point. can do.

In detail, a house and two side line patterns may be extracted from a curling sheet image photographing a near area. At this time, the house is composed of an elliptical first ellipse, a second ellipse, the side line may be photographed to meet at the first vanishing point.

A first vanishing point, which is an intersection point of two side lines, and a first contact pair (first point and second point), which are two contacts of a house, may be extracted. Next, the house is divided into a first ellipse and a second ellipse, and a connection point between the specific point and the first vanishing point on the connection line of the first pair of contacts (first point and the second point) is drawn. Find the intersections (a, a ', b, b') where the connecting line of the vanishing point meets the first ellipse and the second ellipse, respectively. After finding the third vanishing point, which is the intersection of the tangents of the intersections (a, a ') with the first ellipse, and the fourth vanishing point, which is the intersection of the tangents of the intersections (b, b') with the second ellipse, the third vanishing point Check if and the fourth vanishing point match. In this case, two points that meet the second ellipse among the connection lines of the specific points that correspond to the third vanishing point and the fourth vanishing point may be extracted as the second contact pair. According to the above method, the third vanishing point and the fourth vanishing point made at the intersection point of the connecting line passing through the center line among the connecting lines of the first contact pair and the first and second ellipses meet, and consequently curling. In the sheet image, the point passing through the centerline can be found. In this way, four feature points can be extracted.

Referring to FIG. 12, in the method of recognizing a curling sheet according to an embodiment of the present invention, when the hograin is not included in the near area, the four vanishing points and the centers of the first and second ellipses of the house are four. Feature points can be extracted.

A first vanishing point, which is an intersection point of two side lines, and a first contact pair (first point and second point), which are two contacts of a house, may be extracted. Next, when the house is divided into a first ellipse and a second ellipse, and the center of the first ellipse is defined as the first center and the center of the second ellipse as the second center, the connection line between the first center and the second center and the above The point where the connection line of the first contact pair meets becomes the center of the house. In this way, after extracting the center of the house, the point that meets the second ellipse (that is, the house) of the line connecting the first vanishing point and the center of the house is used as the second contact pair (the third point and the fourth point). Can be. Using this method, the calculation amount for finding the center of the house is reduced than in FIG. 11, and four feature points can be extracted more efficiently.

According to the present invention, since the first posture value of the first camera photographing the near-area can be derived using the fixed pattern of the curling sheet, it is not necessary to add a separate landmark. Also, by estimating the second posture value of the second camera photographing the far-area using the relation between the first posture value of the first camera and the external parameters of the cameras, the coordinates of the far-field image of the second camera, which are difficult to extract feature points, are estimated. It can be calculated easily.

The above description is merely illustrative of the technical idea of the present invention, and those skilled in the art to which the present invention pertains may make various modifications and changes without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to describe the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

100: recognition device for curling sheet 110: first camera
120: second camera 130: control unit

Claims

Calculating a relational expression between an internal parameter of a first camera, an internal parameter of a second camera, an external parameter of the first camera, and an external parameter of the second camera through pre-calibration;
Acquiring a culling sheet image of the near area through a first camera and a culling sheet image of the far area through a second camera;
Extracting a house, two side lines, and a hogline pattern from the curling sheet image of the near area, and extracting four feature points that know world coordinates;
Estimating a first posture value using the three-dimensional posture estimation of the first camera using the four feature points;
Estimating a second posture value through three-dimensional posture estimation of the second camera by using the first posture value and the relational expression; And
Calculating a culling sheet coordinate of an arbitrary point of the culling sheet image of the near region and the culling sheet image of the far region using the first posture value and the second posture value;
Recognition method of a curling sheet comprising a.

The method of claim 1,
Extracting the four feature points,
The first vanishing point, which is the intersection of the two side lines, and the first contact pair, which is two contacts of the house, the second vanishing point, which is the intersection of the line connecting the first contact pair, and the hogline, and the two contacts of the house. A method for recognizing a curling sheet, wherein the second pair of contacts are extracted as four feature points.

The method of claim 2,
Extracting the four feature points,
The first contact pair corresponds to the intersection of the tee line of the curling sheet and the house, and the second contact pair corresponds to the intersection of the center line of the curling sheet and the house. Recognition method of curling sheet which allocates coordinates.

The method of claim 2,
Extracting the four feature points,
In the curling sheet image of the near area, when the connection line of the first contact pair is parallel to the hogline and the second vanishing point does not exist, the second contact points are formed in contact with the parallel line of the hogline and the house. A method for recognizing a curling sheet, which is extracted as a pair and used as the four feature points.

The method of claim 2,
Extracting the four feature points,
When the hograin is not included in the curling sheet image of the near area, an ellipse in the house is divided into a first ellipse and a second ellipse, and a specific point on the connection line of the first contact pair and the first vanishing point Finding the intersection of the connecting line, the first ellipse and the second ellipse, respectively, and obtaining the specific point coinciding with the third vanishing point and the fourth vanishing point, which are the intersection points of the tangents of the respective intersection points, A method for recognizing a curling sheet, wherein two points that meet the second ellipse of the connecting line are replaced by a second pair of contacts and used as four feature points.

The method of claim 2,
Extracting the four feature points,
When the hograin is not included in the curling sheet image of the near area, an ellipse in the house is divided into a first ellipse and a second ellipse in the curling sheet image of the near area, and the center of the first ellipse and the second ellipse. Obtaining the center of the house that is the intersection of the straight line connecting the center of the ellipse and the connecting line of the first contact pair, and the second point where the straight line connecting the center of the house and the first vanishing point meets the second ellipse A method for recognizing a curling sheet, which is replaced by a pair of contacts and used as four feature points.

The method of claim 1,
The first camera and the second camera are fixedly arranged in a fixed physical relationship including an angle and a distance,
The curling sheet image of the near area is an image photographed so that the first camera includes the house and the two sidelines.

A first camera set to capture a curling sheet image of a short range region including two side lines and a house of a curling sheet;
A second camera set to capture a culling sheet image of a distant area in which a near area and a partial area of the curling sheet overlap; And
The relational expression between the internal parameter of the first camera, the internal parameter of the second camera, the external parameter of the first camera and the external parameter of the second camera is calculated through pre-calibration, and the short range area is obtained through the first camera. Four feature points are obtained by obtaining a curling sheet image of a far region through a curling sheet image and a second camera, extracting a house, two sidelines, and a hogline pattern from the curling sheet image of the near region. The first posture value is estimated through the three-dimensional posture estimation of the first camera using the four feature points, and the third posture of the second camera is obtained using the first posture value and the relational expression. Estimating a second posture value through estimation, and using the first posture value and the second posture value, a curling sheet of the near area. A controller for calculating the coordinates of the curling sheet and the image of any point on the curling sheet remote area;
Recognizing apparatus of the curling sheet comprising a.