JP5371015B2 - Cross mark detection apparatus and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform stable detection of a cross mark by detecting change of luminance of the surroundings of each pixel without relying on a template. <P>SOLUTION: This cross mark detection device consists of: a cross mark candidate image creation means for determining the center of cross marks for each point in an image to be considered as a cross mark candidate point image; a cross mark candidate image line thickening means for thickening lines of the cross mark candidate image, performing area division to output an area image obtained by giving the same area number to cross mark candidate points in the vicinity; a cross mark center coordinate calculation means for making the cross mark candidate point into an intersection of two straight lines using luminance information in the vicinity to calculate cross mark center coordinates; and a representative point adoption means for considering a point having the maximum score among points having the same area number as a representative point of the area to be determined as the center of the cross mark. <P>COPYRIGHT: (C)2012,JPO&amp;INPIT

Description

  The present invention relates to a cross mark detection device and method for detecting a cross mark from a captured luminance image, and a program

  When it is desired to know a position in an image when a specific point of a three-dimensional object is imaged by a camera, a mark that is less affected by the imaging environment and whose coordinates can be accurately calculated by image processing may be captured. For example, when calculating the parameters required for projective transformation from 3D coordinates to 2D coordinates of a point in the camera's own space, a mark is placed on a point where the 3D coordinates are already known, Calculate parameters based on dimensional coordinates. It is also used when measuring unknown positions and movements in space with projective transformation known.

  One of the marks used in such a case is a cross mark shown in FIG. This is a pattern in which white and black appear alternately on the plane, can be used regardless of the brightness of the background, and the center point of the mark can be calculated as the intersection of two lines, so the subpixel accuracy It is possible to obtain coordinate values with It is also used in Patent Document 1 or Patent Document 2. Some programs for detecting such a cross mark in an image are based on template matching. In this method, a luminance difference between a two-dimensional pattern (template) created in advance and a rectangular luminance pattern (window) of the same size in the image is calculated, and if it is within a threshold value, a cross mark is obtained. There are two problems with the template matching method.

  The first problem is the problem of overlooking existing cross marks. If the place where the cross mark is placed is bright or dark depending on the shooting conditions or the like, it may be determined that the mark is not a mark due to a luminance difference with the template. In addition, as shown in FIG. 2, even when the template is not facing the image plane of the camera and is inclined or rotated, it is determined that the template is not a cross mark due to a difference between the image with the template. There is a case.

  The second problem is when a place that is not a cross mark is detected as a cross mark. Typically, the linear figure shown in FIG. 3 also has a certain degree of similarity with the template, and may be detected as a cross mark depending on threshold conditions.

JP 2005-28468 A JP 2009-279663 A

  The above two problems may be solved by setting a threshold value, but usually give conflicting results. That is, if the threshold value is increased and the luminance difference from the template is not allowed, oversight increases. Conversely, if the threshold value is decreased, the number of cases where a cross mark is erroneously determined increases.

  An object of the present invention is to solve such a problem and to detect a stable cross mark by detecting a change in luminance around each pixel without depending on a template.

  The cross mark detection apparatus of the present invention detects a cross mark from a captured luminance image. This cross mark detection apparatus receives a luminance image having a cross mark to be detected, determines whether each point in the image is the center of the cross mark, and determines the result as a cross mark candidate point image. Cross mark candidate image creating means, and cross mark candidate image thickening means for thickening the cross mark candidate image and outputting a region image obtained by dividing the region and assigning the same region number to a nearby cross mark candidate point; Same as the cross mark center coordinate calculation means for determining the two straight lines using the luminance information in the vicinity of the cross mark candidate point in the cross mark candidate image and calculating the cross mark center coordinate as the intersection. The point with the highest score among the points with the region number of is selected as the representative point of the region, and the representative point of this region is the center of the cross mark. A representative point employed means determines that consists.

  Further, the cross mark detection method of the present invention performs determination as the center of the cross mark for all pixels in the luminance image, sets the result as a cross mark candidate point image, thickens the cross mark candidate point image, Create an area image, perform area division to assign the same area number to the area image where the cross mark candidate point is adjacent, and calculate the luminance pattern of the neighboring points centering on each cross mark candidate point Use this to calculate the center coordinates of the cross mark, and at the same time calculate the score that is the strength of the cross mark, and the cross mark having the highest score among the candidate points in the cross mark candidate point image having the same area number in the area image The candidate point is adopted as a representative point of the region, and the cross mark two-dimensional coordinate values corresponding to the number of region numbers are obtained.

  Further, the cross mark detection program of the present invention performs the determination as the center of the cross mark for all pixels in the luminance image and sets the result as a cross mark candidate point image, and the cross mark candidate point image is thickened. To create a region image, to perform region division for assigning the same region number to the region image adjacent to the cross mark candidate point, and to center each cross mark candidate point Calculating the center coordinates of the cross mark using the luminance pattern of the neighboring points to be calculated, and simultaneously calculating the score that is the strength of the cross mark, and the candidate points in the cross mark candidate point image having the same area number in the area image The cross mark candidate point with the highest score is adopted as the representative point of the area, and the cross mark two-dimensional coordinate value is equal to the number of area numbers. A step of obtaining, to execute the respective steps in a computer comprising a.

  The determination as the center of the cross mark is performed by creating a continuous ring-shaped mask around the target point and using luminance information on the mask. The cross mark center coordinate calculation is performed by detecting a point having a large slope of the luminance distribution in the vicinity of a straight line connecting the point of the original image corresponding to the boundary point and the point of interest, and two parallel lines between the opposing boundary points. And a straight line located at the center thereof is used as a representative straight line, and a two-dimensional coordinate value is calculated as an intersection of two intersecting representative straight lines.

  According to the present invention, stable cross marks can be detected by detecting a change in luminance around each pixel without relying on a template.

It is a figure which illustrates a cross mark. It is a figure explaining distortion of the cross mark which a template does not detect. It is a figure which shows the example of an image which a template detects accidentally. It is a figure which illustrates the cross mark detection apparatus comprised based on this invention. It is a figure which illustrates an operation | movement algorithm. It is a figure explaining a ring mask and the arrangement | sequence R. FIG. It is a figure explaining the determination by a luminance difference. It is a figure explaining the example of the impossibility in the area number determination. It is a figure explaining the symmetry condition of a boundary. It is a figure explaining the conditions which are too narrow. It is a figure explaining template determination. It is a figure explaining the example of area division. It is a figure explaining extraction of an edge point and a segment. It is a figure explaining the example of a segment with a shift | offset | difference. It is a figure explaining inclination calculation of a shifted segment.

  Hereinafter, the present invention will be described based on examples. FIG. 4 is a schematic diagram showing the overall configuration of a cross mark detection apparatus configured according to the present invention. A luminance image I1 having a cross mark to be detected is photographed by the camera and input to the cross mark candidate image creating means to calculate the center position of the cross mark with sub-pixel accuracy from the image. The sub-pixel accuracy (accuracy smaller than one pixel) here means that the center coordinate of the cross mark is calculated as a coordinate value after the decimal point. Therefore, it is determined whether or not each point in the image satisfies the condition for being the center of the cross mark, and the result is set as a cross mark candidate point image I2. The cross mark candidate image thickening means thickens the cross mark candidate image I2, outputs a region image I3 obtained by dividing the region and assigning the same region number to the cross mark candidate points in the vicinity. The cross mark center coordinate calculation means determines two straight lines using luminance information in the vicinity of the cross mark candidate point in the image I2, and calculates the cross mark center coordinate as the intersection with sub-pixel accuracy. The representative point adopting means adopts a point having the maximum score among the points having the same area number in the image I3 as the representative point of the area. It is determined that the representative point of this area is the center of the cross mark.

  Next, the operation of the cross mark detection apparatus shown in FIG. 4 will be further described with reference to FIG. 5 showing an operation algorithm. The example program is roughly divided into four steps (S1 to S4).

  In the first step S1, all pixels in the image I1 are determined as the center of the cross mark, and the result is set as a cross mark candidate point image I2. A combination of the following conditions is used for the cross mark determination performed in this step. In addition, other conditions can be combined as necessary. At this time, as shown in FIG. 6, a continuous ring-shaped mask is created around the target point P for determination, and a one-dimensional array R in which luminance information on the mask is numbered in the circumferential direction is used. . The ring-shaped mask is not limited to the illustrated circle, and may be, for example, a square or a polygon as long as the isotropic surrounding luminance change around a certain point can be seen. Array R is treated as a sequence of head and tail in the ring buffer. In the following description, the length of the array R is m. The reason why the array R is used is to perform processing independent of the posture of the cross mark in the image.

(1) As shown in FIG. 7, when the difference between the maximum luminance and the minimum luminance in the array R is smaller than the threshold value t0, it is impossible.

(2) When the array R is binarized using the threshold value t1 considering the luminance distribution in the array R as shown in FIG. 8, and there are not two high-luminance areas and two low-luminance areas alternately. Is not allowed. That is, if there are not four boundaries across the luminance threshold based on the luminance distribution of the points on the mask, the cross mark candidate is not used.

(3) As shown in FIG. 9, when the distance between the high-brightness region and the even-numbered boundaries A2 and A4 between the high-brightness region and the low-brightness region divided by the threshold t1 does not have a value close to m / 2 Impossible. That is, if the boundary point is not in a symmetric position, it is not a cross mark candidate.

(4) Similarly to the above, it is not possible when the distance between the odd-numbered boundaries A1 and A3 does not have a value close to m / 2.

(5) As shown in FIG. 10, when the number of pixels between adjacent boundaries {Ai, Ai + 1} delimited by the threshold t1 is smaller than the threshold t2, it is not allowed. That is, if the distance between the boundary points is not wide enough, the cross mark candidate is not used.

(6) As shown in FIG. 11, the point corresponding to {A1... A4} in the original image is {B1... B4}, and three template images T1, T2, and T3 are created. The size w1 of the template image window set in the vicinity of the target point P for template determination is set to a size including the ring mask. The template image T1 has a straight line B1-B3 and a straight line B2-B4 as boundary lines. This image is a template corresponding to a cross mark. The template image T2 has a straight line B1-B2 and a straight line B3-B4 as boundary lines. The template image T3 has a straight line B2-B3 and a straight line B4-B1 as boundary lines. Image T2 and image T3 are templates corresponding to a linear pattern. The brightness other than the border line of each template is created so that the brightness state of the area [Ai, Ai + 1] in the array R matches the brightness state of the area [Bi, Bi + 1] in the template. A window having a size w1 around the point P of the image is compared with three templates, and this is not possible if the similarity to T1 is not the closest. In other words, two types of templates with cross mark-like luminance distributions and band-like templates are created by passing through the points of the original image corresponding to the boundary points, and the similarity between the template with the cross mark-like luminance distributions must be the highest. If it is not a cross mark candidate.

  In the second step S2 shown in FIG. 5, as shown in FIG. 12, the cross mark candidate point image I2 is thickened to create a region image I3. The cross mark candidate image I2 is illustrated in the upper left of FIG. The thickening is an operation in which a point away from the non-cross mark candidate point in the image I2 by a pixel d given a distance from the existing cross mark candidate point is also set as a cross mark candidate point. For example, if the cross mark candidate image is a binary image in which the cross mark candidate point is 0 and the non-cross mark candidate point is -1, an operation of setting all -1 points adjacent to a point having a value of 0 to 0 If this is done, thickening of one pixel becomes possible, and d pixels can be thickened by repeating this d times. In the upper right of FIG. 12, a thick line result image is illustrated. Region division is performed in which the same region number is assigned to the region image I3 that has been bolded and adjacent to the cross mark candidate points. As shown in the lower left of FIG. 12, the same region number is assigned to a plurality of thick cross mark candidate points in each of region 1 and region 2. In other words, individual area numbers are assigned to each successive area in the area division process. This processing is necessary because the cross mark candidate points corresponding to the same cross mark are not necessarily continuous regions.

  In the third step S3, the center coordinates of the cross mark are calculated using the luminance pattern of the neighboring points centering on each cross mark candidate point P, and at the same time, the strength (score) of the cross mark is calculated. This cross mark center coordinate calculation detects a point where the gradient of the luminance distribution is large in the vicinity of the straight line connecting the point of the original image corresponding to the boundary point and the point of interest, and creates two parallel lines between the opposing boundary points. The straight line located at the center is used as a representative line, and a two-dimensional coordinate value is calculated as the intersection of two intersecting representative lines.

  For this purpose, first, a neighborhood window of size w2 is set around the point of interest P as shown in FIG. The window size w2 is set by the user so as not to be larger than the cross mark in the image and not too small. If it is too large, an error will occur due to the influence of the location that is not a cross mark, and if it is too small and the number of points used to calculate a straight line decreases, an error will still occur. Next, a segment window is created in the vicinity of the line connecting the point of interest P and the boundary point Bi on the ring mask in the image, and a plurality of edge points are extracted as points having a large luminance change. The edge point is obtained by a method such as differentiation. A set of these points is defined as a segment. A score value reflecting the magnitude of the luminance change is calculated for each extracted edge point. As the score value, for example, the magnitude of the differential value of luminance is used. The sum of the score values calculated for the edge points on a certain segment is taken as the score value for that segment.

  Next, a straight line L13 is calculated for the edge points on the odd-numbered segments s1 and s3, and a straight line L24 is calculated for the edge points on the even-numbered segments s2 and s4. Note that s1 is a segment in the B1 direction as seen from the point P, and the other segments are described in the same manner. The odd-numbered segments or the even-numbered segments are originally the same straight line, but in an actually photographed image, a bright part or a dark part may expand and a shift as shown in FIG. 14 may occur. . Taking these possibilities into consideration, when calculating a straight line, each segment is a parallel straight line slightly shifted in the normal direction, and the central straight line is calculated assuming that the amount of shift is equal.

Hereinafter, a method of calculating the coefficient of the straight line equation will be described using the straight line L13 as an example. If the point of interest P is the center of the cross mark, S1 and S3 are line segments having the same inclination. Here, the linear equation of S1 and S3 is described as follows.
S1: α ₁₃ X + β ₁₃ Y + γ ₁ = 0
S3: α ₁₃ X + β ₁₃ Y + γ ₃ = 0
However, α ₁₃ ² + β ₁₃ ² = 1.

Calculation of α ₁₃ , β ₁₃ , γ ₁ , and γ ₃ is performed as follows. As shown in FIG. 15, the center of gravity of the point group {C ₁ i} constituting S1 is G ₁ , and the center of gravity of the point {C ₃ i} constituting S3 is G ₃ . At this time, C ₁ i−G ₁
And C ₃ i−G ₃ vectors, and the eigenvalues and eigenvectors of their covariance matrices. α ₁₃ and β ₁₃ are eigenvectors corresponding to the minimum eigenvalue. γ ₁ and γ ₃ are calculated using α ₁₃ and β ₁₃ as follows.
γ ₁ = −α ₁₃ G ₁ x−β ₁₃ G ₁ y
γ ₃ = −α ₁₃ G ₃ x−β ₁₃ G ₃ y

However, G ₁ x, G ₁ y X and Y components of the center of gravity _{G 1, G 3 x, G} 3 y indicates the X and Y components of G _3. The difference between γ ₁ and γ ₃ | γ ₁ −γ ₃ | indicates the distance between the parallel lines of the two segments. If this value is large, it is assumed that the fitting of the line fails. If it is determined that the straight line fitting is successful, the central straight line of the two segments is set as the straight line in the B1B3 direction. The formula is expressed as follows.
Line B1B3: α ₁₃ X + β ₁₃ Y + (γ ₁ + γ ₃ ) / 2 = 0

  The straight line B2B4 is calculated in the same manner as the straight line B1B3. Thereafter, the intersection of the straight line B1B3 and the straight line B2B4 is calculated and set as the two-dimensional coordinate value of the cross mark. As the cross mark score, the sum of the scores of all the segments is used.

In the fourth step S4, the cross mark candidate point having the maximum score among the candidate points in the cross mark candidate point image I2 having the same area number in the area image I3 is adopted as the representative point of the area. As a result, as many cross mark two-dimensional coordinate values as the number of region numbers are obtained.

Claims (11)

In a cross mark detection device that detects a cross mark from a captured luminance image,
A luminance image having a cross mark to be detected is input, and it is determined whether or not each point in the image is the center of the cross mark, and the result is used as a cross mark candidate point image creation means. When,
Cross mark candidate image thickening means for thickening the cross mark candidate image and outputting a region image obtained by dividing the region and assigning the same region number to a cross mark candidate point in the vicinity;
Cross mark center coordinate calculating means for determining a two-line using luminance information of the cross mark candidate point in the cross mark candidate image and calculating a cross mark center coordinate as the intersection point;
A point having the maximum score among points having the same region number in the region image is adopted as a representative point of the region, and representative point adopting means for determining that the representative point of this region is the center of the cross mark; ,
A cross mark detection device comprising: In a cross mark detection method for detecting a cross mark from a captured luminance image,
Perform determination as the center of the cross mark for all pixels in the luminance image, the result is a cross mark candidate point image,
The cross mark candidate point image is thickened to create an area image,
Perform area division to assign the same area number to the area image adjacent to the cross mark candidate points,
Calculate the center coordinate of the cross mark using the brightness pattern of the neighboring points centered on each cross mark candidate point, and simultaneously calculate the score that is the strength of the cross mark,
The cross mark candidate point having the largest score among the candidate points in the cross mark candidate point image having the same area number in the area image is adopted as the representative point of the area, and the cross mark two-dimensional coordinate values corresponding to the number of the area numbers are used. Get the
A cross mark detection method comprising:   The cross mark detection method according to claim 2, wherein the determination as the center of the cross mark is performed by creating a continuous ring-shaped mask around a target point and using luminance information on the mask.   The cross mark detection method according to claim 3, wherein the determination is not possible when a difference between the maximum luminance and the minimum luminance in the one-dimensional array is smaller than a threshold value.   The determination is made impossible when the array is binarized using a threshold value taking into account the luminance distribution in the one-dimensional array, and there are not two high-luminance regions and two low-luminance regions alternately. 4. The cross mark detection method according to 3.   In the determination, the distance between the even-numbered boundary or the odd-numbered boundary between the high-brightness area and the low-brightness area divided by the threshold considering the luminance distribution in the one-dimensional array is the length of the one-dimensional array. The cross mark detection method according to claim 3, wherein m is not allowed when m is not a value close to m / 2.   4. The determination according to claim 3, wherein the determination is not allowed when the number of pixels between adjacent boundaries delimited by a first threshold considering a luminance distribution in a one-dimensional array is smaller than a second threshold. Cross mark detection method.   In the determination, two types of templates having a cross mark-like luminance distribution and a band-like template are created through the points of the original image corresponding to the boundary points, and the similarity between the template having the cross mark-like luminance distribution is the highest. 4. The cross mark detection method according to claim 3, wherein the cross mark detection method is not allowed unless it is high.   The cross mark center coordinate calculation is performed by detecting a point having a large slope of the luminance distribution in the vicinity of a straight line connecting the point of the original image corresponding to the boundary point and the point of interest, and two parallel lines between the opposing boundary points. The cross mark detection method according to claim 2, wherein a two-dimensional coordinate value is calculated as an intersection of two intersecting representative lines.   The cross mark detection method according to claim 9, wherein when fitting two parallel straight lines between the opposing boundary points, the calculation fails when the distance between the two straight lines is larger than a threshold value. In a cross mark detection program that detects a cross mark from a captured luminance image,
A procedure for determining all pixels in the luminance image as the center of the cross mark and setting the result as a cross mark candidate point image;
A procedure for thickening the cross mark candidate point image and creating an area image;
A procedure for performing region division for assigning the same region number to the region image adjacent to the cross mark candidate points;
A procedure for calculating the center coordinate of the cross mark using the luminance pattern of the neighboring points centered on each cross mark candidate point, and simultaneously calculating a score that is the strength of the cross mark,
The cross mark candidate point having the largest score among the candidate points in the cross mark candidate point image having the same area number in the area image is adopted as the representative point of the area, and the cross mark two-dimensional coordinate values corresponding to the number of the area numbers are used. And the procedure to get
A cross mark detection program for causing a computer to execute each of the steps.