JP2021118516A - Intersection detection method and device and camera calibration device - Google Patents

Abstract

To detect an intersection for camera calibration on the basis of a camera image of a line segment, and outputs a camera parameter.SOLUTION: An image acquisition unit 10 acquires a camera image I in unit of frames. A competition area extraction unit 20 extracts a competition area from the camera image I. A line segment extraction unit 30 extracts a line segment from the camera image I. An intersection coordinate calculation unit 40 calculates the coordinates of the intersection of each line segment. A patch image extraction unit 50 extracts a patch image from the camera image I at each intersection such that the center of each patch image coincides with the coordinates of each intersection. A pre-processing unit 61 performs pre-processing to smooth the line segment in the patch image after thinning the line segment. A line segment identification unit 62 identifies the line segment on the patch image after the preprocessing. An intersection correction unit 60 corrects each intersection to the intersection of the identified line segment for each patch image. A homography matrix determination unit 90 determines a homography matrix for camera calibration on the basis of the corrected intersection.SELECTED DRAWING: Figure 1

Description

The present invention relates to an intersection detection method and device and a camera calibration device, and more particularly to an intersection detection method and device and a camera calibration device suitable for calibrating a camera for photographing a field competition.

Patent Document 1 discloses a technique of detecting a white line from a camera image and performing camera calibration by calculating a homography matrix using the detected white line and model data. In Patent Document 1, the area where the white line appears is extracted by threshold processing using a statistically obtained threshold. After that, the Hough transform is applied to the camera image, and the intersection is obtained by detecting the white line. The homography matrix is calculated by associating the detected intersections and directions of the white lines with the intersections and directions of the white lines in the model data.

In Patent Document 1, a plurality of homography matrices are calculated, but each homography matrix is applied to the n-th frame image and the m-th frame image, and the homography matrix obtained there is the minimum displacement of the camera point. Is selected and calibration is performed based on the selected homography matrix.

Non-Patent Document 1 discloses a technique called a Line Segment Detector (LSD) that extracts a region to be a line segment and detects the line segment by using the gradient of the edge of the input image. LSD is known as a line segment detection method with high detection accuracy.

Non-Patent Documents 2 and 3 disclose the latest line segment detection method using a convolutional neural network. The line segment detection method using a neural network is said to have higher line segment detection accuracy than the LSD of Non-Patent Document 1.

Patent Document 2 discloses a technique for extracting a competition area by threshold processing using the average value and standard deviation of the brightness and color difference of an input image.

Japanese Unexamined Patent Publication No. 2018-137667 Japanese Patent No. 6456244

R. G. V. Gioi, J. Jakubowicz, J. M. Morel and G. Randall, "LSD: a Line Segment Detector", Image Processing On Line, pp. 35-55, 2012. Y. Zhou, H. Qi and Y. Ma, "End-to-End Wireframe Parsing", pp. 962-971, ICCV 2019. N. Xue, S. Bai, F. Wang, G.-S. Xia, T. Wu and L. Zhang, "Learning Attraction Field Representation for Robust Line Segment Detection", pp.1595-1603, CVPR 2019.

In Patent Document 1, the success rate of camera calibration is low because the extraction accuracy of the area where the white line is reflected is low and the line segment detection accuracy is low because the classical method is used for the line segment detection method. There is a problem that it is low.

As a pre-stage process for detecting white lines, there is a process for extracting an area where white lines are reflected. In Patent Document 1, when extracting the region, a statistical value is uniquely determined, and the region is extracted by threshold processing using the determined value. There is a very difficult problem in determining the statistical value uniquely for each video scene. Further, Patent Document 1 does not disclose a specific method for obtaining a statistical value.

An object of the present invention is to solve the above technical problems and to provide an intersection detection method and device capable of accurately detecting an intersection suitable for calibrating a camera for shooting a field competition, and a camera calibration device.

In order to achieve the above object, the present invention is characterized in that it has the following configuration in an intersection detection device that detects an intersection for camera calibration based on a camera image showing a line segment.

(1) A means for extracting line segments from a camera image, a means for calculating the coordinates of intersections of each line segment, a means for extracting a patch image including each intersection from a camera image, and a means for thinning the line segments in the patch image. It is provided with a means for performing a pretreatment for smoothing later, a means for identifying a line segment on the patch image after the pretreatment, and a means for correcting each intersection for each patch image to the intersection of the identified line segments. did.

(2) The means for extracting the competition area from the camera image is provided, and the means for extracting the competition area from the camera image is a threshold value using the statistical value of the camera image after estimating the entire area of the camera image as the competition area. The threshold value was updated in the iterative process of gradually narrowing the area estimated to be the competition area by the processing, and the area estimated to be the competition area was extracted when a predetermined convergence condition was satisfied.

(3) The means for extracting a line segment from the camera image is a means for detecting a line segment from the camera image and a perpendicular line is drawn from a predetermined end of the camera image on each line segment, and the angle and length of the perpendicular line are determined. A means for calculating and a means for integrating line segments having a predetermined relationship between the angles and lengths of the perpendicular lines are provided.

(4) In a camera calibration device that outputs parameters for camera calibration based on a camera image showing a line segment, a means for extracting the line segment from the camera image and a means for calculating the coordinates of the intersection of each line segment. , A means for extracting a patch image including each intersection from a camera image, a means for performing a preprocessing for smoothing a line segment in the patch image after thinning, and a means for identifying a line segment on the preprocessed patch image. Means, means for correcting each intersection to the intersection of the identified line segment for each patch image, means for storing the intersection of the competition area model, and means for associating the corrected intersection with the intersection of the competition area model. It was provided with means for determining the homography matrix based on the mutually corresponding modified intersections and the intersections of the competition area model, and means for calculating the camera parameters based on the homography matrix. The means for determining the homography matrix are a means for generating a plurality of homography matrix candidates based on the mutually corresponding modified intersections and the intersections of the competition area model, and the means modified for each of the homography matrix candidates. Means for calculating the humming distance between the feature vector of each intersection and the feature vector of each intersection of the competition area model, each modified intersection for each of the homography matrix candidates, and each intersection of the competition area model are the homography matrix. A means for calculating the intersection error with each intersection projected based on the candidate, a line segment extending from each corrected intersection for each of the homography matrix candidates, and each intersection of the competition area model based on the homography matrix candidate. A means for calculating a direction error with a line segment extending from each intersection point projected by I tried to do it.

According to the present invention, the following effects are achieved.

(1) Extract the patch image including the intersections extracted from the camera image, perform the preprocessing to smooth the line segments in the patch image after thinning, and identify the line segments on the preprocessed patch image. Since each intersection is corrected to the intersection of the identified line segment for each patch image, the intersection can be calculated based on the line segment clarified and clarified by the preprocessing. Therefore, even if an intersection is erroneously detected based on a line segment located near the line segment constituting the original intersection, an accurate intersection can be recalculated with a small calculation load.

(2) The means for extracting the competition area from the camera image is to estimate the entire area of the camera image as the competition area and then gradually narrow the area to be estimated as the competition area by threshold processing using the statistical values of the camera image. Since the threshold value is updated in the process and the area estimated to be the competition area is extracted when a predetermined convergence condition is satisfied, the competition area can be extracted accurately.

(3) The means for extracting a line segment from a camera image is to detect a line segment from the camera image, draw a perpendicular line from a predetermined end of the camera image on each line segment, and calculate the angle and length of the perpendicular line. Since the line segments having a predetermined relationship in the angle and length of the perpendicular are integrated, the angle and length of the perpendicular as the feature amount can be obtained as a practical value regardless of the direction of the perpendicular. Become.

(4) For each homography matrix candidate, the humming distance is calculated, the intersection error is calculated, the direction error is calculated, and at least two of the humming distance, the intersection error, and the direction error are below a predetermined threshold. Since the homography matrix is determined, the accuracy of the homography matrix can be improved.

It is a functional block diagram of the camera calibration apparatus to which this invention is applied. It is a flowchart which showed the extraction procedure of the competition area. It is a figure which showed the method of obtaining the line segment feature amount. It is a functional block diagram of the intersection correction part. It is a figure which showed the correction method of the intersection point. It is a figure which showed the setting method of a feature vector element. It is a figure which showed the generation method of the camera intersection feature vector. It is the figure which showed the matching method of the feature vector Vc, Vm. It is a functional block diagram of the homography matrix determination part.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a functional block diagram showing a configuration of a main part of a camera calibration device 1 to which the intersection detection device of the present invention is applied.

Such a camera calibration device can be configured by implementing an application (program) that realizes each function on a general-purpose computer or server. Alternatively, it can be configured as a dedicated machine or a single-purpose machine in which a part of the application is made into hardware or software.

The image acquisition unit 10 acquires the camera image I of the field competition in frame units. The camera image I may be acquired directly from the camera during which the field competition is being photographed, or may be saved in an HDD or the like and then read out and acquired.

The competition area extraction unit 20 extracts the competition area from the camera image I in which the competition area of the field competition is captured together with the spectators' seats and the like. In many competitions, the competition area is composed of one color or a similar color, and the outside of the competition area is often a different color. In addition, it is empirically recognized that the color of the competition area occupies a larger area than the color of the other areas in the camera image of the competition.

Therefore, in the present embodiment, the feature peculiar to the camera image of the field competition is used, the entire area of the camera image is assumed to be the competition area, and then the competition area is determined by the threshold processing using the statistical value for the camera image I. The threshold value is automatically updated in the iterative process of gradually narrowing the estimated area, and the area estimated to be the competition area is extracted as the competition area when a predetermined convergence condition is satisfied.

FIG. 2 is a flowchart showing a procedure for extracting the competition area by the competition area extraction unit 20, and in step S1, the color space of the camera image I input in frame units is converted into the HSV color space. In step S2, the mask image mask (i, j) of the camera image I is initialized. In the present embodiment, "1" is registered for the pixel estimated to be the competition area of the mask image mask (i, j), and "0" is registered to the pixel estimated to be outside the competition area. When it is done, all the pixels become "1".

In step S3, it is determined whether or not the mask image update condition is satisfied based on the ratio of the competition area to the camera image. In the present embodiment, X is the number of pixels in the horizontal direction of the camera image I, Y is the number of pixels in the vertical direction of the camera force image I, and r is the upper limit ratio of the competition area to the camera image I (fixed coefficient value of 1 or less). , C is the value obtained by counting the number of mask (i, j) values of "1", and if the following equation (1) is satisfied, it is judged that the update condition of the mask image mask (i, j) is satisfied. ..

Immediately after the mask image mask (i, j) is initialized, the value of c is sufficiently large, and it is determined that the above equation (1) does not hold, so the process proceeds to step S4. ^{In step S4, the average value μ H} and standard deviation σ ^H of the H component (hue) of the camera image I, and the average value μ ^S and standard of the S component (saturation) of the camera image I using the following equations (2) and (3). The deviation σ ^S (formula omitted for the S component) is obtained. Here, I ^H is an array of H components, and i and j are subscripts representing pixel positions.

In step S5, the three threshold values used in the update equation (5) of the mask image mask (i, j) are updated based on the following equation (4). Here, α is a coefficient value of 1 or more that adjusts the range of the threshold processing of the update equation (5).

In step S6, the mask image mask (i, j) is updated based on the following equation (5). In the present embodiment _{, if the H component of pixels I i and j} is equal to or greater than the predetermined lower limit value H _MIN and equal to or less than the upper limit value H _MAX , and the S component is equal to or greater than the _{predetermined lower limit value S MIN, then "1".} If it is other than, it is set to "0".

In step S7, noise is removed by applying a median filter to the updated mask image mask (i, j). In step S8, the convergence test of the competition area estimation is made based on the following equation (6).

Here, the difference D is the difference between the current standard deviation σ ^H of the H component and the previous standard deviation σ ^H ', and is obtained by the following equation (7). If the above equation (6) is not satisfied, the process returns to step S3 and each of the above processes is repeated.

After that, if the determination in step S3 becomes affirmative, the process proceeds to step S10, and the update step width t less than 1 is added to the α. In step S11, the mask image mask (i, j) is replaced with the mask image mask_ (i, j) before the update.

After that, if the determination of the convergence test of the competition area estimation is affirmative in step S8, the process proceeds to step S9. In step S9, by applying the current mask image mask (i, j) to the camera image I based on the following equation (8), the images F _{i and j} of the competition area are extracted and output to the next stage.

In the above embodiment, the HSV color space has been described as an example, but the present invention is not limited to the color space.

Returning to FIG. 1, the line segment extraction unit 30 detects the line segments constituting the field line from the competition area of the camera image I. In the line segment extraction unit 30, the line segment detection unit 31 applies a well-known LSD to the competition area to detect a segment (line segment) which is a component of the field line. Each line fragment is defined by the start point coordinates and the end point coordinates. As shown in FIG. 3, the line segment feature amount calculation unit 32 draws a perpendicular line Pe from, for example, the upper left top of the camera image I on each line segment Sg or an extension line thereof, and the angle θ between the perpendicular line Pe and the x-axis. And the length ρ of the perpendicular line Pe is calculated as the feature quantity of each line segment Sg.

The line segment integration unit 33 calculates the difference between the angle θ and the length ρ as feature quantities for each pair of line segments, and integrates the pair of line segments that satisfy a predetermined condition. In the present embodiment, for two arbitrarily extracted line segments Sg ₁ = (ρ ₁ , θ ₁ ) and Sg ₂ = (ρ ₂ , θ ₂ ), one of them (for example, the line segment Sg ₁ ) is the midpoint thereof. Define the line segment Sg ₁ '= (ρ ₁ ', θ _{2 ) rotated by θ 2} − θ ₁ degree around the center, _{and make the line segment pairs Sg 1} and Sg ₂ that satisfy the following equation (9) the same. It is determined that it is a line segment that constitutes a line segment and integrated.

Here, T _θ and T _ρ are constant threshold values. Further, even if the line segments are on the same straight line, they are not integrated when the distance between the end points is equal to or more than the threshold value. The line segment integration unit 33 outputs the coordinates of each end point of the integrated line segment. The line segment pruning unit 34 classifies _{each integrated line segment into either a horizontal line segment L H} or a vertical line segment L _{V based on its slope k.}

In the present embodiment, the classification threshold value γ is defined in advance, and if | ki |> γ, _{it is classified as a vertical line segment L V} , and if | ki | ≤ γ, it is classified as a _{horizontal line segment L H.} In addition, the classification threshold value may differ depending on the position of the camera and the shooting direction. Therefore, after the classification process, the horizontal line segment L _H and the vertical line segment L _V are two subsets.

The sorting unit 35 sorts the horizontal line segment L _H from top to bottom based on its coordinates, and sorts the vertical line segment L _V from left to right based on its coordinates. When sorting the horizontal line segments L _H , use offsets to sort the corresponding line segments from top to bottom. Sorting of the vertical line L _V, focusing on the top of the horizontal line segment L _H to calculate the intersection between the horizontal line segment L _H and the vertical line L _V, each vertical line in accordance with the x-coordinate value of the intersection point Sort the minutes L _V.

The intersection coordinate calculation unit 40 detects the intersection of each sorted horizontal line segment and the vertical line segment, and calculates the coordinates thereof. The patch image extraction unit 50 extracts an N × M size patch image centered on the coordinate position of the intersection for each intersection from the camera image I.

As shown in FIG. 4, the intersection correction unit 60 includes a preprocessing unit 61, a line segment identification unit 62, an intersection calculation unit 63, a feature vector element generation unit 64, and a camera intersection feature vector generation unit 65, and is grayscaled. On each of the patch images, the position of the intersection calculated by the intersection coordinate calculation unit 40 is corrected, and the information of each corrected intersection is vectorized.

In the intersection correction unit 60, the preprocessing unit 61 rotates the patch image so that the line segment is in a vertical posture as preprocessing for each extracted line segment, and (2) applies a bilateral filter. Then, (3) the image is converted into a binary image of 0 or 255 by a predetermined threshold processing, (4) a median filter is applied to remove noise, and (5) the line segment from which noise is removed is shown in the figure. A preprocessed patch image is obtained by thinning the line segment as shown in 5 (a) and (6) giving a gradient to the thinned line segment by smoothing with a Gaussian filter applied.

By such a pretreatment, a line segment that can be extracted only incompletely by the line segment extraction unit 30 due to unclearness and unclearness can be clarified and clarified so that the line segment can be easily detected. Further, such preprocessing generally requires a large amount of calculation, but in the present embodiment, since the preprocessing is performed only on the patch image, it affects the intersection detection while suppressing the increase in the calculation amount. It is possible to easily detect substantially all line segments.

The line segment identification unit 62 identifies a line segment by applying a well-known LSD (Line Segment Detector) to the preprocessed patch image. In the present embodiment, as shown in FIG. 5B, by raster scanning of the preprocessed patch image, two line segments are detected along one side edge and the other side edge for each line segment. Therefore, the difference between the directions of the two line segments | θ | is obtained. Further, for a line segment pair satisfying | θ | <threshold TH, the midpoint coordinates of each of the one-sided and the other-side coordinates detected at each position of the raster scan are obtained, and the distance between the midpoint coordinates is the longest. The line segment passing through the point coordinate pair is identified as the modified line segment.

The intersection calculation unit 63 recalculates the intersection of the corrected line segments, and corrects the coordinates of the intersection based on the recalculation result as shown in FIG. 5 (c). The feature vector element generation unit 64 generates a feature vector element for each recalculated intersection based on the presence or absence of a line segment extending from the intersection in each direction. In the present embodiment, as shown in FIG. 6, the presence or absence of a line segment extending in each direction at each intersection is assigned to each bit of the feature vector element composed of 4-bit binary data, for example, "UP". The presence or absence in the direction is assigned to the most significant bit (MSB1).

Similarly, the presence / absence in the "RIGHT" direction is assigned to MSB2, the presence / absence in the "DOWN" direction is assigned to MSB3, and the presence / absence in the "LEFT" direction is assigned to the least significant bit (MSB4). Then, for each bit, "1" is set if the field line exists, and "0" is set if the field line does not exist. Therefore, the feature vector element of the intersection where the vertical field line extending in the "UP" direction and the vertical field line extending in the "LEFT" direction is detected is "1001".

The camera intersection feature vector generation unit 65 arranges the feature vector elements of each intersection in a predetermined order, for example, in the raster scan order as shown in FIG. 7, based on the camera intersections, so that the camera intersection features 65 Generate the vector Vc. In the example of FIG. 7, the camera intersection feature vector Vc is as shown in the following equation (10).

Vc = [1110, 0101, 0011, 1110, 0011, 1010, 1110, 0101, 1001] (10)

As described above, in the present embodiment, preprocessing is executed for each patch image to clarify and clarify the line segments in the patch image, and then each line segment is identified again, and the intersection is calculated based on the result of the identification. NS. Therefore, even if the line segment extraction unit 30 erroneously detects an intersection based on a line segment other than the line segment constituting the original intersection, the accurate intersection can be recalculated with a small calculation load.

Returning to FIG. 1, in the model intersection feature vector (Vm) element storage unit 70, for each intersection (model intersection) of each field line of the competition area model, the above is described in advance based on the number and direction of the field lines passing through the intersection. The model intersection feature vector element generated by the same procedure as in is stored.

The corresponding point setting unit 80 includes a matching candidate extraction unit 81, a model intersection feature vector generation unit 82, and a matching unit 83, and sets corresponding intersections (corresponding points) between the camera image and the competition area model.

The matching candidate extraction unit 81 defines each feature vector of the camera intersection feature vector Vc according to the number of _{horizontal field lines L H} and vertical field line L _V that define the intersections related to each feature vector element of the camera intersection feature vector Vc. A model with a similar array of elements and intersections Extracts a combination of intersection feature vector elements.

The model intersection feature vector generation unit 82 arranges each feature vector element for each combination of the model intersection feature vector elements to generate a plurality of model intersection feature vectors Vm. The matching unit 83 executes matching between the camera intersection feature vector Vc and each model intersection feature vector Vm, and corresponds based on the correspondence between the model intersection feature vector Vm having the highest similarity and the camera intersection feature vector Vc. Set a point.

FIG. 8 is a diagram schematically showing a matching method between the camera intersection feature vector Vc and the model intersection feature vector Vm.

If the camera intersection feature vector Vc is _{composed of, for example, six feature vector elements related to the intersection of three horizontal field lines L H} and two vertical field lines L _V connected in raster scan order, matching candidate extraction is performed. From the Vm element storage unit 70, the unit 81 is set to the intersection of the three horizontal field lines and the two vertical field lines of the competition area model, that is, the intersections of the camera images and the intersections (horizontal and vertical positions) similar to each other. All the sets of the six feature vector elements are extracted and connected in the order of raster scan to generate the model intersection feature vector Vm.

That is, if the competition area model is _{composed of 6 horizontal field lines L H} and 7 vertical field lines L _V , 3 arbitrarily selected from 6 horizontal field lines L _{H (6} C _3). = 20 ways) and 6 feature vector element combinations for each intersection of 2 lines ( ₇ C _{2 = 21 ways) arbitrarily selected from 7 vertical field lines L V} are acquired from the Vm element storage unit 70. Then, 6 feature vector elements are connected for each combination to generate 420 model intersection feature vectors Vm.

As shown in FIG. 9, the homography matrix determination unit 90 includes a homography matrix generation unit 91, a humming distance calculation unit 92, an intersection error calculation unit 93, and a direction error calculation unit 94, and includes a humming distance, an intersection error, and a direction. The homography matrix in which at least two of the errors are less than the threshold is determined as the homography matrix for camera calibration.

The homography matrix generation unit 91 generates a homography matrix by using the correspondence between the intersection coordinates of the camera image and the intersection coordinates of the competition area model, which are in the relationship of the corresponding points. In the present embodiment, nh lines from Nh horizontal line segments and nv lines from Nv vertical line segments are selected, and nv × nh intersections are determined. At this time, nh must be 2 or more and Nh or less, and nv must be 2 or more and Nv or less.

At this time, the line segment is selected in _Nh C _nh ways in the horizontal direction and _Nv C _nv ways in the vertical direction, so the combination pattern is _Nh C _nh × _Nv C _nv ways. In this embodiment, a homography matrix is required for all these patterns.

The Hamming distance calculation unit 92 calculates the Hamming distance between the camera intersection feature vector Vc and the model intersection feature vector Vm. The calculation result of the Hamming distance is used as an index for evaluating the validity of the camera intersection and the model intersection used for generating the homography matrix as corresponding points.
The intersection error calculation unit 93 calculates an error between the coordinates of each intersection of the competition area model projected using the homography matrix generated by the homography matrix generation unit 91 and the coordinates of each intersection of the camera image. The direction error calculation unit 94 extends from the line segment (white line) extending from each coordinate obtained by projecting each intersection coordinate of the competition area model using the homography matrix generated by the homography matrix generation unit 91 and the intersection coordinate of the camera image. Calculate the difference in direction from the line segment.

The homography matrix determination unit 90 selects a homography matrix in which at least two of the Hamming distance, the intersection error, and the direction error are less than a predetermined threshold value. If there is no homography matrix in which at least two are below the threshold value, an error is output and the calibration fails. If there are multiple candidates for the homography matrix, it can be narrowed down to one by the following procedure.

Step 1: Draw each horizontal and vertical line segment detected in the competition area with a white line on an image with a black background to generate a binary (0: black or 255: white) binary white line image W ⁰ . do.

Step 2: Convert the intersection coordinates of the competition area model using each homography matrix candidate, draw a white line on the image with a black background using the converted intersection coordinates, and generate ^{a white line image W (1 to K).} (The white line images generated here are the number of candidates for the homography matrix (K images)).

Step 3: The white line image W ₀ generated in step 1 and the white line image W ^{(1 to K)} generated in step 2 are overlapped with each ^{other, and the white line image W L} (L) having the largest area overlapping with the white line of ^{W 0 is obtained.} Determines the homography matrix used to generate (an integer between 1 and K) as the output homography matrix.

The camera parameter calculation unit 100 calculates and outputs a camera calibration matrix based on the determined homography matrix and the internal parameters of the camera.

10 ... image acquisition unit, 20 ... competition area extraction unit, 30 ... line segment extraction unit, 40 ... intersection coordinate calculation unit, 50 ... patch image extraction unit, 60 ... intersection correction unit, 61 ... preprocessing unit, 62 ... line segment Identification unit, 63 ... Intersection calculation unit, 64 ... Feature vector element generation unit, 65 ... Camera intersection feature vector generation unit, 70 ... Vm element storage unit, 80 ... Corresponding point setting unit, 81 ... Matching candidate extraction unit, 82 ... Intersection point Feature vector generation unit, 83 ... matching unit, 90 ... homography matrix determination unit, 91 ... homography matrix generation unit, 92 ... humming distance calculation unit, 93 ... intersection error calculation unit, 94 ... direction error calculation unit, 100 ... camera Parameter calculation unit

Claims (15)

In an intersection detection device that detects an intersection for camera calibration based on a camera image showing a line segment,
A means to extract line segments from camera images,
A means to calculate the coordinates of the intersection of each line segment,
A means to extract a patch image including each intersection from a camera image,
A means of performing preprocessing to smooth the line segments in the patch image after thinning,
A means of identifying line segments on the preprocessed patch image,
An intersection detection device comprising means for correcting each intersection for each patch image to the intersection of the identified line segments. The intersection detection device according to claim 1, wherein the means for extracting the patch image extracts each patch image so that the center of each patch image coincides with the coordinates of each intersection. The intersection detection device according to claim 1 or 2, wherein the means for performing the preprocessing is to perform the preprocessing on a grayscaled patch image. The means for identifying the line segment is
A means for detecting edge coordinate pairs on one end side and the other end side by performing edge detection for each line segment in the patch image after preprocessing is provided.
Claims 1 to 3 are characterized in that the center coordinates of each edge coordinate are obtained for each edge coordinate pair, and the line segment passing through each center coordinate of the center coordinate pair having the longest distance between the center coordinates is used as the identification result. The intersection detection device according to any one of. Equipped with a means to extract the competition area from the camera image,
The intersection detection device according to any one of claims 1 to 4, wherein the means for extracting the line segment is for extracting the line segment from the competition area. The means for extracting the competition area updates the threshold in an iterative process in which the entire area of the camera image is estimated as the competition area and then the area estimated to be the competition area is gradually narrowed by the threshold processing using the statistical values of the camera image. The intersection detection device according to claim 5, wherein a region estimated to be a competition region is extracted when a predetermined convergence condition is satisfied. The means for extracting a line segment from the camera image is
A means to detect line segments from camera images,
A means of drawing a perpendicular line from a predetermined end of a camera image on each line segment and calculating the angle and length of the perpendicular line.
The intersection detection device according to any one of claims 1 to 6, further comprising a means for integrating line segments having a predetermined relationship between the angles and lengths of the perpendicular lines. In an intersection detection method in which a computer detects an intersection for camera calibration based on a camera image showing a line segment.
Extract the line segment from the camera image and
Calculate the coordinates of the intersection of each line segment
Extract the patch image including each intersection from the camera image and
Pre-processing is performed to smooth the line segments in the patch image after thinning them.
Identify the line segment on the preprocessed patch image and
A method for detecting an intersection, which comprises modifying the coordinates of each intersection for each patch image to the coordinates of the intersection of the identified line segment. When extracting a line segment from the camera image
Detects line segments from camera images and
A perpendicular line is drawn from a predetermined end of the camera image on each line segment, and the angle and length of the perpendicular line are calculated.
The intersection detection method according to claim 8, wherein the line segments having a predetermined relationship between the angles and lengths of the perpendicular lines are integrated. In a camera calibration device that outputs parameters for camera calibration based on a camera image showing a line segment.
A means to extract line segments from camera images,
A means to calculate the coordinates of the intersection of each line segment,
A means to extract a patch image including each intersection from a camera image,
A means of performing preprocessing to smooth the line segments in the patch image after thinning,
A means of identifying line segments on the preprocessed patch image,
A means for correcting each intersection to the intersection of the identified line segments for each patch image, and
A means of memorizing the intersections of the competition area model,
A means of associating the modified intersection with the intersection of the competition area model,
A means of determining the homography matrix based on the mutually corresponding modified intersections and the intersections of the competition area model,
A camera calibration apparatus including a means for calculating camera parameters based on the homography matrix. The means for determining the homography matrix is
A means of generating multiple homography matrix candidates based on the mutually corresponding modified intersections and the intersections of the competition area model,
A means for calculating the Hamming distance between the modified feature vector of each intersection and the feature vector of each intersection of the competition area model for each of the homography matrix candidates.
For each of the homography matrix candidates, a means for calculating the intersection error between each corrected intersection and each intersection projected from each intersection of the competition area model based on the homography matrix candidate.
As a means for calculating the directional error between the line segment extending from each of the modified intersections and the line segment extending from each intersection projected from each intersection of the competition area model based on the homography matrix candidate for each of the homography matrix candidates. Equipped with
The camera calibration apparatus according to claim 10, wherein at least two of the Hamming distance, the intersection error, and the direction error are less than a predetermined threshold value, and the homography matrix candidate is determined as the homography matrix. The means for identifying the line segment is
A means for detecting edge coordinate pairs on one end side and the other end side by performing edge detection for each line segment in the patch image after preprocessing is provided.
10. The camera calibrator described in. Equipped with a means to extract the competition area from the camera image,
The camera calibration apparatus according to any one of claims 10 to 12, wherein the means for extracting the line segment is to detect the line segment from the competition area. The means for extracting the competition area updates the threshold in an iterative process in which the entire area of the camera image is estimated as the competition area and then the area estimated to be the competition area is gradually narrowed by the threshold processing using the statistical values of the camera image. The camera calibration apparatus according to claim 13, wherein a region estimated to be a competition region is extracted when a predetermined convergence condition is satisfied. The means for extracting a line segment from the camera image is
A means to detect line segments from camera images,
A means of drawing a perpendicular line from a predetermined end of a camera image on each line segment and calculating the angle and length of the perpendicular line.
The camera calibration apparatus according to any one of claims 10 to 14, further comprising a means for integrating line segments having a predetermined relationship between the angles and lengths of the perpendicular lines.

Images