CN114882058B

CN114882058B - Corner detection method, corner detection device and calibration plate

Info

Publication number: CN114882058B
Application number: CN202210446262.5A
Authority: CN
Inventors: 李怡康; 闫国行; 刘卓纯
Original assignee: Shanghai AI Innovation Center
Current assignee: Shanghai AI Innovation Center
Priority date: 2022-04-26
Filing date: 2022-04-26
Publication date: 2024-06-07
Anticipated expiration: 2042-04-26
Also published as: CN114882058A

Abstract

The application belongs to the technical field of image processing, and particularly relates to a corner detection method, a corner detection device and a calibration plate. The problem that the inner corner points are missed in the process of detecting the inner corner points of the calibration plate in the image recognition mode in the prior art can be solved. The method comprises the steps of obtaining an image to be detected, wherein the image to be detected comprises a calibration plate, the calibration plate comprises a plurality of inner corner points, and a preset arrangement relation exists between a first part of corner points and a second part of corner points in the plurality of inner corner points; identifying first partial corner points in the image to be detected; and according to a preset arrangement relation, determining a second part of corner points in the image to be detected by combining the first part of corner points.

Description

Corner detection method, corner detection device and calibration plate

Technical Field

The application belongs to the technical field of image processing, and particularly relates to a corner detection method, a corner detection device and a calibration plate.

Background

The intelligent driving assistance system (ADVANCEDDRIVERASSISTANCESYSTEM, ADAS) can collect environmental data inside and outside the vehicle during the running of the vehicle by using different kinds of sensors installed on the vehicle, such as millimeter wave radar, laser radar, single/double camera, satellite navigation and the like, so as to assist a user to drive the vehicle.

The ADAS function requires calibration of the ADAS camera before use, including calibration of vanishing points, ground homography matrix, angular displacement, etc. of the ADAS camera. At present, an ADAS camera is usually calibrated by using a calibration plate, during calibration, a calibration image needs to be acquired first, the positions of all inner corner points of the calibration plate on the calibration image are determined, and then subsequent calibration is performed based on the positions of all inner corner points.

At present, the positions of all inner corner points of a calibration plate in a calibration image are usually detected by adopting an image recognition mode. However, in the process of detecting the inner corner points, the variability of the scene environment (such as the change of the ambient brightness) may cause the ambiguity of the calibration image acquired by the equipment to be calibrated, so that all the inner corner points belonging to the calibration plate in the calibration image cannot be identified, and the condition of missed detection of the inner corner points occurs.

Disclosure of Invention

In view of the above, the embodiments of the present application provide a method and an apparatus for detecting an angular point, and a calibration board, so as to solve the problem in the prior art that an inner angular point is missed when an inner angular point of the calibration board is detected by means of image recognition.

A first aspect of an embodiment of the present application provides a corner detection method, including: obtaining an image to be detected, wherein the image to be detected comprises a calibration plate, the calibration plate comprises a plurality of inner corner points, and a preset arrangement relation exists between a first part of corner points and a second part of corner points in the plurality of inner corner points; identifying the first partial corner points in the image to be detected; and according to the preset arrangement relation, the second partial corner points in the image to be detected are determined by combining the first partial corner points.

With reference to the first aspect, in a first possible implementation manner of the first aspect, the calibration board includes M groups of calibration cells, M-1 common intersecting lines exist between the M groups of calibration cells, the intersecting lines include an edge line and a middle line, the middle line is an intersecting line located at a middle position in the M-1 intersecting lines, and the edge line is an intersecting line other than the middle line in the M-1 intersecting lines; the first part of corner points comprise any two inner corner points on each side line and inner corner points at two ends on the center line, wherein M is more than 2 and is an even number; the second part of corner points are other inner corner points except the first part of corner points in the calibration plate.

With reference to the first possible implementation manner of the first aspect, in a second possible implementation manner of the first aspect, in each set of calibration cells, each calibration cell is adjacent to or is arranged at intervals along a first direction, and the M sets of calibration cells are sequentially arranged along a second direction in units of sets, where the second direction is perpendicular to the first direction; the calibration grids are the same in size and are square.

With reference to the first aspect, in a third possible implementation manner of the first aspect, identifying the first partial corner in the image to be detected includes: identifying all corner points in the image to be detected; determining a straight line model according to the positions of every two corner points; screening M-1 item mark straight line models comprising the inner angle points from all the straight line models; and identifying the first part of corner points from the M-1 item mark straight line model.

With reference to the first aspect, in a fourth possible implementation manner of the first aspect, selecting an M-1 item mark straight line model including the inner angle point from all the straight line models includes: screening all the straight line models which accord with the preset slope conditions and the corner number conditions from all the straight line models; performing linear combination clustering on all the screened linear models in a mode of similar slope; and screening all the clustered straight line combinations according to the distance between the inner corner points, and determining the M-1 item standard straight line model.

With reference to the first aspect, in a fifth possible implementation manner of the first aspect, the slope condition is determined according to a positional relationship between all straight lines in the straight line model and a first direction; and the corner number condition is determined according to the number of actual inner corner points on each intersection line in the calibration area.

With reference to the first aspect, in a sixth possible implementation manner of the first aspect, performing linear combination clustering on all screened linear models in a manner that slopes are similar includes: determining linear models with similar slopes in all the linear models, and generating a plurality of linear combinations; determining a midline of each of the combinations of lines; determining all straight line combinations with similar slopes; according to the preset position relation between the corner points on the side lines and the central line, the side lines in all the linear combinations are screened out from the linear combinations with similar slopes; and clustering the line combinations of the midline and the side line of each line combination.

With reference to the first aspect, in a sixth possible implementation manner of the first aspect, determining, according to the preset arrangement relationship, the second partial corner point in the image to be detected by combining the first partial corner point includes: determining the position of the first partial corner points; and determining the second partial corner points in the image to be detected according to the group number and the setting position of the calibration cells in the calibration plate.

A second aspect of an embodiment of the present application provides a corner detection apparatus, including: the device comprises an acquisition unit, a detection unit and a detection unit, wherein the acquisition unit is used for acquiring an image to be detected, the image to be detected comprises a calibration plate, the calibration plate comprises a plurality of inner corner points, and a preset arrangement relation exists between a first part of corner points and a second part of corner points in the plurality of inner corner points; the identification unit is used for identifying the first partial corner points in the image to be detected; the determining unit is used for determining the second part of corner points in the image to be detected by combining the first part of corner points according to the preset arrangement relation.

A third aspect of the embodiments of the present application provides a calibration plate, including a substrate and a calibration area on the substrate; the calibration area comprises a plurality of corner points, and a preset corresponding relation exists between a first part of corner points and a second part of corner points in the plurality of corner points; the preset corresponding relation is used for determining the second part angular point when the electronic equipment identifies the first part angular point but does not identify the second part angular point.

A fourth aspect of the embodiments of the present application provides a corner detection device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, the processor implementing the steps of the method according to any one of the first aspects when the computer program is executed.

A fifth aspect of the embodiments of the present application provides a computer readable storage medium storing a computer program which, when executed by a processor, implements the steps of the method according to any of the first aspects.

Compared with the prior art, the embodiment of the application has the beneficial effects that: the application provides a corner detection method, a corner detection device and a calibration plate. According to the method, under the condition that the inner corner points of the calibration plate are not detected, all the undetected inner corner points can be detected according to the preset arrangement relation, and the problem that the inner corner points are not detected when the inner corner points of the calibration plate are detected in an image recognition mode in the prior art is solved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a schematic structural view of a calibration plate according to an embodiment of the present application;

FIG. 2 is a schematic structural view of another calibration plate according to an embodiment of the present application;

FIG. 3 is a schematic structural view of another calibration plate according to an embodiment of the present application;

FIG. 4 is a schematic structural view of another calibration plate according to an embodiment of the present application;

FIG. 5 is a schematic structural view of another calibration plate according to an embodiment of the present application;

FIG. 6 is a schematic diagram of a calibration image provided by an embodiment of the present application;

fig. 7 is a flowchart of a corner detection method according to an embodiment of the present application;

FIG. 8 is a schematic illustration of another calibration image provided by an embodiment of the present application;

fig. 9 is a schematic diagram of the positions of the first corner point and the second corner point provided by the embodiment of the application;

FIG. 10 is a schematic diagram of a straight line model in a calibration image according to an embodiment of the present application;

FIG. 11 is a schematic view of the positions of the start point and the end point of a straight line according to an embodiment of the present application;

FIG. 12 is a schematic view of a linear assembly provided by an embodiment of the present application;

FIG. 13 is a schematic view of projection points in a straight line combination according to an embodiment of the present application;

FIG. 14 is a schematic view showing the positional relationship between the line spacing and the dot spacing according to the embodiment of the present application;

FIG. 15 is a schematic diagram of a correspondence between first partial corner points and second partial corner points of a calibration plate according to an embodiment of the present application;

fig. 16 is a schematic diagram of a corner detection device according to an embodiment of the present application;

Fig. 17 is a schematic diagram of a corner detection apparatus according to an embodiment of the present application.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth such as the particular system architecture, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

The technical scheme provided by the application is explained in detail below with reference to specific embodiments.

In recent years, the domestic ADAS market size increases at a rate close to twice a year, and more vehicles have ADAS functions, and in order to make the ADAS functions normally used, calibration of ADAS cameras is required. The ADAS camera calibration comprises calibration of parameters such as vanishing points, ground homography matrix, angular displacement and the like. In ADAS camera calibration, calibration images are first acquired by a device to be calibrated (e.g., an ADAS camera). Then, the image processing device carries out corner detection on the calibration image in an image recognition mode, and positions of all inner corners on the calibration plate are determined. Wherein the calibration image comprises an image of a calibration plate. And finally, completing the subsequent calibration process based on the position of each inner corner point, and finally obtaining parameters such as vanishing points, a ground homography matrix, angular displacement and the like. The image processing device may be a device to be calibrated, or may be other electronic devices, which is not limited in this embodiment.

In this embodiment, the corner points are extreme points in the index image. For example, the corner point may be the intersection point of two lines in the calibration image, or may be a point on two adjacent target objects in different main directions in the calibration image. The inner corner points are corner points which are not contacted with the edges of the calibration areas in the calibration areas of the special calibration plates.

When ADAS camera calibration is carried out, the detection of the internal angle point of the calibration plate is a main factor affecting the smooth progress of the whole calibration flow. At present, all inner corner points of a calibration plate in the calibration image are usually detected by adopting an image identification mode. However, in the process of detecting the inner corner points, variability of the scene environment (such as variation of ambient brightness) may cause unclear calibration images acquired by the device to be calibrated, so that the image processing device cannot identify all the corner points belonging to the calibration plate in the calibration images, and corner point missing detection occurs. It can be understood that, because the calibration process needs to use the position information of all the inner corner points on the calibration plate, once the inner corner points of the calibration image are missed, the subsequent calibration process will fail.

When parameters such as vanishing points, ground homography matrix, angular displacement and the like are calculated through the traditional calibration plate, a calibration algorithm used by the calibration plate is required to calibrate the detection position of each inner corner point in a calibration image. When the inner corner detection is performed by the image recognition method, the corner detection is often failed due to the variability of the scene environment under the influence of various factors such as brightness, calibration background and the like. Once the corner missing detection condition exists, the subsequent flow cannot be executed, and the whole calibration flow is judged to be failed.

On the other hand, in order to detect all the inner corners in the calibration plate, the conventional calibration process generally needs to use a calibration plate with a larger size (the width is usually about 2.5 meters), and in order to facilitate the calculation process, the user needs to adjust the placement position of the calibration plate so as to keep the center height of the calibration plate to be the same as the center height of the camera to be calibrated. In the calibration process, the movement and placement process of the calibration plate are relatively complicated, the calibration efficiency is low, and the user experience is poor.

Therefore, whether the placement position and the inner angle point of the calibration plate are successfully detected directly influences the calibration efficiency and success rate of ADAS camera calibration.

Based on the above, the technical scheme of the application provides the calibration plate and the corner detection method based on the calibration plate, and by the technical scheme, the calibration equipment can still detect the interior corner points which are missed to be detected on the calibration plate without influencing the subsequent calibration flow under the condition that all the interior corners of the calibration plate on the calibration image are not identified by the image identification algorithm. In addition, the method has no special requirements on the size and the placement position of the calibration plate in the process of detecting the angular points, and the calibration plate is convenient to use.

Fig. 1 is a schematic structural diagram of a calibration plate according to an embodiment of the present application. Referring to fig. 1, the calibration board includes a calibration substrate and a calibration area on a plane of the calibration substrate, where the calibration area is used for calibrating external parameters of a camera to be calibrated. The camera to be calibrated can be a vehicle-mounted camera, a digital camera and the like, and the external parameters of the camera to be calibrated refer to the size of an actual object in an image shot by the camera to be calibrated.

In some embodiments, the calibration area includes at least four even sets of calibration cells, each of the same size and square. In each set of calibration cells, the calibration cells are adjacent or spaced apart along the first direction. In the even number of calibration cells, each group of calibration cells is sequentially arranged along a second direction by taking the group as a unit, and the second direction is mutually perpendicular to the first direction.

In one example, referring to fig. 1, the first direction is vertical, the second direction is horizontal, and each calibration cell in each set of calibration cells is disposed adjacent or spaced apart along the vertical direction, and each set of calibration cells is disposed adjacent to each other in sequence along the horizontal direction.

In another example, referring to fig. 2, the first direction is transverse, the second direction is vertical, and each calibration grid in each set of calibration grids is disposed adjacent or spaced apart along the transverse direction, and each set of calibration grids is disposed adjacent to each other along the vertical direction.

In addition, in this embodiment, each set of calibration cells arranged along the second direction is sequentially referred to as a first set of calibration cells, a second set of calibration cells, a third set of calibration cells, a fourth set of calibration cells … …, and so on according to the left-to-right sequence of the drawing. It should be noted that, in the plurality of calibration cells (for example, the second calibration cell and the third calibration cell in fig. 1) located in the middle portion, each calibration cell in each set of calibration cells needs to be set at intervals by one calibration cell in turn, that is, a space is left between every two calibration cells, and the space and the calibration cells have the same size. While in the two sets of calibration cells located on the outer side (for example, the first set of calibration cells and the fourth set of calibration cells in fig. 1), the number and the position of each calibration cell may be arbitrarily set.

Because each group of calibration grids are adjacently and sequentially arranged along the second direction, a common intersecting line exists between every two adjacent groups of calibration grids. In addition, two sets of calibration grids on two sides have two outer side contour lines. It will be appreciated that the intersecting lines are parallel to each other and to the first direction. In this embodiment, the intersection line located at the middle position among all the intersection lines is referred to as a center line, and the intersection lines on both sides of the center line are referred to as side lines.

It should be understood that, in the calibration area of the calibration plate of this embodiment, since the calibration grid sets are disposed adjacently or at intervals by a plurality of calibration grids which are identical in size and are square, the center line is equal to each side line or the interval between the side lines in this calibration area.

In this embodiment, the inner corner points in the calibration plate are points indicating the intersection points between adjacent calibration grids in the calibration area, or inflection points on each calibration grid, and the inner corner points are located on the center line and the side line, and the corner points on the contour lines on both sides in the calibration area do not belong to the inner corner points. Such as the illustrated crossover points and inflection point locations in fig. 3.

In the calibration area of the calibration plate provided in this embodiment, the number of internal angle points on each intersection is greater than a preset number. In one example, the preset number is 6.

For example, taking the calibration plate including four groups of calibration cells as an example, referring to fig. 3, the calibration plate includes three intersecting lines, where a common intersecting line between the first group of calibration cells and the second group of calibration cells is a first side line, a common intersecting line between the second group of calibration cells and the third group of calibration cells is a middle line, and a common intersecting line between the third group of calibration cells and the fourth group of calibration cells is a second side line. The number of corner points on the straight lines of the first side line, the middle line and the second side line is required to be larger than 6.

The number of intersections in the calibration area increases with the number of calibration cells. For example, when six calibration cells are preset in the calibration area, the number of intersecting lines is five, including a center line between the third calibration cell set and the fourth calibration cell set, and four side lines formed between the other calibration cell sets, as shown in fig. 4. When eight calibration grids are preset in the calibration area, the number of intersecting lines is seven, and the intersecting lines comprise a central line between the fourth calibration grid set and the fifth calibration grid set and six side lines formed between the other calibration grid sets, as shown in fig. 5.

It should be understood that in the process of detecting the corner points in the calibration plate by using the algorithm specifically, since the complexity of the algorithm increases successively in the calculation of the subsequent corner points as the number of straight lines increases, in order to adapt to the size of the calibration plate and the complexity of the algorithm, the number of intersecting lines in the calibration area is typically selected to be three, see the calibration plate shown in fig. 3. Therefore, the size of the calibration plate is adapted, and interference caused by too many straight lines can be avoided during subsequent corner detection.

Alternatively, in order to be suitable for practical applications, the width of the calibration plate in this embodiment may be set to about 0.5 meters.

Based on the calibration plate provided in the above embodiment, the embodiment of the present application further provides a corner detection method, which is used for determining a corner belonging to the calibration plate in the calibration image in the process of calibrating the camera.

In the process of angular point detection, the placement positions of the calibration plates in the calibration images can be set to be any positions, including vertical placement and horizontal placement. In the embodiment, the calibration plate in vertical arrangement during corner detection is called a vertical calibration plate, and can be shown in fig. 1, 3, 4 and 5; and the transversely arranged calibration plate is referred to as a transverse calibration plate, see fig. 2. The method for detecting the corner point of the calibration plate provided in this embodiment is explained in detail below by taking the vertical calibration plate shown in fig. 6 as an example. The calibration plate appearing hereinafter defaults to a vertical calibration plate when no special description is made.

Fig. 7 is a flowchart of a corner detection method provided in this embodiment. Referring to fig. 7, the corner detection method includes the following steps S1 to S7.

S1, the image processing equipment acquires a calibration image, wherein the calibration image is an image comprising a calibration plate.

The calibration image is an image taken by the camera to be calibrated, and the calibration plate included in the calibration image may be a calibration plate as described above. The calibration image may be a black and white image or a color image.

The image processing device and the camera to be calibrated can be the same device or different devices. When the image processing equipment and the camera to be calibrated are the same equipment, the image shot by the camera to be calibrated is the calibration image. When the image processing device and the camera to be calibrated are different devices, the camera to be calibrated can send the shot calibration image to the image processing device, and then the image processing device performs further processing.

In order to improve the accuracy of the corner detection and calibration results, the calibration image acquired by the image processing device may not include other background information. Of course, background information, i.e. information outside the calibration plate, may also be included in the shooting process, and in order to improve the accuracy of corner detection, a noise image similar to the corner of the calibration area should be avoided from appearing in the background image in the shot calibration image.

Fig. 6 is a schematic diagram of a calibration image according to an embodiment of the present application, and fig. 8 is a schematic diagram of a calibration image according to another embodiment of the present application. Both calibration images include the calibration plate and the background information of the calibration plate, the background information in fig. 8 is more complex than the background information in fig. 6, in the corner detection process, the accuracy of corner detection in fig. 8 is lower than that in fig. 6, and noise images similar to the corner points of the calibration area are not contained in the background information of both images.

S2, performing gray processing on the calibration image by the image processing equipment to obtain an image to be detected.

Optionally, after the image processing device obtains the calibration image, gray processing is required to be performed on the calibration image to obtain the image to be detected.

It should be understood that if the camera to be calibrated captures a calibration image that is not a gray image, gray processing needs to be performed on the calibration image to obtain a gray image corresponding to the calibration image. If the calibration image acquired by the image processing device is a gray image, the electronic device does not need to perform gray processing on the calibration image, that is, S2 is not needed to be executed after S1, and S3 is directly executed.

S3, the image processing equipment acquires all corner points in the image to be detected.

All the corner points in the image to be detected comprise inner corner points belonging to the marked area in the marking plate and the corner points of the object image in the background information. For example, referring to fig. 9, the first corner in the figure is the inner corner of the marked area in the marking plate, and the second corner is the corner of the object image in the background information.

In some embodiments, the image processing device may employ a corner detection algorithm to determine all corners in the image to be detected. The corner recognition algorithm may be any one of Shi-Tomasi corner detection algorithm, harris corner detection algorithm, moravec corner detection algorithm, for example.

Taking Shi-Tomasi corner detection algorithm as an example, the image processing device can label all possible corners in the whole image to be detected according to gradient information of colors in the image to be detected. In one example, the number of corner points determined by the Shi-Tomasi corner point detection algorithm may be set within 0-500.

Referring to fig. 7, fig. 7 is a schematic diagram of distribution of all corner points obtained by the Shi-Tomasi corner point detection algorithm in an embodiment of the present application. The positions of all corner points in the image to be detected after detection by the corner point detection algorithm are also shown in fig. 6 and 8 described above, for example.

The angular point detection in the embodiment is performed based on the gray level image, the adaptability of the detection process to different illumination environments is high, and all angular points in the image can be detected in multiple directions in the detection process, so that all potential angular points can be detected under the condition that the angular points are coincident, and the detection method has certain redundancy and can detect all angular points in the image to be detected.

S4, the image processing equipment establishes angular point linear models according to all angular points in the image to be detected, and determines all linear models which accord with the slope close to the slope of the linear in the first direction.

Similar means that the difference between the slope of the straight line where the corner point is located and the slope of the straight line where the first direction is located is within a preset range, and the preset range may be 0-10 ^-2 mm, for example.

In some embodiments, the image processing apparatus may determine all the straight line models that meet the preset conditions sequentially through the processes of determining straight lines, calculating the distances from corner points to straight lines, setting screening conditions, and the like, as described in detail below.

Firstly, a random sampling consensus (Random Sample Consensus, RANSAC) algorithm is adopted to determine a straight line according to every two corner points in all corner points of an image to be detected. Taking the example that the two corner points are { P ₀,P₁ }, the electronic device can establish a linear equation ax+by+c=0 according to the two corner points, wherein specific numerical values of a, b and c are determined according to specific coordinate values of { P ₀,P₁ }. It will be appreciated that if there are N corner points in the image to be detected, it is possible to determineAnd (5) straight lines. For example, when N is 30,I.e., the electronic device may determine 435 straight lines.

Next, the point on the straight line is denoted as P _n＝{x_n,y_n, and the distance from the point to the straight line is determinedTraversing each straight line in the image to be detected, and screening all angular points contained in each straight line, wherein the specific value of abc is determined according to the specific straight line traversed as required. The distance between each angular point in the image to be detected and each straight line is determined, and if the distance between the angular point and a certain straight line is within a threshold range (such as 0-10 ^-2 mm), the angular point is determined as the angular point on the straight line. By this method, all corner points included on each straight line can be determined.

Then, the image processing apparatus screens out straight lines that satisfy the slope condition and satisfy the corner number condition from all the straight lines. The slope condition means that the difference value between the slope of the straight line where the corner point is located and the slope of the straight line where the first direction is located is in a preset range, and the corner point quantity condition means that the difference value between the corner point quantity and the actual corner point quantity on each straight line in the calibration area is in the preset range.

It should be understood that, in this embodiment, when the calibration plate used in the calibration process is the vertical calibration plate shown in fig. 3, the first direction is the vertical direction in fig. 3, and the slope condition is set such that the slope k of the straight line is greater than 10 or less than-10; the number of corner points on each vertical line is at least 16, and therefore, when the condition of the number of corner points is set, the range can be set to 10-30.

In other embodiments, when the calibration plate used in the calibration process is a transverse calibration plate, that is, the first direction is a transverse direction, the slope condition may be that the value of the slope k ranges from 0 to 0.5, or-0.5 to 0, and the number of corner points is the same as the above.

It should be noted that, in practical application, in order to adapt to a specific algorithm, when the calibration plate is selected, the advantage of the vertical calibration plate compared with the transverse calibration plate is usually more prominent. When the slope of the straight line is obtained, the formula of the sine function enables the change of the slope at the unit angle of about 90 degrees to be relatively large, and the method is suitable for filtering during straight line screening. While the transverse calibration plate can achieve the same function, the transverse calibration plate has more linear interference than the vertical calibration plate in terms of the environment calibrated by a workshop. Of course, the selection may be performed according to an actual application environment, only one application scenario and implementation manner are provided in this embodiment, and the adjustment may be specifically performed according to an actual situation during application, which is not limited in this embodiment.

The image processing device repeatedly filters and screens all straight lines based on the slope conditions and the corner number conditions, so that all vertical straight lines which accord with the specific corner number and have the slope within a certain range are obtained. As shown in fig. 10, after the straight lines established by all the corner points in fig. 8 are screened, several vertical straight line diagrams as shown in fig. 10 are obtained.

S5, the image processing equipment performs linear combination clustering on all the screened linear models in a mode that slopes are similar.

In this embodiment, the image processing apparatus may sequentially determine reference straight lines, calculate distances from corner points on each straight line in the image to be detected to the reference straight lines, and sort corner points on each vertical straight line in the image to be detected, as shown in detail below.

First, the image processing apparatus determines a reference straight line, which may be a straight line adjacent to a straight line for which corner sorting is required, or may be any straight line other than the straight line.

Then, the distance from each corner point P _n＝{x_n,y_n on each vertical straight line in the image to be detected to the reference straight line is calculated according to the direction vector vec=p ₁-p₀ of the reference straight line. Where p ₁ and p ₀ are any two points on the reference line.

And finally, sorting each angular point on each vertical straight line according to the distance from the angular point on each vertical straight line to the reference straight line, and screening out the starting point and the ending point of the angular point on the vertical straight line.

In this embodiment, the starting point refers to a first corner point at one end of a straight line in the image to be detected; the end point refers to the first corner point at the other end of the line.

When the image processing device performs corner sorting on each vertical straight line in the image to be detected, the image processing device can perform corner sorting on each vertical straight line according to the distance from the corner point on each vertical straight line to the reference straight line and a preset linear relation between the straight lines. The preset linear relation may be that the starting point on the straight line to be ordered is the point with the largest distance from the straight line to be ordered to the reference straight line, and the end point is the point with the smallest distance from the straight line to be ordered to the reference straight line. Or the preset linear relation can be that the starting point on the straight line to be ordered is the point with the largest distance from one end of the straight line to be ordered to the reference, and the end point is the point with the largest distance from the other end of the straight line to be ordered to the reference.

For example, referring to fig. 11, taking the line a in fig. 11 as an example, the distance from the corner point on the line B to the line a is calculated, that is, the line a is a reference line, the line B is a line to be subjected to corner point sorting, and the image processing apparatus sorts the corner points on the line B according to the distance, so as to obtain the start point and the end point on the line B. The image processing apparatus determines a start point and an end point on a line B, which is a point on the line B having the largest distance to the line a and an end point which is a point on the line B having the smallest distance to the line a, according to a preset linear relationship, that is, the start point on the line to be sorted is a point on the line to be sorted having the largest distance to the reference line, and the end point is a point on the line B having the smallest distance to the line a. On the contrary, the distance from the corner point on the line A to the line B is calculated, so that the corner points on the line A are ordered, and the starting point and the ending point on the line A are obtained. As shown in fig. 11, the start point and the end point of the straight line a and the start point and the end point of the straight line B are each positioned as shown in the figure.

In other embodiments, please refer to fig. 11, taking the straight line C in fig. 11 as an example, the distance from the corner point on the straight line D to the straight line C is calculated, and the corner points on the straight line D are ordered according to the distance, so as to obtain the start point and the end point on the straight line D. The image processing apparatus determines a start point and an end point on a line D according to a preset linear relationship, that is, a start point on a line to be ordered is a point with a maximum distance from one end of a reference line in the line to be ordered, an end point is a point with a maximum distance from the other end of the reference line in the line to be ordered, in this example, the start point on the line D is a point with a maximum distance from one end of a line C to the line D, and the end point is a point with a maximum distance from the other end of the line C to the line D. On the contrary, the distance from the corner point on the straight line C to the straight line D is calculated, so that the corner points on the straight line C are ordered, and the starting point and the ending point on the straight line C are obtained. As shown in fig. 11, the start and end points of the straight line C and the start and end points of the straight line D are each positioned as shown in the figure.

It should be understood that, in this embodiment, when calculating the start point and the end point on the straight line, the image processing apparatus may sort all the points on the straight line according to the distance from the point to the straight line and then store the sorted points, for example, the distances are stored in order from the large to the small. The algorithm adopted in calculating the starting point and the ending point is calculated according to the positions of the ordered angular points, and in order to reduce the time complexity, the angular point positions on each straight line are ordered in advance, so that the complexity of the subsequent algorithm is reduced, and the calculation of the starting point and the ending point is facilitated. If the sorting is not performed, the corner positions need to be sorted again when the end point and the start point are calculated later, so that the calculation workload is large and the calculation complexity is high.

The image processing equipment traverses the angular points on all the straight lines in the image to be processed according to the mode, sorts the angular points on each straight line, and finally obtains the starting point and the ending point on each straight line.

It is to be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," and the like are merely for convenience in describing and simplifying the description based on the orientation or positional relationship shown in the drawings, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and thus are not to be construed as limiting the application.

After the image processing device orders the corner points on each straight line, since the corner points on the same straight line may be corner points belonging to different objects, for example, a certain corner point in the image may be a corner point in the calibration area or a corner point in the background information. Therefore, it is necessary to further determine the specific judgment of the corner point.

For example, by calculating the distance between the angular points on each straight line, rough clustering is performed on the angular points with equal distance on the straight line, namely, firstly, a plurality of line segments on the straight line are determined through the angular points on the straight line, then, the angular points with equal distance in each line segment are clustered, so that all the straight line models with equal angular point distances are obtained, and all the angular points which do not meet the condition are removed.

For another example, the slope of the line segment formed by the corner points is calculated, and the line segments of the corner points with the slope in the same range are clustered. And finally, clustering the straight lines with equal or similar distances between the corner points, and clustering the straight lines where the corner point line segments with the slopes in the same range are located, so as to eliminate the corner points which do not belong to the clustering straight lines.

In this embodiment, the calibration board is a vertical calibration board shown in fig. 3, so the image processing apparatus can find, according to the characteristic attribute of the calibration board, a straight line combination of two side lines and a middle line, which meet the parallel and equidistant conditions, in all the straight line models screened out.

In some embodiments, the image processing apparatus first determines all midlines in the calibration plate that meet the requirements, including in particular: and screening all straight lines with equal slope for each straight line in the screened all straight line models, determining the straight lines with the same slope as a straight line combination, and determining the central line in each straight line combination. For example, the image processing apparatus first arbitrarily selects one straight line, then screens all straight lines having the same slope as the one straight line among all the straight line models, sets the three straight lines as one straight line combination if there are two straight lines having the same slope as the one straight line, and determines the middle straight line of the three straight lines as the center line.

The image processing apparatus screens out all combinations of all three straight lines satisfying the same slope as the center line among all the straight line models in the above manner. And then dividing the combined straight line into an edge line positioned on one side of the central line and an edge line positioned on the other side of the central line according to the position relation between the corner point on the edge line and the central line in each combination. And finally, screening out all possible three linear combinations according to the condition that the two side lines are equidistant from the central line, and clustering each linear combination.

S6, the image processing equipment determines a straight line combination of straight lines where the inner corners of the calibration plate are located.

In some embodiments, the image processing device processes all the screened straight line combinations, projects the corner points on the two side edges on the center line in each straight line combination, after the projection is completed, the corner points on the two side edges are possibly combined with the corner points on the middle center line, and then further judges the center line after the projection and the corner points are combined. Fig. 12 is a schematic diagram of a straight line combination selected, as shown in fig. 12, and the corner points on the left and right side lines in fig. 12 are projected onto the center line, so as to obtain the schematic diagram of a straight line combination shown in fig. 13.

It should be understood that in this embodiment, according to the actual independent parameters of the calibration plate, that is, the intersecting lines formed between the calibration grid sets are parallel and equidistant, and the distances between the inner corners on each straight line are equal, so that the distances between the two side lines and the center line in the straight line combination are equal to the distances between the corner points, and the parallel and equidistant corner points can be projected onto the center line only if the parallel and equidistant corner points are satisfied.

In some embodiments, since there may be missed detection of the corner points on the two side lines and the center line, the corner points on the two side lines need to be projected onto the center line, and then further screening processing is performed on the original corner points on the center line and the projected corner points.

The image processing device firstly calculates the angular point spacing a on the central line after projection merging, and then compares the angular point spacing a on the central line after projection merging with the angular point spacing a on the calculated projection merging by taking the line spacing of three lines in each linear combination as an estimated linear point spacing b based on the characteristic attribute of the calibration plate, namely that the distances between every two angular points on the central line are equal. As shown in fig. 14, in general, the values of a and b are consistent, and when the calculated value of a is obviously larger or smaller during calculation, the corner point is not included in the corner point on the calibration plate, and the projection line segment conforming to the length of the middle line segment of the calibration plate is screened according to the mode, so that the corner point on the outer side of the straight line, which is not included in the calibration plate, is eliminated. In the calculation process, if a plurality of combinations meeting the requirements exist, three straight line combinations with the maximum equidistant line segments are selected from the combinations, and the three straight line combinations with the maximum equidistant line segments are the straight line combinations of the inner angle point parts of the calibration plate.

It should be understood that, in the above-mentioned straight line combination of the corner portions in the calibration plate, all the corner points detected in the straight line combination are the first partial corner points.

S7, the image processing equipment corresponds the inner corner points on the straight line combination with the actual inner corner points in the marking area in the marking plate, and detects all the inner corner points in the straight line combination.

In some embodiments, after the image processing apparatus screens out the straight line combinations corresponding to the calibration boards, first, a start point and an end point on the straight line are obtained, where the start point and the end point are points detected by the manner of sorting the corner points of the straight line. And then combining the principle that the distances between the central line and the two side lines and the point distances between the corner points on each straight line are equal, namely, the corner points in the marked area in the marked plate can be in one-to-one correspondence, and all the undetected inner corner points in the straight line combination are detected, wherein the undetected inner corner points are the corner points of the second part.

According to the technical scheme provided by the embodiment, in the straight line combination of the calibration plate, as long as 8 inner corner points can be detected in each group of straight lines, the inner corner points of the calibration plate can be successfully detected. Specifically, the 8 inner corner points include: at least two inner angle points on each straight line in the three straight line combinations are used for confirming one straight line, at least four inner angle points on the straight line of the center line are needed to be detected for confirming the starting point and the end point of the center line, the four inner angle points are the two angle points for confirming the center line, the starting point and the end point, and after the starting point and the end point of the center line are confirmed, the inner angle points on all calibration plates on the three straight lines can be detected according to the size of an actual calibration plate. It can be seen that at least two inner corner points need to be detected on two side lines, and the two inner corner points, the starting point and the ending point of the straight line are confirmed on the central line, and the total is four points. Therefore, in the three-line combination, at least 8 inner corner points need to be detected, and detection of all the inner corner points can be realized.

In this embodiment, the manner of detecting the above straight line combinations and 8 inner corner points is described in the above embodiment, so, by the method provided in this embodiment, as shown in fig. 15, through the correspondence between the first part of corner points and the second part of corner points, the algorithm can still extract the corner points of the calibration board and find the corresponding positions under the condition that there is missing detection of the corner points.

It should be understood that the sequence number of each step in the foregoing embodiment does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application.

Fig. 16 is a schematic diagram of an angular point detection device according to an embodiment of the present application, as shown in fig. 16, where the device includes:

The device comprises an acquisition unit, a detection unit and a detection unit, wherein the acquisition unit is used for acquiring an image to be detected, the image to be detected comprises a calibration plate, the calibration plate comprises a plurality of inner corner points, and a preset arrangement relation exists between a first part of corner points and a second part of corner points in the plurality of inner corner points;

The identification unit is used for identifying a first part of corner points in the image to be detected;

the determining unit is used for determining a second part of corner points in the image to be detected by combining the first part of corner points according to a preset arrangement relation.

Fig. 17 is a schematic diagram of a corner detection apparatus according to an embodiment of the present application. As shown in fig. 17, the corner detection apparatus 4 of this embodiment includes: a processor 40, a memory 41 and a computer program 42, such as a corner detection program, stored in said memory 41 and executable on said processor 40. The steps of the respective corner detection method embodiments described above are implemented when the processor 40 executes the computer program 42. Or the processor 40, when executing the computer program 42, performs the functions of the modules/units of the apparatus embodiments described above.

Illustratively, the computer program 42 may be partitioned into one or more modules/units that are stored in the memory 41 and executed by the processor 40 to complete the present application. The one or more modules/units may be a series of computer program instruction segments capable of performing a specific function for describing the execution of the computer program 42 in the corner detection device 4.

The corner detection device 4 may be a computing device such as a tablet computer, a desktop computer, a notebook computer, a palm computer, a cloud server, etc. The corner detection device may include, but is not limited to, a processor 40, a memory 41. It will be appreciated by a person skilled in the art that fig. 17 is only an example of the corner detection device 4 and does not constitute a limitation of the corner detection device 4, and may comprise more or less components than shown, or may be combined with certain components, or different components, e.g. the corner detection device may further comprise an input-output device, a network access device, a bus, etc.

The Processor 40 may be a central processing unit (Central Processing Unit, CPU), other general purpose Processor, digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 41 may be an internal storage unit of the corner detection device 4, for example a hard disk or a memory of the corner detection device 4. The memory 41 may also be an external storage device of the corner detection device 4, such as a plug-in hard disk, a smart memory card (SMART MEDIA CARD, SMC), a Secure Digital (SD) card, a flash memory card (FLASH CARD) or the like, which are provided on the corner detection device 4. Further, the memory 41 may also include both an internal memory unit and an external memory device of the corner detection device 4. The memory 41 is used for storing the computer program as well as other programs and data required by the corner detection device. The memory 41 may also be used for temporarily storing data that has been output or is to be output.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of the functional units and modules is illustrated, and in practical application, the above-described functional distribution may be performed by different functional units and modules according to needs, i.e. the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-described functions. The functional units and modules in the embodiment may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit, where the integrated units may be implemented in a form of hardware or a form of a software functional unit. In addition, the specific names of the functional units and modules are only for distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working process of the units and modules in the above system may refer to the corresponding process in the foregoing method embodiment, which is not described herein again.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and in part, not described or illustrated in any particular embodiment, reference is made to the related descriptions of other embodiments.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other manners. For example, the apparatus/terminal device embodiments described above are merely illustrative, e.g., the division of the modules or units is merely a logical function division, and there may be additional divisions in actual implementation, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via interfaces, devices or units, which may be in electrical, mechanical or other forms.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated modules/units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on this understanding, the present application may also be implemented by implementing all or part of the procedures in the methods of the above embodiments, and the computer program may be stored in a computer readable storage medium, where the computer program when executed by a processor may implement the steps of the respective method embodiments. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth.

The above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application.

Claims

1. A corner detection method, comprising:

Obtaining an image to be detected, wherein the image to be detected comprises a calibration plate, the calibration plate comprises a plurality of inner corner points, and a preset arrangement relation exists between a first part of corner points and a second part of corner points in the plurality of inner corner points;

identifying the first partial corner points in the image to be detected;

According to the preset arrangement relation, the second partial corner points in the image to be detected are determined by combining the first partial corner points;

The calibration plate comprises M groups of calibration grids, M-1 common intersecting lines exist among the M groups of calibration grids, each intersecting line comprises an edge line and a central line, the central line is the intersecting line at the middle position in the M-1 intersecting lines, and the edge lines are the intersecting lines except the central line in the M-1 intersecting lines;

the first part of corner points comprise any two inner corner points on each side line and inner corner points at two ends on the center line, wherein M is more than 2 and is an even number;

the second part of corner points are other inner corner points except the first part of corner points in the calibration plate;

In each group of calibration grids, the calibration grids are adjacent or spaced along a first direction, the M groups of calibration grids are sequentially arranged along a second direction by taking the groups as units, and the second direction is mutually perpendicular to the first direction; the calibration grids are the same in size and are square;

identifying the first partial corner points in the image to be detected comprises the following steps:

Identifying all corner points in the image to be detected;

determining a straight line model according to the positions of every two corner points;

Screening M-1 item mark straight line models comprising the inner angle points from all the straight line models;

And identifying the first part of corner points from the M-1 item mark straight line model.

2. The method of claim 1, wherein screening out M-1 item mark line models including the interior corner points from all the line models comprises:

screening all the straight line models which accord with the preset slope conditions and the corner number conditions from all the straight line models;

performing linear combination clustering on all the screened linear models in a mode of similar slope;

and screening all the clustered straight line combinations according to the distance between the inner corner points, and determining the M-1 item standard straight line model.

3. The method of claim 2, wherein the slope condition is determined based on a positional relationship of all lines in the line model to a first direction;

And the corner number condition is determined according to the number of actual inner corner points on each intersection line in the calibration area.

4. The method according to claim 2, wherein the linear combination clustering of all the linear models screened according to the similar slope comprises:

Determining linear models with similar slopes in all the linear models, and generating a plurality of linear combinations;

determining a midline of each of the combinations of lines;

Determining all straight line combinations with similar slopes;

According to the preset position relation between the corner points on the side lines and the central line, the side lines in all the linear combinations are screened out from the linear combinations with similar slopes;

And clustering the line combinations of the midline and the side line of each line combination.

5. The method according to claim 1, wherein determining the second partial corner in the image to be detected in combination with the first partial corner according to the preset arrangement relation comprises:

Determining the position of the first partial corner points;

And determining the second partial corner points in the image to be detected according to the group number and the setting position of the calibration cells in the calibration plate.

6. A corner detection device, the device comprising:

The identification unit is used for identifying the first partial corner points in the image to be detected;

The determining unit is used for determining the second part of corner points in the image to be detected by combining the first part of corner points according to the preset arrangement relation;

Identifying all corner points in the image to be detected;

7. The calibration plate is characterized by comprising a substrate and a calibration area on the substrate;

the calibration area comprises a plurality of inner corner points, and a preset arrangement relation exists between a first part of corner points and a second part of corner points in the plurality of inner corner points; the preset arrangement relation is used for determining the second part angular point when the electronic equipment identifies the first part angular point but does not identify the second part angular point;

When the corner detection is performed on the calibration plate, the electronic equipment adopts the corner detection method according to any one of claims 1-5.