CN114979469B - Camera mechanical error calibration method and system based on machine vision comparison - Google Patents
Camera mechanical error calibration method and system based on machine vision comparison Download PDFInfo
- Publication number
- CN114979469B CN114979469B CN202210497583.8A CN202210497583A CN114979469B CN 114979469 B CN114979469 B CN 114979469B CN 202210497583 A CN202210497583 A CN 202210497583A CN 114979469 B CN114979469 B CN 114979469B
- Authority
- CN
- China
- Prior art keywords
- camera
- markers
- offset
- offset vector
- reference frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/695—Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
Abstract
The invention discloses a camera mechanical error calibration method based on machine vision comparison, which comprises the following steps: (1) Enabling the camera to be corrected to execute a command of positioning to a correction position, and obtaining a current frame; (2) Detecting all possible markers in the frame, and matching the plurality of markers according to the plurality of markers of the reference frame; (3) Comparing the difference between the coordinates of a plurality of markers in the current frame and the coordinates of corresponding markers in a preset reference frame, and determining a camera offset vector; when the offset vector of the camera is smaller than a preset threshold value or the occurrence of calibration oscillation is judged, calibration is completed, and error calibration is finished; otherwise, entering the step (4); (4) And determining the number of motion compensation steps according to the camera offset vector, correcting the mechanical position of the camera according to the number of motion compensation steps, and repeating the steps. The invention can automatically calibrate the orientation of the camera without human intervention.
Description
Technical Field
The invention belongs to the field of intelligent manufacturing, and particularly relates to a camera mechanical error calibration method and system based on machine vision comparison.
Background
Machine vision is widely used at present, and plays a role in supporting industry business and high-efficiency industry flow in the intelligent manufacturing industry. The application of machine vision is that video data are collected by a spherical camera and reasoning analysis is carried out.
The spherical camera has the advantage that the shooting angle and the focal length can be flexibly adjusted, so that the spherical camera can be flexibly installed in practical application, has small construction amount and is convenient to maintain, and therefore, the spherical camera is widely popularized and built in the application field of machine vision.
However, in the long-term use of the spherical camera, mechanical errors caused by the rotation direction are accumulated, and finally, the shooting picture of the spherical camera is obviously deviated. In a large number of application fields needing precise visual positioning and identification, such as identification of meter readings of equipment, identification of states of densely arranged objects, identification of information of specific positions on the surfaces of the objects, and the like, the offset of a shooting picture can cause inaccurate identification and even incorrect identification, and influence the normal functions of business application.
At present, the phenomenon of inaccurate identification often occurs when the mechanical error accumulation of the camera is manually corrected, so that the normal working process is disturbed.
Disclosure of Invention
Aiming at the defects or improvement demands of the prior art, the invention provides a camera mechanical error calibration method and a camera mechanical error calibration system based on machine vision comparison, which aim to infer a camera bias vector based on position detection of a marker by machine vision and calculate compensation, so that the camera position is reversely adjusted to carry out error correction, and further solve the technical problem that a spherical camera is offset in long-term use in the fields of precise vision positioning and identification application, so that an identification result is inaccurate or even incorrect.
To achieve the above object, according to one aspect of the present invention, there is provided a camera mechanical error calibration method based on machine vision alignment, comprising the steps of:
(1) Enabling the camera to be corrected to execute a command of positioning to a correction position, and acquiring a picture captured by the camera in the state as a current frame;
(2) Detecting all possible markers in the current frame obtained in the step (1), matching the markers according to a plurality of markers of a reference frame with preset correction positions, and determining the successfully matched markers as the markers in the current frame to acquire the coordinates of the successfully matched markers in the current frame;
(3) Comparing the difference between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame, and determining a camera offset vector according to the difference between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame; when the offset vector of the camera is smaller than a preset threshold value or the occurrence of calibration oscillation is judged, calibration is completed, and error calibration is finished; otherwise, entering the step (4); the preset is the maximum allowable offset.
(4) Determining a motion compensation step number according to the camera offset vector obtained in the step (3), correcting the mechanical position of the camera according to the motion compensation step number, and repeating the steps (1) to (3).
Preferably, the number of the markers in the camera mechanical error calibration method based on machine vision alignment is at least 3, and is generally less than or equal to 6.
Preferably, the method for calibrating mechanical error of camera based on machine vision alignment, wherein the marker is selected from targets obtained by detection of target detection algorithm, preferably determined according to the following method:
s1, acquiring a picture shot when a camera moves to a specified position as a reference frame;
s2, performing target detection on the reference frame in the step S1 by using a target detection algorithm, and taking all detected targets as candidate markers;
s3, selecting a preset number of candidate markers from the candidate markers obtained in the step S2 to serve as markers of the reference frame.
Preferably, the area of the marker accounts for 1/16 to 1/48 of the area of the reference frame picture, and the shape and/or the color of the marker are preferably unique.
Preferably, the method for calibrating mechanical error of camera based on machine vision alignment, wherein the step (2) specifically includes:
(2-1) scanning the current frame picture according to the characteristics of the markers of the reference frame to obtain all possible markers;
preferably, the marker of the reference frame is a target object detected by adopting a specific target detection algorithm, and the scanning of the current frame picture according to the characteristics of the marker of the reference frame is specifically performed by adopting the target detection algorithm.
(2-2) for each of the plurality of markers of the reference frame, performing image similarity comparison among the possible markers obtained in the step (2-1), respectively, thereby matching to obtain an image of the marker in the current frame in the reference frame;
(2-3) determining the coordinates of the plurality of markers in the current frame obtained in the step (2-2) according to the same method as the determination method of the coordinates of the markers in the reference frame, respectively.
Preferably, in the method for calibrating mechanical errors of a camera based on machine vision comparison, the judging that the correction oscillation occurs and the calibration is completed by the camera offset vector in the step (3) is specifically as follows:
judging whether the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds a preset threshold value, when the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds the preset threshold value, judging that calibration oscillation occurs, selecting the smaller of the current camera offset vector and the last-judged camera offset vector as a result, and directly ending error calibration when the last-judged camera offset vector is smaller; otherwise, when the current camera offset vector is smaller, determining the motion compensation step number according to the current camera offset vector, correcting the mechanical position of the camera according to the motion compensation step number, and ending error calibration.
Preferably, the camera offset vector is preferably an average value of offset vectors of markers in all current frames; the offset vector of the marker in the current frame is a vector taking the coordinate of the marker in the reference frame as a starting point and taking the coordinate of the marker in the current frame as an end point.
Preferably, the machine vision alignment-based camera mechanical error calibration method comprises the following steps:
(4-1) vector decomposition is carried out on the offset vector of the camera according to the movement directions supported by the camera, so that the offset in each movement direction of the camera is obtained;
(4-2) taking the ratio of the offset in each moving direction to the moving step length of the camera as an integer part according to the offset in each moving direction of the camera and the moving step length of the camera in the corresponding moving direction obtained in the step (4-1) as the number of the moving compensation steps in the moving direction;
(4-3) adjusting the camera to move in each direction of movement by a corresponding number of motion compensation steps in a direction opposite to the offset, thereby correcting the mechanical position of the camera.
According to another aspect of the present invention, there is provided a camera mechanical error calibration system based on machine vision alignment, comprising:
the device comprises a current frame acquisition module, a marker detection module, a camera offset vector calculation module and a compensation reset module;
the current frame acquisition module is used for enabling the camera to be corrected to execute a command for positioning to a correction position, acquiring a picture captured by the camera in the state, taking the picture as a current frame, and submitting the current frame to the marker detection module;
the marker detection module is used for storing a preset reference frame, detecting all possible markers in the current frame, matching the markers in the reference frame according to the preset correction position, determining the successfully matched markers as the markers in the current frame, acquiring coordinates of the successfully matched markers in the current frame, and submitting the successfully matched markers to the camera offset calculation module;
the camera offset calculation module is used for storing the coordinates of the markers in the reference frame, comparing the differences between the coordinates of the markers in the current frame and the coordinates of the corresponding markers in the preset reference frame, determining a camera offset vector according to the differences between the coordinates of the markers in the current frame and the coordinates of the corresponding markers in the preset reference frame, and submitting the camera offset vector to the compensation reset module;
the compensation resetting module is used for judging whether error calibration is finished, and finishing error calibration when the offset vector of the camera is smaller than a preset threshold value or calibration oscillation is judged to occur and calibration is finished; otherwise, determining the motion compensation step number according to the camera offset vector, and correcting the mechanical position of the camera according to the motion compensation step number.
According to another aspect of the invention, there is provided the steps of implementing the camera mechanical error calibration method based on machine vision alignment provided by the invention when the computer program is executed by a processor.
In general, the above technical solutions conceived by the present invention, compared with the prior art, enable the following beneficial effects to be obtained:
the invention can detect the specific marker or extract the specific characteristic information of the monitoring picture of the camera through the machine vision technology; comparing the information of the standard orientation and the information of the orientation which is possibly shifted at the time to confirm whether the orientation of the camera is shifted, in what direction and how much; and controlling the camera to adjust towards the opposite direction of the offset based on the information, performing offset judgment again according to the process after the adjustment, and completing the calibration of the camera when the offset is smaller than the offset error allowed by the application service. The invention can automatically calibrate the orientation of the camera without human intervention, liberates manpower, provides important support for a large number of unmanned guard application scenes which are originally analyzed and identified by using a machine vision technology to monitor images, reduces operation and maintenance cost, provides operation reliability and operation efficiency, and provides guarantee for the automatic operation of a production environment for large-scale application of machine vision.
Drawings
FIG. 1 is a schematic flow chart of a camera mechanical error calibration method based on machine vision comparison provided by the invention;
fig. 2 is a reference frame photographed by a camera according to an embodiment of the present invention;
fig. 3 is a current frame photographed by a camera according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of detecting a reference frame marker according to an embodiment of the present invention;
fig. 5 is a schematic diagram of detecting a current frame marker according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. In addition, the technical features of the embodiments of the present invention described below may be combined with each other as long as they do not collide with each other.
The invention provides a camera mechanical error calibration method based on machine vision comparison, which is shown in fig. 1 and comprises the following steps:
(1) Enabling the camera to be corrected to execute a command of positioning to a correction position, and acquiring a picture captured by the camera in the state as a current frame;
(2) Detecting all possible markers in the current frame obtained in the step (1), matching the markers according to a plurality of markers of a reference frame with preset correction positions, and determining the successfully matched markers as the markers in the current frame to acquire the coordinates of the successfully matched markers in the current frame; the method specifically comprises the following substeps:
(2-1) scanning the current frame picture according to the characteristics of the markers of the reference frame to obtain all possible markers;
preferably, the marker of the reference frame is a target object detected by adopting a specific target detection algorithm, and the scanning of the current frame picture according to the characteristics of the marker of the reference frame is specifically performed by adopting the target detection algorithm.
(2-2) for each of the plurality of markers of the reference frame, performing image similarity comparison among the possible markers obtained in the step (2-1), respectively, thereby matching to obtain an image of the marker in the current frame in the reference frame;
(2-3) determining the coordinates of the plurality of markers in the current frame obtained in the step (2-2) according to the same method as the determination method of the coordinates of the markers in the reference frame, respectively.
The number and the object of the markers are the key to determine the speed and effect of the invention;
the number of markers, as shown in FIG. 4, is at least 3 and generally 6 or less. Because the camera is rotated in positioning, the acquired picture is a projection plane of spatial stereo, that is, the picture acquired by the camera is actually three-dimensional projection of the real world on a two-dimensional plane, and visual deviation exists, at least 3 points are needed to determine the plane position, and thus the error offset is determined. In the case of too few markers, it may happen that the camera has actually been shifted, but the selected marker still can coincide well with the corresponding marker in the reference frame, so that the shift situation of the camera cannot be found effectively, and calibration and reset cannot be performed. In general, the mechanical movement accumulation error does not excessively deviate from the reference position, so that 4 or more markers are provided as alternatives for matching, and the problem that the number of matched markers is less than three can be avoided. If the number of matched markers is less than three, this indicates that the deviation is too great and manual correction is required. On the other hand, the calibration and reset of the camera have errors and cannot be reset to the extent of no difference from the initial picture, so that under the condition of excessive number of markers, the fact that the actual errors are within a specified error range and the accumulated error value is large can occur, the camera makes longer reset attempts, and the automatic calibration and reset efficiency of the camera is affected. If the number of the markers is more than 6, the position deviation requirement of more markers needs to be met simultaneously when the error offset is calculated, so that larger calculated amount is brought, and the calculated amount of 7 markers is about 2.3 times that of 6 markers. Based on verification tests, when the number of the markers is 3 to 6, good offset comparison effect and efficient calibration reset efficiency can be achieved, and the calibration aging is as follows.
Preferably the marker is selected from the group consisting of targets detected by a target detection algorithm, and is determined as follows:
s1, acquiring a picture shot when a camera moves to a specified position as a reference frame;
s2, performing target detection on the reference frame in the step S1 by using a target detection algorithm, and taking all detected targets as candidate markers;
s3, selecting a preset number of candidate markers from the candidate markers obtained in the step S2 to serve as markers of a reference frame;
in the method, the detected markers are selected without directly marking the positions by a user, so that the markers can be matched in the automatic detection process, the influence of subjective factors is avoided, the markers are automatically detected by a program, and the matching is performed.
Preferably, the area of the marker occupies 1/16 to 1/48 of the area of the reference frame picture. The oversized marker can not be shot completely after the offset of the camera, so that the marker is incomplete and offset comparison can not be performed; the too small marker has less picture information, and after the camera deflects, the camera is easy to generate larger deformation, and the too small marker can not be detected, so that the offset comparison can not be performed.
Preferably, the shape and/or the color of the markers are unique, so that different markers have obvious differentiation, and the matching of the same marker image between the current frame and the reference frame is effectively realized. If the discrimination of the markers is not obvious, the target detection may not be able to effectively distinguish the markers, so that the corresponding matching of the marker image of the current frame and the marker image of the reference marker is not able to be realized, and the comparison of the position relations between the markers is not able to be realized, so that the calibration and the reset are affected.
(3) Comparing the difference between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame, and determining a camera offset vector according to the difference between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame; when the offset vector of the camera is smaller than a preset threshold value or the occurrence of calibration oscillation is judged, calibration is completed, and error calibration is finished; otherwise, entering the step (4); the preset is the maximum allowable offset.
The camera offset vector judges that correction oscillation occurs and completes calibration specifically comprises the following steps:
judging whether the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds a preset threshold value, when the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds the preset threshold value, judging that calibration oscillation occurs, selecting the smaller of the current camera offset vector and the last-judged camera offset vector as a result, and directly ending error calibration when the last-judged camera offset vector is smaller; otherwise, when the offset vector of the camera is smaller, determining the number of motion compensation steps according to the offset vector of the camera, correcting the mechanical position of the camera according to the number of motion compensation steps, and ending error calibration;
the camera offset vector is preferably the average value of offset vectors of marks in all current frames; the offset vector of the marker in the current frame is a vector starting from the coordinate of the marker in the reference frame and ending from the coordinate of the marker in the current frame, as shown in fig. 5.
(4) Determining a motion compensation step number according to the camera offset vector obtained in the step (3), correcting the mechanical position of the camera according to the motion compensation step number, and repeating the steps (1) to (3).
The mechanical position of the camera is preferably corrected as follows:
(4-1) vector decomposition is carried out on the offset vector of the camera according to the movement directions supported by the camera, so that the offset in each movement direction of the camera is obtained;
(4-2) taking the ratio of the offset in each moving direction to the moving step length of the camera as an integer part according to the offset in each moving direction of the camera and the moving step length of the camera in the corresponding moving direction obtained in the step (4-1) as the number of the moving compensation steps in the moving direction;
(4-3) adjusting the camera to move in each direction of movement by a corresponding number of motion compensation steps in a direction opposite to the offset, thereby correcting the mechanical position of the camera.
The following are examples:
the camera mechanical error calibration method based on machine vision comparison is applied to precise vision positioning and recognition, such as instrument recognition, and is automatically and regularly operated by a system after the initialization flow is stably operated, so as to confirm the offset of the camera and calibrate and reset.
The method comprises the following steps:
(1) Causing the camera to be corrected to execute a command of positioning to a correction position, and acquiring a picture captured by the camera in the state as a current frame, as shown in fig. 3;
(2) Detecting all possible markers in the current frame obtained in the step (1), matching the markers according to a plurality of markers of a reference frame with preset correction positions, and determining the successfully matched markers as the markers in the current frame to acquire the coordinates of the successfully matched markers in the current frame;
the initialization flow is responsible for recording an initial monitoring picture of the camera and the positions of markers in the picture, wherein the markers are selected from targets obtained by detection of a target detection algorithm, and the markers are determined according to the following method:
s1, acquiring a picture shot when the camera moves to a specified position as a reference frame, wherein the reference frame is shown in fig. 2.
S2, performing a target detection algorithm on the reference frame in the step S1, wherein the target detection algorithm can be selected according to specific shot image characteristics, and comprises, but is not limited to, faster R-CNN (Ren S, he K, girshick R, et al Faster R-CNN: towards real-time object detection with region proposal networks [ J ]), yolo (Heimer R Z, myrseth K O R, schoenle R.Yolo: mortality beliefs and household finance puzzles [ J ]), SSD (Liu W, anguelov D, erhan D, et al Ssd: single shot multibox detector [ C ]), and performing target detection by using Yolo in the embodiment, wherein all detected targets are used as alternative markers;
s3, selecting a preset number of candidate markers from the candidate markers obtained in the step S2 to serve as markers of a reference frame;
the recorded reference frame and marker position information can play a role of reference in the system stable operation flow, and are used for judging whether the video camera is offset, what direction of offset occurs and how much offset occurs. In the initialization flow, the user designates an initial monitoring screen as a "reference frame".
In the stable operation flow, the system needs to automatically judge whether the camera is deviated or not; therefore, in the stable operation flow, the system can only position the marker in the real-time picture of the camera through an automatic means, namely a target detection algorithm, but cannot be directly marked by a user; in order to ensure that the marker positioning standards are consistent, so that the current frame and the reference frame have uniform positioning in the camera calibration process to ensure successful calibration, a target detection algorithm is used for positioning the markers, and then a user selects the detected markers without directly marking the marker positions by the user.
Based on verification tests, when the number of the markers is 3 to 6, good offset comparison effect and efficient calibration and reset efficiency can be achieved, as shown in the following table.
| Number of markers | 3 | 4 | 5 | 6 | 7 |
| Calibration time (seconds) | 10 | 12 | 14 | 17 | 40 |
| Minimum number of detections | 2 | 3 | 3 | 4 | 5 |
The size of the marker should be proper, not too large, nor too small; the oversized marker can not be shot completely after the offset of the camera, so that the marker is incomplete and offset comparison can not be performed; the too small marker has less picture information, and after the camera deflects, the camera is easy to generate larger deformation, and the too small marker can not be detected, so that the offset comparison can not be performed. Proved by verification, the size of the marker occupies 1/16 to 1/48 of the picture area; the test results are shown in the following table:
in the same picture, different markers have obvious differences in aspects of morphology, color and the like; if the discrimination of the markers is not large, the target detection may not be able to effectively distinguish the markers, so that the correspondence between the current markers and the reference markers is not realized, and the comparison of the position relationship between the current markers and the reference markers is not realized, so that the calibration and the reset are affected.
(2-1) scanning the current frame picture according to the characteristics of the markers of the reference frame to obtain all possible markers;
preferably, the marker of the reference frame is a target object detected by adopting a specific target detection algorithm, and the scanning of the current frame picture according to the characteristics of the marker of the reference frame is specifically performed by adopting the target detection algorithm.
(2-2) for each of the plurality of markers of the reference frame, comparing the possible marker image similarities obtained in step (2-1), respectively, to match the images of the markers in the current frame in the reference frame; when the feature of the marker is very obvious, for example, the marker is the unique red color block in the picture, the similarity comparison of the marker images can be simplified into color matching, and the similarity is similar as shape matching and the like.
(2-3) determining the coordinates of the plurality of markers in the current frame obtained in the step (2-2) according to the same method as the determination method of the coordinates of the markers in the reference frame, respectively.
(3) Comparing the difference between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame, and determining a camera offset vector according to the difference between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame; when the offset vector of the camera is smaller than a preset threshold value or the occurrence of calibration oscillation is judged, the calibration is completed, and the error calibration is ended; otherwise, entering the step (4);
the camera offset vector judges that correction oscillation occurs specifically as follows:
judging whether the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds a preset threshold value, when the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds the preset threshold value, judging that calibration oscillation occurs, selecting the smaller of the current camera offset vector and the last-judged camera offset vector as a result, and directly ending error calibration when the last-judged camera offset vector is smaller; otherwise, when the offset vector of the camera is smaller, determining the number of motion compensation steps according to the offset vector of the camera, correcting the mechanical position of the camera according to the number of motion compensation steps, and ending error calibration;
in the process of calibrating and resetting the camera, if the minimum step length which can be supported by the camera is not small enough and the offset generated by the camera is larger than the allowable minimum offset but smaller than the minimum step length which can be supported by the camera, when the camera performs calibrating and resetting, a new offset can be generated by one time adjustment of the camera towards the opposite direction of the original offset, and if the offset of the new offset is still larger than the allowable minimum offset error, the calibration of the camera can be trapped into oscillation which cannot be stopped at the moment. In order to solve the above problems, the system determines whether an oscillation occurs by comparing the change of the offset direction in the continuous multiple adjustments, and if the oscillation occurs, uses the minimum offset (the minimum offset refers to each adjustment performed when the oscillation occurs, and records the offset after the current adjustment, when the offset after a certain adjustment is greater than the previous offset, the camera orientation and focal length of the last offset are regarded as the minimum offset) as the result of the calibration reset, and informs the user that the user can determine the recognition result under the condition of the result of the calibration reset, and if the recognition result is not affected, uses the result of the calibration reset as the final confirmation result of the calibration reset.
The camera offset vector is preferably the average value of offset vectors of marks in all current frames; the offset vector of the marker in the current frame is a vector taking the coordinate of the marker in the reference frame as a starting point and taking the coordinate of the marker in the current frame as an end point.
The offset vector allows the system to determine whether the camera to be calibrated is offset, what direction of offset is occurring, and how much offset is occurring. And comparing the current frame marker coordinates with corresponding marker coordinates in the reference frame, and calculating an offset vector. Because the offset vector has a plurality of markers, each marker has a correspondence relationship of a plurality of points, a single offset vector does not exist, and the calculation processing method of the offset vector is described herein:
the method for calculating and processing a plurality of offset vectors formed by a plurality of points in a single marker comprises the following steps:
in the target detection, a single marker is represented by a target detection algorithm in the form of a rectangular frame, and the rectangular frame has four vertexes, so that the four points can be used for representing the position of one marker. The same marker can form 4 offset vectors at the positions of four points in the current picture and the positions of four points in the reference frame picture; because the same marker does not have larger displacement on the premise of regularly calibrating the camera, and because the marker has a certain size, the marker does not have larger deformation, so that four offset vectors are almost consistent, the system can randomly select one offset vector or average the four offset vectors for use, and the offset vector of a single marker is obtained.
The method for calculating the offset vector of the camera is characterized in that the following processing is carried out on a plurality of marker offset vectors in a single picture:
the offset vectors of the markers in a single picture may have obvious deviation, so that the method of randomly selecting one offset vector as the offset vector of the whole picture is not applicable any more, but the basic directions of the offset vectors of the markers in the same picture are consistent, and the offset amounts may have differences, so that the system takes the average value of the offset vector values of all the markers as the offset vector of the camera. Therefore, the offset is possibly smaller or slightly larger than the actual picture offset, and errors exist between the offset direction and the actual picture offset direction, but the offset comparison and calibration reset processes are performed for a plurality of times, so that errors caused by taking an average offset vector are eliminated, and the method is effective as old.
(4) Determining a motion compensation step number according to the camera offset vector obtained in the step (3), correcting the mechanical position of the camera according to the motion compensation step number, and repeating the steps (1) to (3).
The mechanical position of the camera is corrected as follows:
(4-1) vector decomposition is carried out on the offset vector of the camera according to the movement directions supported by the camera, so that the offset in each movement direction of the camera is obtained;
and (3) calibrating and resetting the camera based on the offset vector of the picture camera obtained by offset comparison. The system splits the offset vector into two directions, the present embodiment uses offset scalar values in the horizontal and vertical directions.
(4-2) taking the ratio of the offset in each moving direction to the moving step length of the camera as an integer part according to the offset in each moving direction of the camera and the moving step length of the camera in the corresponding moving direction obtained in the step (4-1) as the number of the moving compensation steps in the moving direction;
the two offset scalars in the horizontal direction and the vertical direction are respectively divided into the submultiple, and the motion compensation step number is determined by taking the minimum step length of the single direction adjustment of the camera as a unit.
(4-3) adjusting the camera to move in each direction of movement by a corresponding number of motion compensation steps in a direction opposite to the offset, thereby correcting the mechanical position of the camera.
One adjustment of the camera is performed according to the length of the two offset scalar products and the opposite direction.
The following is exemplified here:
the lower left vertex of the picture is taken as an origin, the right and the upper are taken as positive directions, and the pixel is taken as a coordinate unit. Assuming that the offset vector of the current picture is (-200,400), the offset vector is decomposed into-200 and 400 two offset scalars in the horizontal and vertical directions, and then the two offset scalars are approximately divided into-1 and 2. Therefore, the camera is rightwards adjusted by 1 minimum step of camera orientation adjustment, and 2 minimum steps of camera orientation adjustment are downwards adjusted, so that the calibration and the reset can be completed.
And repeating the steps, continuously adjusting until the length of the offset vector of the camera is smaller than the allowable offset error, finishing the automatic calibration and reset of the camera, storing the current orientation in the camera, and covering the previously set orientation.
The system will perform multiple offset comparison and calibration reset procedures to ensure that the camera is eventually successfully reset.
In the process of calibrating and resetting the camera, if the minimum step length which can be supported by the camera is not small enough and the offset generated by the camera is larger than the allowable minimum offset but smaller than the minimum step length which can be supported by the camera, when the camera performs calibrating and resetting, a new offset can be generated by one time adjustment of the camera towards the opposite direction of the original offset, and if the offset of the new offset is still larger than the allowable minimum offset error, the calibration of the camera can be trapped into oscillation which cannot be stopped at the moment.
The need for human intervention to resolve the concussion is due to the fact that the adjustment step length of the camera is larger than the allowed offset error, namely the adjustment precision is smaller than the required precision, at the moment, the system adjusts the camera to the minimum offset error which can be adjusted, and then, the user performs one-time service identification. If the identification result is correct, the allowable offset error is set to be too small, the user can adjust the value, and the automatic calibration function can work normally subsequently; if the recognition result is wrong, the camera adjustment step length is too large, and the user needs to perform manual calibration.
The minimum adjustment step length of the cameras sold in the market is small enough, and the vibration condition does not occur most of the time.
It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the invention and is not intended to limit the invention, but any modifications, equivalents, improvements or alternatives falling within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (9)
1. The camera mechanical error calibration method based on machine vision comparison is characterized by comprising the following steps of:
(1) Enabling the camera to be corrected to execute a command of positioning to a correction position, and acquiring a picture captured by the camera in the state as a current frame;
(2) Detecting all possible markers in the current frame obtained in the step (1), matching the markers according to a plurality of markers of a reference frame with preset correction positions, determining the successfully matched markers as the markers in the current frame, and acquiring coordinates of the successfully matched markers in the current frame;
(3) Comparing the difference between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame, and determining a camera offset vector according to the difference between the coordinates of the plurality of markers in the current frame picture and the reference frame; when the offset vector of the camera is smaller than a preset threshold value or the occurrence of calibration oscillation is judged, calibration is completed, and error calibration is finished; otherwise, entering the step (4);
(4) Determining a motion compensation step number according to the camera offset vector obtained in the step (3), correcting the mechanical position of the camera according to the motion compensation step number, and repeating the steps (1) to (3);
the step (3) of judging that the calibration oscillation occurs and completing the calibration specifically comprises the following steps:
judging whether the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds a preset threshold value, when the included angle between the current camera offset vector and the direction of the last-judged camera offset vector exceeds the preset threshold value, judging that calibration oscillation occurs, selecting the smaller of the current camera offset vector and the last-judged camera offset vector as a result, and directly ending error calibration when the last-judged camera offset vector is smaller; otherwise, when the current camera offset vector is smaller, determining the motion compensation step number according to the current camera offset vector, correcting the mechanical position of the camera according to the motion compensation step number, and ending error calibration.
2. The machine vision alignment-based camera mechanical error calibration method of claim 1, wherein the number of markers is at least 3 and less than or equal to 6.
3. The machine vision alignment-based camera mechanical error calibration method of claim 1, wherein the markers are determined as follows:
s1, acquiring a picture shot when a camera moves to a specified position as a reference frame;
s2, performing target detection on the reference frame in the step S1 by adopting a target detection algorithm, and taking all detected targets as candidate markers;
s3, selecting a preset number of candidate markers from the candidate markers obtained in the step S2 to serve as markers of the reference frame.
4. The machine vision alignment-based camera mechanical error calibration method of claim 1, wherein the marker area occupies 1/16 to 1/48 of the reference frame picture area, and the shape and/or color of the marker is unique.
5. The method for calibrating mechanical error of camera based on machine vision alignment of claim 1, wherein the step (2) specifically comprises:
(2-1) scanning the current frame picture according to the characteristics of the markers of the reference frame to obtain all possible markers;
the marker of the reference frame is a target object detected by adopting a specific target detection algorithm, and the scanning of the current frame picture according to the characteristic of the marker of the reference frame is particularly carried out by adopting the target detection algorithm;
(2-2) for each of the plurality of markers of the reference frame, performing image similarity comparison among the possible markers obtained in the step (2-1), respectively, thereby matching to obtain an image of the marker in the current frame in the reference frame;
(2-3) determining the coordinates of the plurality of markers in the current frame obtained in the step (2-2) according to the same method as the determination method of the coordinates of the markers in the reference frame, respectively.
6. The machine vision alignment-based camera mechanical error calibration method of claim 1, wherein the camera offset vector is an average of offset vectors of markers in all current frames; the offset vector of the marker in the current frame is a vector taking the coordinate of the marker in the reference frame as a starting point and taking the coordinate of the marker in the current frame as an end point.
7. The machine vision alignment-based camera mechanical error calibration method of claim 1, wherein step (4) corrects the mechanical position of the camera as follows:
(4-1) vector decomposition is carried out on the offset vector of the camera according to the movement directions supported by the camera, so that the offset in each movement direction of the camera is obtained;
(4-2) taking the ratio of the offset in each moving direction to the moving step length of the camera as an integer part according to the offset in each moving direction of the camera and the moving step length of the camera in the corresponding moving direction obtained in the step (4-1) as the number of the moving compensation steps in the moving direction;
(4-3) adjusting the camera to move in each direction of movement by a corresponding number of motion compensation steps in a direction opposite the offset, thereby correcting the mechanical position of the camera.
8. A machine vision alignment-based camera mechanical error calibration system for implementing the machine vision alignment-based camera mechanical error calibration method of any one of claims 1 to 7, comprising:
the device comprises a current frame acquisition module, a marker detection module, a camera offset vector calculation module and a compensation reset module;
the current frame acquisition module is used for enabling the camera to be corrected to execute a command for positioning to a correction position, acquiring a picture captured by the camera in the state, taking the picture as a current frame, and submitting the current frame to the marker detection module;
the marker detection module is used for storing a preset reference frame, detecting all possible markers in the current frame, matching the markers in the reference frame according to the preset correction position, determining the successfully matched markers as the markers in the current frame, acquiring coordinates of the successfully matched markers in the current frame, and submitting the successfully matched markers to the camera offset calculation module;
the camera offset calculation module is used for storing the coordinates of the markers in the reference frame, comparing the differences between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the preset reference frame, determining a camera offset vector according to the differences between the coordinates of the plurality of markers in the current frame and the coordinates of the corresponding markers in the reference frame, and submitting the camera offset vector to the compensation reset module;
the compensation resetting module is used for judging whether error calibration is finished, and finishing error calibration when the offset vector of the camera is smaller than a preset threshold value or calibration oscillation is judged to occur and calibration is finished; otherwise, determining the motion compensation step number according to the camera offset vector, and correcting the mechanical position of the camera according to the motion compensation step number.
9. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the machine vision alignment based camera mechanical error calibration method of any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210497583.8A CN114979469B (en) | 2022-05-09 | 2022-05-09 | Camera mechanical error calibration method and system based on machine vision comparison |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210497583.8A CN114979469B (en) | 2022-05-09 | 2022-05-09 | Camera mechanical error calibration method and system based on machine vision comparison |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114979469A CN114979469A (en) | 2022-08-30 |
| CN114979469B true CN114979469B (en) | 2023-09-26 |
Family
ID=82981051
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210497583.8A Active CN114979469B (en) | 2022-05-09 | 2022-05-09 | Camera mechanical error calibration method and system based on machine vision comparison |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114979469B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115914844A (en) * | 2022-12-22 | 2023-04-04 | 深圳小湃科技有限公司 | Camera-based guard position correction method and device and readable storage medium |
| CN116840243B (en) * | 2023-09-01 | 2023-11-28 | 湖南睿图智能科技有限公司 | Correction method and system for machine vision object recognition |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1507407A2 (en) * | 2003-08-13 | 2005-02-16 | Topcon Corporation | Electronic image pickup apparatus with image correction function and method thereof |
| CN102937816A (en) * | 2012-11-22 | 2013-02-20 | 四川华雁信息产业股份有限公司 | Method and device for calibrating preset position deviation of camera |
| CN103502866A (en) * | 2011-04-01 | 2014-01-08 | 富士胶片株式会社 | Imaging device and program |
| CN111932609A (en) * | 2020-07-08 | 2020-11-13 | 广州科易光电技术有限公司 | Cloud deck calibration method and device for valve hall equipment inspection robot and storage medium |
| CN114401370A (en) * | 2022-01-26 | 2022-04-26 | 重庆紫光华山智安科技有限公司 | Movement calibration method and device, camera equipment and readable storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5402431B2 (en) * | 2009-09-11 | 2014-01-29 | 沖電気工業株式会社 | Camera control device |
-
2022
- 2022-05-09 CN CN202210497583.8A patent/CN114979469B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP1507407A2 (en) * | 2003-08-13 | 2005-02-16 | Topcon Corporation | Electronic image pickup apparatus with image correction function and method thereof |
| CN103502866A (en) * | 2011-04-01 | 2014-01-08 | 富士胶片株式会社 | Imaging device and program |
| CN102937816A (en) * | 2012-11-22 | 2013-02-20 | 四川华雁信息产业股份有限公司 | Method and device for calibrating preset position deviation of camera |
| CN111932609A (en) * | 2020-07-08 | 2020-11-13 | 广州科易光电技术有限公司 | Cloud deck calibration method and device for valve hall equipment inspection robot and storage medium |
| CN114401370A (en) * | 2022-01-26 | 2022-04-26 | 重庆紫光华山智安科技有限公司 | Movement calibration method and device, camera equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114979469A (en) | 2022-08-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114979469B (en) | Camera mechanical error calibration method and system based on machine vision comparison | |
| CN108230397B (en) | Multi-view camera calibration and correction method and apparatus, device, program and medium | |
| US11227144B2 (en) | Image processing device and method for detecting image of object to be detected from input data | |
| CN110580724B (en) | Method and device for calibrating binocular camera set and storage medium | |
| US11699247B2 (en) | System and method for runtime determination of camera miscalibration | |
| CN108229475B (en) | Vehicle tracking method, system, computer device and readable storage medium | |
| CN108470356B (en) | Target object rapid ranging method based on binocular vision | |
| US8538198B2 (en) | Method and apparatus for determining misalignment | |
| CN110087049A (en) | Automatic focusing system, method and projector | |
| JP2010041417A (en) | Image processing unit, image processing method, image processing program, and imaging apparatus | |
| CN110611770B (en) | Method and system for judging whether line frequency of linear array camera is matched with object motion speed | |
| CN110084842B (en) | Servo secondary alignment method and device for robot holder | |
| US9832455B2 (en) | Stereo camera and automatic range finding method for measuring a distance between stereo camera and reference plane | |
| CN113538583A (en) | Method for accurately positioning position of workpiece on machine tool and vision system | |
| US10652521B2 (en) | Stereo camera and image pickup system | |
| CN109815822A (en) | Inspection figure components target identification method based on Generalized Hough Transform | |
| CN111681186A (en) | Image processing method and device, electronic equipment and readable storage medium | |
| Kruger et al. | In-factory calibration of multiocular camera systems | |
| CN116563391B (en) | Automatic laser structure calibration method based on machine vision | |
| CN116433780B (en) | Automatic calibration method for laser structured light based on machine vision | |
| CN112991372B (en) | 2D-3D camera external parameter calibration method based on polygon matching | |
| CN112597832B (en) | Rectangular piece installation deviation judgment method and device | |
| CN114549653A (en) | Camera calibration method, system, computer equipment and storage medium | |
| CN112991296A (en) | Positioning method, detection system and computer readable storage medium | |
| Kollmitzer | Object detection and measurement using stereo images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |