CN114612447A - Image processing method and device based on data calibration and image processing equipment - Google Patents
Image processing method and device based on data calibration and image processing equipment Download PDFInfo
- Publication number
- CN114612447A CN114612447A CN202210264411.6A CN202210264411A CN114612447A CN 114612447 A CN114612447 A CN 114612447A CN 202210264411 A CN202210264411 A CN 202210264411A CN 114612447 A CN114612447 A CN 114612447A
- Authority
- CN
- China
- Prior art keywords
- image
- coordinate
- determining
- calibration plate
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 29
- 239000011159 matrix material Substances 0.000 claims abstract description 159
- 238000006243 chemical reaction Methods 0.000 claims abstract description 151
- 230000009466 transformation Effects 0.000 claims description 46
- 238000000034 method Methods 0.000 claims description 24
- 241001292396 Cirrhitidae Species 0.000 claims description 14
- 238000005520 cutting process Methods 0.000 claims description 4
- 230000008685 targeting Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 description 10
- 238000009434 installation Methods 0.000 description 9
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000033001 locomotion Effects 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000001915 proofreading effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000011217 control strategy Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000003754 machining Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T7/85—Stereo camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30244—Camera pose
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The application provides an image processing method and device based on data calibration and image processing equipment, and relates to the technical field of image processing. An image processing method, comprising: determining the pixel coordinates of the current image to be shot; determining the shooting position of the camera according to the pixel coordinates and a first conversion matrix and a second conversion matrix which are predetermined; the first conversion matrix is a coordinate conversion relation between a pixel coordinate system and a calibration plate coordinate system, and the second conversion matrix is a coordinate conversion relation between the calibration plate coordinate system and a mechanical coordinate system; controlling the camera to move to a shooting position so that the camera shoots a current image to be shot at the shooting position; and splicing the shot images into the current spliced image corresponding to the whole image based on the pixel coordinates. The image processing method is used for calibrating the mounting non-perpendicularity of the XY axes and the mounting deviation of the camera on the premise of low equipment cost, and obtaining a high-precision image processing result.
Description
Technical Field
The present application relates to the field of image processing technologies, and in particular, to an image processing method and apparatus based on data calibration, and an image processing device.
Background
In the industrial inspection field, in order to achieve image acquisition of a specific object, it is necessary to mount an XY axis (mounting axis of a camera) and a camera. Among them, the XY axis may not be absolutely perpendicular to the mounting of the XY axis due to the defects of the manufacturing and mounting processes, and the mounting positions of the XY axis and the camera may not be ensured to be consistent.
In the prior art, the problems that the XY axis is not perpendicular to the installation direction and the camera is deviated from the installation direction are solved through the laser interferometer, and on one hand, the laser interferometer needs to be introduced, so that the equipment cost is high; on the other hand, the laser interferometer cannot completely ensure that the deviation is eliminated, and the finally obtained image may still have deviation from the real object.
Disclosure of Invention
An object of the embodiments of the present application is to provide an image processing method and apparatus based on data calibration, and an image processing device, so as to achieve calibration of non-perpendicularity of XY axes and installation deviation of a camera on the premise of low device cost, and obtain a high-precision image processing result.
In a first aspect, an embodiment of the present application provides an image processing method based on data calibration, including: determining the pixel coordinates of the current image to be shot; the pixel coordinates are used for representing coordinates of the current image to be shot in the corresponding whole image; determining the shooting position of the camera according to the pixel coordinates and a first conversion matrix and a second conversion matrix which are determined in advance; the first conversion matrix is a coordinate conversion relation between a pixel coordinate system and a calibration plate coordinate system, the second conversion matrix is a coordinate conversion relation between the calibration plate coordinate system and a mechanical coordinate system, and the mechanical coordinate in the mechanical coordinate system is used for representing the shooting position of the camera; controlling the camera to move to the shooting position so that the camera shoots the current image to be shot at the shooting position; and splicing the shot images into the current spliced image corresponding to the whole image based on the pixel coordinates.
In the embodiment of the present application, the whole image is obtained by using a jigsaw method, that is, a plurality of images of a specific object are respectively collected and then the jigsaw is performed to obtain a complete image, which does not depend on other devices (such as a laser interferometer). In the process of acquiring the overall image by the jigsaw puzzle, the pixel coordinate of the image (namely the image to be shot) which needs to be acquired each time in the overall image is determined, and the calibrated mechanical coordinate (namely the coordinate of the camera in the XY axis, namely the shooting position) can be determined through the determined pixel coordinate and the predetermined first conversion matrix and second conversion matrix, so that the image shot by the camera at the corresponding shooting position is the image subjected to deviation calibration, the problem of low precision of the shot image caused by non-perpendicularity of the installation of the XY axis and the installation deviation of the camera is avoided, the calibrated image is spliced into the spliced image, and the overall image is spliced after the shooting of the image corresponding to each pixel coordinate is completed. And further, on the premise of low equipment cost, the calibration of the mounting non-perpendicularity of the XY axes and the mounting deviation of the camera is realized, and a high-precision image processing result is obtained.
As a possible implementation, the determining process of the first conversion matrix includes: acquiring a target sample image; the target sample image is a sub-image of a calibration plate shot by the camera at a first shooting position; determining calibration plate coordinates of each position of the sub-image of the calibration plate; determining pixel coordinates of each position of the sub-image of the calibration plate; the pixel coordinates of each position are used for representing the coordinates of each position in the whole image of the calibration plate; and determining the first conversion matrix according to the calibration plate coordinates of each position and the pixel coordinates of each position.
In the embodiment of the present application, for the first conversion matrix, the determination is performed using the sub-image of the calibration plate captured by the camera at the first capturing position. Firstly, respectively determining the coordinate of a calibration plate at each position in the sub-image and the pixel coordinate at each position, and then realizing effective determination of a first conversion matrix based on the coordinate of the calibration plate and the pixel coordinate, so that the pixel coordinate and the coordinate of the calibration plate can be mutually converted by the first conversion matrix.
As a possible implementation manner, the determining pixel coordinates of each position of the sub-image of the calibration board includes: and determining the pixel coordinates of each black circle by using a Hough circle finding algorithm of Opencv or a circle fitting algorithm of Halcon.
In the embodiment of the application, the calibration plate comprises coordinates of a plurality of black circles, and further, effective and accurate determination of pixel coordinates of each black circle can be realized by utilizing a Hough circle finding algorithm or a Halcon circle fitting algorithm.
As a possible implementation manner, the determining the pixel coordinates of each black circle by using a hough circle finding algorithm of Opencv or a circle fitting algorithm of Halcon includes: performing image cutting processing on the black circle of each pixel coordinate to be determined to obtain a cut image corresponding to the black circle of the pixel coordinate to be determined; determining the pixel coordinate of a black circle of the pixel coordinate to be determined through a Hough circle finding algorithm of Opencv or a fitting circle algorithm of Halcon; and correspondingly storing a tangent diagram corresponding to the black circle of the pixel coordinate to be determined and the pixel coordinate of the black circle of the pixel coordinate to be determined so as to determine the corresponding relation between each black circle and each determined pixel coordinate.
In the embodiment of the application, when the pixel coordinates of each black circle are determined based on the circle finding algorithm, the tangent map processing may be performed on each black circle, and then the obtained tangent map corresponds to the determined coordinates of the black circle to determine the corresponding relationship between each black circle and each determined pixel coordinate, so that the determined pixel coordinates of each black circle are more accurate.
As a possible implementation manner, the determining process of the second transformation matrix includes: acquiring sample image information; the sample image information includes: the camera comprises a sample image and a mechanical coordinate corresponding to the sample image, wherein the sample image is a sub-image of a calibration plate shot by the camera at a shooting position represented by the mechanical coordinate; determining calibration plate coordinates of the center position of the subimage of the calibration plate; and determining the second conversion matrix according to the mechanical coordinates and the calibration plate coordinates.
In the embodiment of the application, when the second conversion matrix is determined, the sample image is shot at the shooting position corresponding to the known mechanical coordinate, so that the mechanical coordinate of the sample image is known, the calibration plate coordinate of the center position of the sample image is determined, and the second conversion matrix is effectively and accurately determined according to the mechanical coordinate and the calibration plate coordinate.
As a possible implementation manner, the determining calibration plate coordinates of the center position of the sub-image of the calibration plate includes: determining a third transformation matrix; the third conversion matrix is a conversion relation between a pixel coordinate system corresponding to the sub-image of the calibration plate and a coordinate system of the calibration plate; determining pixel coordinates of the central position of the sub-image of the calibration plate; and determining the calibration plate coordinate of the central position of the sub-image of the calibration plate according to the pixel coordinate of the central position of the sub-image of the calibration plate and the third conversion matrix.
In the embodiment of the application, the conversion relation between the pixel coordinate system corresponding to the sub-image of the calibration plate and the coordinate system of the calibration plate is determined, then the pixel coordinate of the central position of the sub-image of the calibration plate is determined, and the conversion relation and the pixel coordinate are combined, so that the effective determination of the coordinate of the calibration plate of the central position can be realized.
As a possible implementation manner, the image processing method further includes: determining a spliced image corresponding to the whole image as a spliced image; determining whether the spliced image is consistent with the whole image; and if the spliced image is inconsistent with the whole image, optimizing the first conversion matrix and/or the second conversion matrix.
In the embodiment of the application, after the complete image is obtained based on the splicing mode, whether the complete image is accurately calibrated or not can be determined, and if the complete image is not accurately calibrated, the first conversion matrix and/or the second conversion matrix can be optimized so as to improve the precision of the next image processing result.
As a possible implementation manner, the determining a shooting position of a camera according to the pixel coordinates, a predetermined first conversion matrix and a predetermined second conversion matrix includes: determining a third conversion matrix according to the first conversion matrix and the second conversion matrix; the third conversion matrix is a coordinate conversion relation between a pixel coordinate system and a mechanical coordinate system; determining a converted calibration plate coordinate according to the pixel coordinate and the first conversion matrix; determining a transformed first mechanical coordinate from the transformed calibration plate coordinate and the second transformation matrix; determining a second converted mechanical coordinate according to the pixel coordinate and the third conversion matrix; judging whether the converted first mechanical coordinate and the converted second mechanical coordinate are consistent; if the converted first mechanical coordinate is consistent with the converted second mechanical coordinate, determining the shooting position according to the first mechanical coordinate and/or the second mechanical coordinate; if the converted first mechanical coordinate and the converted second mechanical coordinate are not consistent, determining a final mechanical coordinate according to the converted first mechanical coordinate and the converted second mechanical coordinate; and determining the shooting position according to the final mechanical coordinate.
In the embodiment of the application, a mechanical coordinate can be directly obtained based on the first conversion matrix and the second conversion matrix; and further obtaining a third transformation matrix based on the first transformation matrix and the second transformation matrix, and obtaining a mechanical coordinate by utilizing the third transformation matrix; comparing the first mechanical coordinate with the second mechanical coordinate, if the first mechanical coordinate and the second mechanical coordinate are consistent, the accuracy of the two conversion matrixes is higher, and the shooting position can be determined directly based on any one mechanical coordinate; if the two matrixes are not consistent, the precision of the two matrixes can be optimized, and at the moment, the shooting position can be determined by combining the two mechanical coordinates so as to avoid influencing the precision of the final shooting result.
In a second aspect, an embodiment of the present application provides an image processing apparatus based on data calibration, including: the functional modules are configured to implement the image processing method based on data calibration as described in the first aspect and any one of the possible implementation manners of the first aspect.
In a third aspect, an embodiment of the present application provides an image processing apparatus, including: a processor; and a memory communicatively coupled to the processor; wherein the memory stores instructions executable by the processor to enable the processor to perform the method for image processing based on data targeting as described in the first aspect and any one of the possible implementations of the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computer, the method for processing an image based on data calibration as described in the first aspect and any possible implementation manner of the first aspect is performed.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic diagram of a shooting principle provided in an embodiment of the present application;
fig. 2 is a flowchart of an image processing method based on data calibration according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an image processing apparatus based on data calibration according to an embodiment of the present application;
fig. 4 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present application.
Icon: 300-image processing apparatus based on data calibration; 310-a processing module; 320-a control module; 400-an image processing device; 410-a processor; 420-memory.
Detailed Description
The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.
The technical scheme provided by the embodiment of the application can be applied to various image acquisition scenes, such as: industrial inspection scenarios, in which image acquisition of a specific object is usually required for relevant inspection based on the acquired image. For example: and carrying out image acquisition on the circuit board.
In these image capturing scenarios, in order to achieve capturing of an image, the position of the object to be detected is usually fixed, and the position of the fixing device of the image capturing device (usually a camera) is also usually fixed, and the fixing device of the camera is movable, by controlling the movement of the fixing device, to achieve a change in the position of the camera.
It can be understood that in the field of industrial detection, the area of the object to be detected may be relatively large, or the shape is relatively complex, etc., and an image of the whole object to be detected often cannot be acquired through one-time image acquisition. Therefore, in the embodiment of the present application, the image acquisition of the whole object to be detected is realized by acquiring the image for multiple times and then performing the puzzle image acquisition. Wherein, the images collected each time are not repeated, and the same part does not exist.
In this way of image acquisition, three coordinate systems are involved, a first coordinate system being a coordinate system in which the object to be detected is located, which may be understood as a physical coordinate system. The second coordinate system is a pixel coordinate system, and the pixel coordinates in the pixel coordinate system are used for representing the position of the sub-image acquired at each time in the whole image. And the third coordinate system is a mechanical coordinate system, the mechanical coordinates in the mechanical coordinate system are used for representing the shooting position of the camera, and when the camera is controlled to collect images, the control is also carried out based on the mechanical coordinates.
In these three coordinate systems, the pixel coordinate system and the physical coordinate system are usually accurate, but due to installation deviation of the fixing device (for example, mechanical XY axis) of the camera, the mechanical coordinates of the mechanical coordinate system may not be accurate, and therefore, calibration of the mechanical coordinates is required to ensure that the camera captures a matched sub-image at a correct capturing position when capturing an image.
For convenience of understanding, please refer to fig. 1, which is a schematic diagram of a shooting scene provided in an embodiment of the present application, wherein a mechanical Y axis and a mechanical X axis are used to fix a camera, and the camera is provided with a light source. Through the motion of mechanical Y axle and mechanical X axle, can drive the camera and remove to the assigned position and take a picture. The object to be shot can be placed on the platform, the object to be shot corresponds to a calibration plate coordinate system Y axis and a calibration plate coordinate system X axis, and the calibration plate is the shot object used when the calibration matrix is determined in the previous period, so that the coordinate system where the object to be shot is located is called the calibration plate coordinate system. And, further comprising a pixel coordinate system Y-axis and a pixel coordinate system X-axis.
The following describes the jigsaw puzzle principle and general inventive concept used in the embodiments of the present application, based on fig. 1.
Firstly, the calibration objects related to the embodiment of the present application include two: one is the mechanical XY axis and one is the camera, the camera is fixed on the mechanical X axis as shown in fig. 1, and the camera takes pictures at different positions by moving the mechanical X axis and the mechanical Y axis. There are problems in that:
1. the mechanical X, Y axes cannot be made absolutely perpendicular when mounted.
2. The pixel size of a picture taken by a camera is fixed, but it cannot be known in detail how many mm the physical size of each pixel corresponds to.
3. In the actual working process, an object (usually a circuit board) to be shot cannot be completely shot by one picture, so that multiple pictures need to be shot. But require a large picture of the complete object and therefore a puzzle.
4. The real data of the image cannot be lost due to the requirement of jigsaw puzzle, so the image cannot be processed by using a related algorithm in the jigsaw puzzle process, and only the coordinate can be spliced.
As shown in fig. 1, for convenience, several parameters are first set: the size of the image taken by the camera is 4000pixels by 3000pixels (pixel unit), each pixel being 10um in size, so that the camera can only take 40mm by 30mm of the object per picture.
Assuming that the actual size of the object to be photographed is 160mm by 120mm, 4 by 4 to 16 pictures are taken and then they are spliced together to obtain a complete large image.
Furthermore, in the picture splicing process, the obtained 16 pictures are shot, the coordinates of the upper left corner of each picture are (0,0), the coordinates of the lower right corner of each picture are (4000,3000), the coordinates of the upper left corner of a large picture formed by splicing the 16 pictures are (0,0), and the coordinates of the lower right corner of the large picture are (16000, 12000); the 16 graphs all have position coordinate information of themselves in the big graph, the position is represented by the upper left corner of each graph, the position of the first graph is (0,0), the position coordinate of the second graph in the first row is (1 x 4000, 0 x 3000), and so on, the data of each graph is copied to the designated position, and the jigsaw can be completed.
Further, as can be seen from fig. 1, there is a pixel coordinate system, which is composed of two axes 8 and 9, and the pixel coordinate system is established when the camera takes the first picture, and the upper left corner of the first picture is taken as the origin of coordinates. Fig. 1 has 3 coordinate systems in total, one is a mechanical coordinate system (due to machining, installation and other reasons, the X and Y may not be a 90 ° included angle), and the other is a punctuation plate coordinate system (the distance between the centers of every two adjacent small black circles is 10mm, which is considered to be absolutely accurate). The origin of the coordinate system can be used for self-specifying the center of one of the circles, wherein the circle at the lower left corner of the calibration plate is selected.
The problem now is that to achieve a mosaic that is accurate and free of deviations, the coordinate system that can be selected is only the pixel coordinate system.
Examples are: there is now a circuit board size of 160mm 1200mm, and the size of the entire circuit board is 16000 x 12000 in pixels, depending on the set parameters. The size of a single picture taken by the camera is 4000 x 3000pixels, so that a total of 16 pictures of 4 rows and 4 columns need to be taken to be spliced into a large picture of 16000 x 12000 pixels, and it is known that the pixel position coordinates (i x 4000, j 3000) of each small picture can be calculated, wherein i represents a row, j represents a column, and the value range is 0-3. If the first picture needs to be taken now, the pixel position is the upper left corner (0,0) of the circuit board, and there is an offset between the upper left corner of the circuit board and the origin of the pixel coordinate system established during calibration, so that the camera should move to a position (0,0) + offset + (4000/2,3000/2) below the pixel coordinate system to take the first picture, and since the camera needs to move to the center of the first picture, it needs to be added (4000/2,3000/2).
The movement of the camera requires a program to control the motors to move the mechanical X and Y axes to a designated position, but the mechanical axis is not vertical, and on the premise that the pixel coordinate is known, the pixel coordinate needs to be converted into the mechanical coordinate.
Based on the above application scenarios and introduction of the inventive concept, the hardware operating environment of the technical solution provided in the embodiment of the present application may be an image processing apparatus, for example: the computer and other electronic equipment, the image processing equipment is connected with the camera and the control device of the mechanical X axis and the mechanical Y axis in a communication way, on one hand, the image processing equipment can send control instructions to the control device to enable the control device to control the movement of the mechanical X axis and the mechanical Y axis; on the other hand, the image collected by the camera is sent to the image processing equipment, and the image processing equipment performs jigsaw processing. Specifically, how the image processing apparatus generates the control instruction, and an embodiment of the control device includes: hardware implementation, control strategy, etc., do not belong to the importance of the embodiments of the present application, and refer to the mature technology in the field, which will not be described in detail in the embodiments of the present application.
Referring to fig. 2, a flowchart of an image processing method based on data calibration according to an embodiment of the present application is shown, where the image processing method includes:
step 210: and determining the pixel coordinates of the current image to be shot. The pixel coordinates are used for representing the coordinates of the current image to be shot in the corresponding whole image.
Step 220: and determining the shooting position of the camera according to the pixel coordinates and the first conversion matrix and the second conversion matrix which are determined in advance. The first conversion matrix is a coordinate conversion relation between a pixel coordinate system and a calibration plate coordinate system, the second conversion matrix is a coordinate conversion relation between the calibration plate coordinate system and a mechanical coordinate system, and the mechanical coordinate in the mechanical coordinate system is used for representing the shooting position of the camera.
Step 230: and controlling the camera to move to the shooting position so that the camera shoots the current image to be shot at the shooting position.
Step 240: and splicing the shot images into the current spliced image corresponding to the whole image based on the pixel coordinates.
In the embodiment of the present application, the whole image is obtained by using a jigsaw method, that is, a plurality of images of a specific object are respectively collected and then the jigsaw is performed to obtain a complete image, which does not depend on other devices (such as a laser interferometer). In the process of acquiring the overall image by the jigsaw puzzle, the pixel coordinate of the image (namely the image to be shot) which needs to be acquired each time in the overall image is determined, and the calibrated mechanical coordinate (namely the coordinate of the camera in the XY axis, namely the shooting position) can be determined through the determined pixel coordinate and the predetermined first conversion matrix and second conversion matrix, so that the image shot by the camera at the corresponding shooting position is the image subjected to deviation calibration, the problem of low precision of the shot image caused by non-perpendicularity of the installation of the XY axis and the installation deviation of the camera is avoided, the calibrated image is spliced into the spliced image, and the overall image is spliced after the shooting of the image corresponding to each pixel coordinate is completed. And further, on the premise of low equipment cost, the calibration of the mounting non-perpendicularity of the XY axes and the mounting deviation of the camera is realized, and a high-precision image processing result is obtained.
A detailed embodiment of the image processing method will be described below.
In step 210, the pixel coordinates of the current image to be captured are determined. In combination with the foregoing description of the embodiment, in the case that the pixel coordinate of the whole image is the given coordinate, for any one of the images to be captured, the pixel coordinate of the image to be captured may be determined according to the image to be captured in the several rows and several columns and the several images. For example as exemplified in the previous embodiments, (i x 4000, j x 3000). Of course, if the whole image has other pixels, 4000 and 3000 may be changed accordingly.
Therefore, whether other images are shot or not currently or images are shot for the first time, the pixel coordinate of the current image to be shot can be determined by utilizing the preset pixel coordinate relationship between the sub-image and the whole image.
In step 220, the shooting position of the camera is determined according to the pixel coordinates, the predetermined first conversion matrix and the predetermined second conversion matrix. The first conversion matrix is a coordinate conversion relation between a pixel coordinate system and a calibration plate coordinate system, the second conversion matrix is a coordinate conversion relation between the calibration plate coordinate system and a mechanical coordinate system, and the mechanical coordinate in the mechanical coordinate system is used for representing the shooting position of the camera.
As an optional implementation, the determining process of the first conversion matrix includes: acquiring a target sample image; the target sample image is a sub-image of a calibration plate shot by a camera at a first shooting position; determining calibration plate coordinates of each position of the sub-image of the calibration plate; determining pixel coordinates of each position of the sub-image of the calibration plate; the pixel coordinates of each position are used for representing the coordinates of each position in the whole image of the calibration plate; and determining a first conversion matrix according to the calibration plate coordinates of each position and the pixel coordinates of each position.
In this embodiment, the first shooting position may be understood as the first position where the camera takes an image of the calibration plate, for example: the upper left corner.
The calibration plate coordinates of each position of the sub-image of the calibration plate can be understood as calibration plate coordinates of a representative position in the calibration plate. In some embodiments, the calibration plates are plates comprising black circles, in other embodiments the calibration plates are checkered (black and white). Then, the positions here may be the positions of the black circles, or the positions of the grids.
For convenience of understanding, taking the black circle calibration plate as an example, the calibration plate includes a plurality of black circles, each position is a position of each black circle, and determining pixel coordinates of each position of a sub-image of the calibration plate includes: and determining the pixel coordinates of each black circle by using a Hough circle finding algorithm of Opencv or a circle fitting algorithm of Halcon.
In this embodiment, the circle finding algorithm of hough of Opencv or the circle fitting algorithm of Halcon can be used to locate each circle, and after the circle finding algorithm is located, the pixel coordinates of each circle can be determined.
In the embodiment of the application, the calibration plate comprises coordinates of a plurality of black circles, and further, effective and accurate determination of pixel coordinates of each black circle can be realized by utilizing a Hough circle finding algorithm or a Halcon circle fitting algorithm.
As an optional implementation, determining the pixel coordinates of each black circle by using hough circle finding algorithm of Opencv or circle fitting algorithm of Halcon includes: performing image cutting processing on the black circle of each pixel coordinate to be determined to obtain a cut image corresponding to the black circle of the pixel coordinate to be determined; determining the pixel coordinate of a black circle of the pixel coordinate to be determined through a Hough circle finding algorithm of Opencv or a fitting circle algorithm of Halcon; and correspondingly storing the tangent graph corresponding to the black circle of the pixel coordinate to be determined and the pixel coordinate of the black circle of the pixel coordinate to be determined so as to determine the corresponding relation between each black circle and each determined pixel coordinate.
In this embodiment, when determining the pixel coordinates of each black circle based on the circle finding algorithm, the tangent map processing may be performed on each black circle, and then the obtained tangent map may be associated with the coordinates of the determined black circle to determine the corresponding relationship between each black circle and each determined pixel coordinate, so as to make the determined pixel coordinates of each black circle more accurate.
Further, as an alternative embodiment, the calibration plate includes a plurality of black circles with the same size, and the distance between two adjacent black circles is a preset distance (e.g. 10mm), then, in combination with the number of black circles in the calibration plate image and the preset distance, the calibration plate coordinates of each black circle can be determined. For example: the first black circle in the upper left corner has the coordinates (10, 10).
Therefore, in the embodiment of the present application, the calibration plate coordinates of each position in the sub-image and the pixel coordinates of each position are respectively determined, and then the effective determination of the first conversion matrix is realized based on the calibration plate coordinates and the pixel coordinates, so that the first conversion matrix can perform mutual conversion on the pixel coordinates and the calibration plate coordinates.
It can be understood that, assuming that the deviation of the included angle of the mechanical coordinate XY axes is not too large, if there are 400 dots with 20 × 20 size on the calibration plate, the distance between the centers of every two adjacent dots is 10mm, and the size of the shot by the camera is 40mm × 30mm, then the camera position adjusted properly can shoot just 4 × 3 to 12 small circles, and there are also 12 non-repeated circles in the next picture when the X axis is moved by 40mm or the Y axis is moved by 30mm, and if the deviation of the mechanical X axis and the Y axis is too large, some small black circles may be incompletely shot, or may be lost. Therefore, the requirements for each captured image are: all black circles on the calibration board in the current visual field are shot, and all shot photos require that the black circles are not repeated and not omitted, so that the coordinates of all dots on each picture on the calibration board can be conveniently determined.
Further, after obtaining the pixel coordinates of each black circle of the sub-image and the calibration plate coordinates, a first conversion matrix may be generated according to the pixel coordinates of each black circle and the calibration plate coordinates, for example: the first conversion matrix is a matrix formed by ratios between pixel coordinates of each black circle and coordinates of the calibration plate, or other forms, which are not limited herein, as long as correct conversion of the two coordinates can be ensured.
In some embodiments, the transformation matrix between the two coordinate systems may also be obtained by affine transformation.
In combination with the determination method of the first transformation matrix, as an optional implementation, the determination process of the second transformation matrix includes: acquiring sample image information; the sample image information includes: the method comprises the steps that a sample image and a mechanical coordinate corresponding to the sample image are obtained, wherein the sample image is a sub-image of a calibration plate shot by a camera at a shooting position represented by the mechanical coordinate; determining calibration plate coordinates of the central position of the sub-image of the calibration plate; and determining a second conversion matrix according to the mechanical coordinates and the calibration plate coordinates.
In this embodiment, the sample image does not define the shooting position, and may be a plurality of sub-images shot at a plurality of shooting positions, respectively, and finally, when determining the second conversion matrix, the coordinates of the plurality of sub-images are combined and determined.
As an alternative embodiment, determining calibration plate coordinates of the center position of the sub-image of the calibration plate includes: determining a third transformation matrix; the third conversion matrix is the conversion relation between the pixel coordinate system corresponding to the sub-image of the calibration plate and the coordinate system of the calibration plate; determining pixel coordinates of the central position of the sub-image of the calibration plate; and determining the calibration plate coordinate of the central position of the sub-image of the calibration plate according to the pixel coordinate of the central position of the sub-image of the calibration plate and the third conversion matrix.
In this embodiment, the third transformation matrix and the first transformation matrix are used to represent the transformation relationship between the pixel coordinate system and the calibration board coordinate system, except that the sub-images corresponding to the first transformation matrix and the third transformation matrix are different. Therefore, the determination manner of the third conversion matrix may refer to the determination manner of the first conversion matrix.
Specifically, it is continuously assumed that the calibration plate includes a plurality of black circles having the same size, the distance between two adjacent black circles is a preset distance, the designated position of the calibration plate is a coordinate origin, and the calibration plate coordinates of each black circle in the sub-image of the calibration plate are determined according to the number of black circles in the sub-image of the calibration plate, the preset distance, and the size of the plurality of black circles.
And then determining the pixel coordinates of each black circle in the sub-image of the calibration plate according to the determination mode of the pixel coordinates of each black circle.
By using the calibration plate coordinates and the pixel coordinates of each black circle of the sub-image, the conversion relationship between the calibration plate coordinate system and the pixel coordinate system corresponding to the sub-image, i.e. the third conversion matrix, can be determined.
Some details are referred to therein, such as: for finding circles, cut maps, etc., reference may be made to the above-described embodiment of the first transformation matrix, and a description thereof will not be repeated.
Further, as for the pixel coordinates of the center position of the sub-image of the calibration plate, since the pixel coordinates of the sub-image of the calibration plate are known, the pixel coordinates of the center position thereof can be simply determined.
And finally, determining the coordinate of the calibration plate of the central position of the sub-image of the calibration plate by using the pixel coordinate of the central position of the sub-image of the calibration plate and the third conversion matrix.
In the embodiment of the application, the conversion relation between the pixel coordinate system corresponding to the sub-image of the calibration plate and the coordinate system of the calibration plate is determined, then the pixel coordinate of the central position of the sub-image of the calibration plate is determined, and the conversion relation and the pixel coordinate are combined, so that the effective determination of the coordinate of the calibration plate of the central position can be realized.
Further, in the case where the mechanical coordinates are known, and the calibration plate coordinates of the sub-images of the calibration plate are also known, the transformation matrix between the calibration plate coordinate system and the mechanical coordinate system, that is, the second transformation matrix, can be calculated by affine transformation as well.
In the embodiment of the application, when the second conversion matrix is determined, the sample image is shot at the shooting position corresponding to the known mechanical coordinate, so that the mechanical coordinate of the sample image is known, the calibration plate coordinate of the center position of the sample image is determined, and the second conversion matrix is effectively and accurately determined according to the mechanical coordinate and the calibration plate coordinate.
In addition, after the first conversion matrix and the second conversion matrix are determined, the pixel size of the combination of the camera and the lens can be correspondingly calculated, and if necessary, the pixel size can be applied.
Assuming that the distance between the two small black circles is 10mm, the pixel coordinates of the centers of the two small black circles can also be known by finding circles. Therefore, the size of one pixel can be calculated, for example, o1 ═ 2000pixels are known by way of circle finding by hough; and o2 equals 3000pixels, then 10mm/(o2-o1 equals 10um/pixel, which is the size of one pixel.
In conjunction with the description of the first transformation matrix and the second transformation matrix, in step 220, assuming that the first transformation matrix is Q1, the second transformation matrix is Q2, and the pixel coordinate is P, the mechanical coordinate can be expressed as: p × Q1 × Q2.
Of course, this is just one embodiment, and as another alternative embodiment, step 220 includes: determining a third conversion matrix according to the first conversion matrix and the second conversion matrix; the third conversion matrix is a coordinate conversion relation between a pixel coordinate system and a mechanical coordinate system; determining a converted calibration plate coordinate according to the pixel coordinate and the first conversion matrix; determining a transformed first mechanical coordinate from the transformed calibration plate coordinate and the second transformation matrix; determining a second converted mechanical coordinate according to the pixel coordinate and the third conversion matrix; judging whether the converted first mechanical coordinate is consistent with the converted second mechanical coordinate; if the converted first mechanical coordinate is consistent with the converted second mechanical coordinate, determining a shooting position according to the first mechanical coordinate and/or the second mechanical coordinate; if the converted first mechanical coordinate is inconsistent with the converted second mechanical coordinate, determining a final mechanical coordinate according to the converted first mechanical coordinate and the converted second mechanical coordinate; and determining the shooting position according to the final mechanical coordinate.
In this embodiment, affine transformation may be performed by using the first transformation matrix and the second transformation matrix to obtain a third transformation matrix, and the third transformation matrix may directly represent a coordinate transformation relationship between the pixel coordinate system and the mechanical coordinate system. However, this embodiment is premised on that the sub-images of the calibration plates corresponding to the first and second transformation matrices should be identical, so that the coordinates in the pixel coordinate system and the mechanical coordinate system can be guaranteed to correspond one to one.
The third transformation matrix may also be determined by the aforementioned affine transformation.
Further, based on the three conversion matrixes, firstly, determining converted coordinates of a calibration board according to the pixel coordinates and the first conversion matrix; the transformed first mechanical coordinates are determined from the transformed calibration plate coordinates and the second transformation matrix. Then, a converted second mechanical coordinate is determined from the pixel coordinate and the third conversion matrix.
And judging whether the converted first mechanical coordinate and the converted second mechanical coordinate are consistent or not based on the first mechanical coordinate and the second mechanical coordinate. If the converted first mechanical coordinate and the converted second mechanical coordinate are consistent, the shooting position is determined according to the first mechanical coordinate and/or the second mechanical coordinate, and if the two coordinates are the same, which one is used for determining the shooting position at will.
If the converted first mechanical coordinate is inconsistent with the converted second mechanical coordinate, determining a final mechanical coordinate according to the converted first mechanical coordinate and the converted second mechanical coordinate; and determining the shooting position according to the final mechanical coordinate. The converted first mechanical coordinate and the converted second mechanical coordinate may be weighted-averaged, or subjected to other arithmetic processing, which is not limited herein, to obtain a final mechanical coordinate.
In the embodiment of the application, a mechanical coordinate can be directly obtained based on the first conversion matrix and the second conversion matrix; and further obtaining a third transformation matrix based on the first transformation matrix and the second transformation matrix, and obtaining a mechanical coordinate by utilizing the third transformation matrix; comparing the first mechanical coordinate with the second mechanical coordinate, if the first mechanical coordinate and the second mechanical coordinate are consistent, the accuracy of the two conversion matrixes is higher, and the shooting position can be determined directly based on any one mechanical coordinate; if the two matrixes are not consistent, the precision of the two matrixes can be optimized, and at the moment, the shooting position can be determined by combining the two mechanical coordinates so as to avoid influencing the precision of the final shooting result.
In any of the embodiments, the mechanical coordinates are determined, and after the mechanical coordinates are determined, the X coordinate of the mechanical coordinates represents the X-axis movement position and the Y coordinate represents the Y-axis movement position.
Further, in step 230, the camera is controlled to move to the shooting position so that the camera shoots the current image to be shot at the shooting position. The specific control method refers to the description of the principle part in the foregoing embodiments, and the description is not repeated here.
And in step 240, the shot images are spliced into the current spliced image corresponding to the whole image based on the pixel coordinates. The implementation of this step can refer to the description of the aforementioned principle, and the description is not repeated here.
And sequentially shooting a plurality of images according to the implementation modes of the steps 210 to 240, and performing picture splicing treatment respectively to finally obtain images after splicing. At this time, the image processing method may further include: determining a spliced image corresponding to the whole image as a spliced image; determining whether the spliced image is consistent with the whole image; and if the spliced image is inconsistent with the whole image, optimizing the first conversion matrix and/or the second conversion matrix.
In this embodiment, the stitched image may be fed back to the worker, and the worker may perform a proofreading on the stitched image and then give a proofreading result. For example: the spliced image is not partially lost, or some sub-images are overlapped, and the like, and the spliced image can be regarded as inconsistent with the whole image.
Furthermore, if the spliced image is consistent with the whole image, the realization of high-precision jigsaw puzzle is demonstrated.
If the stitched image is not consistent with the whole image, the first conversion matrix and/or the second conversion matrix can be optimized.
The optimization method can be as follows: re-determining the first and/or second transformation matrices; alternatively, the first conversion matrix and/or the second conversion matrix may be manually adjusted by a worker, or other embodiments may be implemented, but not limited thereto.
In the embodiment of the application, after the complete image is obtained based on the splicing mode, whether the complete image is accurately calibrated or not can be determined, and if the complete image is not accurately calibrated, the first conversion matrix and/or the second conversion matrix can be optimized so as to improve the precision of the next image processing result.
Based on the same inventive concept, please refer to fig. 3, an embodiment of the present application further provides an image processing apparatus 300 based on data calibration, which includes a processing module 310 and a control module 320.
A processing module 310 configured to: determining the pixel coordinates of the current image to be shot; the pixel coordinates are used for representing coordinates of the current image to be shot in the corresponding whole image; determining the shooting position of the camera according to the pixel coordinates and a first conversion matrix and a second conversion matrix which are determined in advance; the first conversion matrix is a coordinate conversion relation between a pixel coordinate system and a calibration plate coordinate system, the second conversion matrix is a coordinate conversion relation between the calibration plate coordinate system and a mechanical coordinate system, and the mechanical coordinate in the mechanical coordinate system is used for representing the shooting position of the camera. A control module 320, configured to control the camera to move to the shooting position, so that the camera shoots the current image to be shot at the shooting position; the processing module is further used for splicing the shot images into a current spliced image corresponding to the whole image based on the pixel coordinates.
In this embodiment of the application, the processing module 310 is further configured to: acquiring a target sample image; the target sample image is a sub-image of a calibration plate shot by the camera at a first shooting position; determining calibration plate coordinates of each position of the sub-image of the calibration plate; determining pixel coordinates of each position of the sub-image of the calibration plate; the pixel coordinates of each position are used for representing the coordinates of each position in the whole image of the calibration plate; and determining the first conversion matrix according to the calibration plate coordinates of each position and the pixel coordinates of each position.
In this embodiment of the application, the processing module 310 is specifically configured to: and determining the pixel coordinates of each black circle by using a Hough circle finding algorithm of Opencv or a circle fitting algorithm of Halcon.
In this embodiment of the application, the processing module 310 is specifically configured to: performing image cutting processing on the black circle of each pixel coordinate to be determined to obtain a cut image corresponding to the black circle of the pixel coordinate to be determined; determining the pixel coordinate of a black circle of the pixel coordinate to be determined through a Hough circle finding algorithm of Opencv or a fitting circle algorithm of Halcon; and correspondingly storing a tangent diagram corresponding to the black circle of the pixel coordinate to be determined and the pixel coordinate of the black circle of the pixel coordinate to be determined so as to determine the corresponding relation between each black circle and each determined pixel coordinate.
In this embodiment of the application, the processing module 310 is further configured to: acquiring sample image information; the sample image information includes: the camera comprises a sample image and a mechanical coordinate corresponding to the sample image, wherein the sample image is a sub-image of a calibration plate shot by the camera at a shooting position represented by the mechanical coordinate; determining calibration plate coordinates of the center position of the sub-images of the calibration plate; and determining the second conversion matrix according to the mechanical coordinates and the calibration plate coordinates.
In this embodiment of the application, the processing module 310 is specifically configured to: determining a third transformation matrix; the third conversion matrix is a conversion relation between a pixel coordinate system corresponding to the sub-image of the calibration plate and a coordinate system of the calibration plate; determining pixel coordinates of the central position of the sub-image of the calibration plate; and determining the calibration plate coordinate of the central position of the sub-image of the calibration plate according to the pixel coordinate of the central position of the sub-image of the calibration plate and the third conversion matrix.
In this embodiment of the application, the processing module 310 is further configured to: determining a spliced image corresponding to the whole image as a spliced image; determining whether the stitched image is consistent with the whole image; and if the spliced image is inconsistent with the whole image, optimizing the first conversion matrix and/or the second conversion matrix.
In this embodiment of the application, the processing module 310 is specifically configured to: determining a third conversion matrix according to the first conversion matrix and the second conversion matrix; the third conversion matrix is a coordinate conversion relation between a pixel coordinate system and a mechanical coordinate system; determining a converted calibration plate coordinate according to the pixel coordinate and the first conversion matrix; determining a transformed first mechanical coordinate from the transformed calibration plate coordinate and the second transformation matrix; determining a second converted mechanical coordinate according to the pixel coordinate and the third conversion matrix; judging whether the converted first mechanical coordinate and the converted second mechanical coordinate are consistent; if the converted first mechanical coordinate is consistent with the converted second mechanical coordinate, determining the shooting position according to the first mechanical coordinate and/or the second mechanical coordinate; if the converted first mechanical coordinate and the converted second mechanical coordinate are not consistent, determining a final mechanical coordinate according to the converted first mechanical coordinate and the converted second mechanical coordinate; and determining the shooting position according to the final mechanical coordinate.
The image processing apparatus 300 based on data calibration corresponds to the image processing method based on data calibration, and each functional module corresponds to each step of the method, so that the embodiments of each functional module refer to the embodiments of the method, and are not repeated here.
Based on the same inventive concept, please refer to fig. 4, an embodiment of the present application further provides an image processing apparatus 400, which includes a processor 410 and a memory 420, wherein the processor 410 and the memory 420 are in communication connection, and the image processing apparatus 400 is used as an execution subject of the foregoing image processing method.
The processor 410 and the memory 420 are electrically connected directly or indirectly to realize data transmission or interaction. For example, electrical connections between these components may be made through one or more communication or signal buses. The aforementioned image processing methods respectively include at least one software functional module that can be stored in the memory 420 in the form of software or firmware (firmware).
The processor 410 may be an integrated circuit chip having signal processing capabilities. The Processor 310 may be a general-purpose Processor including a CPU (Central Processing Unit), an NP (Network Processor), and the like; but may also be a digital signal processor, an application specific integrated circuit, an off-the-shelf programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components. Which may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present application. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 420 may store various software programs and modules, such as program instructions/modules corresponding to the image processing method and apparatus provided in the embodiments of the present application. The processor 410 executes various functional applications and data processing by executing software programs and modules stored in the memory 420, thereby implementing the methods in the embodiments of the present application.
The Memory 420 may include, but is not limited to, a RAM (Random Access Memory), a ROM (Read Only Memory), a PROM (Programmable Read-Only Memory), an EPROM (Erasable Read-Only Memory), an EEPROM (electrically Erasable Read-Only Memory), and the like.
It will be appreciated that the configuration shown in fig. 4 is merely illustrative, and that the image processing apparatus 400 may also include more or fewer components than shown in fig. 4, or have a different configuration than shown in fig. 4.
Based on the same inventive concept, embodiments of the present application further provide a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a computer, the computer program executes the image processing method according to any of the above embodiments.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. An image processing method based on data calibration is characterized by comprising the following steps:
determining the pixel coordinates of the current image to be shot; the pixel coordinates are used for representing coordinates of the current image to be shot in the corresponding whole image;
determining the shooting position of a camera according to the pixel coordinates and a first conversion matrix and a second conversion matrix which are determined in advance; the first conversion matrix is a coordinate conversion relation between a pixel coordinate system and a calibration plate coordinate system, the second conversion matrix is a coordinate conversion relation between the calibration plate coordinate system and a mechanical coordinate system, and the mechanical coordinate in the mechanical coordinate system is used for representing the shooting position of the camera;
controlling the camera to move to the shooting position so that the camera shoots the current image to be shot at the shooting position;
and splicing the shot images into the current spliced image corresponding to the whole image based on the pixel coordinates.
2. The image processing method according to claim 1, wherein the determination of the first conversion matrix comprises:
acquiring a target sample image; the target sample image is a sub-image of a calibration plate shot by the camera at a first shooting position;
determining calibration plate coordinates of each position of the sub-image of the calibration plate;
determining pixel coordinates of each position of the sub-image of the calibration plate; the pixel coordinates of each position are used for representing the coordinates of each position in the whole image of the calibration plate;
and determining the first conversion matrix according to the calibration plate coordinates of each position and the pixel coordinates of each position.
3. The image processing method according to claim 2, wherein the calibration plate comprises a plurality of black circles, and the respective positions are positions of the respective black circles, and the determining pixel coordinates of the respective positions of the sub-images of the calibration plate comprises:
and determining the pixel coordinates of each black circle by using a Hough circle finding algorithm of Opencv or a circle fitting algorithm of Halcon.
4. The image processing method according to claim 3, wherein the determining the pixel coordinates of each black circle by a Hough circle finding algorithm of Opencv or a circle fitting algorithm of Halcon comprises:
performing image cutting processing on the black circle of each pixel coordinate to be determined to obtain a cut image corresponding to the black circle of the pixel coordinate to be determined;
determining the pixel coordinate of a black circle of the pixel coordinate to be determined through a Hough circle finding algorithm of Opencv or a fitting circle algorithm of Halcon;
and correspondingly storing a tangent map corresponding to the black circle of the pixel coordinate to be determined and the pixel coordinate of the black circle of the pixel coordinate to be determined so as to determine the corresponding relation between each black circle and each determined pixel coordinate.
5. The image processing method according to claim 1, wherein the determining of the second transformation matrix comprises:
acquiring sample image information; the sample image information includes: the camera comprises a sample image and a mechanical coordinate corresponding to the sample image, wherein the sample image is a sub-image of a calibration plate shot by the camera at a shooting position represented by the mechanical coordinate;
determining calibration plate coordinates of the center position of the subimage of the calibration plate;
and determining the second conversion matrix according to the mechanical coordinates and the calibration plate coordinates.
6. The image processing method according to claim 5, wherein the determining the calibration plate coordinates of the center position of the sub-image of the calibration plate comprises:
determining a third transformation matrix; the third conversion matrix is a conversion relation between a pixel coordinate system corresponding to the sub-image of the calibration plate and a coordinate system of the calibration plate;
determining pixel coordinates of the central position of the sub-image of the calibration plate;
and determining the calibration plate coordinate of the central position of the sub-image of the calibration plate according to the pixel coordinate of the central position of the sub-image of the calibration plate and the third conversion matrix.
7. The image processing method according to claim 1, characterized in that the image processing method further comprises:
determining a spliced image corresponding to the whole image as a spliced image;
determining whether the stitched image is consistent with the whole image;
and if the spliced image is inconsistent with the whole image, optimizing the first conversion matrix and/or the second conversion matrix.
8. The image processing method according to claim 1, wherein determining the shooting position of the camera according to the pixel coordinates, a predetermined first conversion matrix and a predetermined second conversion matrix comprises:
determining a third conversion matrix according to the first conversion matrix and the second conversion matrix; the third conversion matrix is a coordinate conversion relation between a pixel coordinate system and a mechanical coordinate system;
determining a converted calibration plate coordinate according to the pixel coordinate and the first conversion matrix;
determining a transformed first mechanical coordinate from the transformed calibration plate coordinate and the second transformation matrix;
determining a second converted mechanical coordinate according to the pixel coordinate and the third conversion matrix;
judging whether the converted first mechanical coordinate and the converted second mechanical coordinate are consistent;
if the converted first mechanical coordinate is consistent with the converted second mechanical coordinate, determining the shooting position according to the first mechanical coordinate and/or the second mechanical coordinate;
if the converted first mechanical coordinate and the converted second mechanical coordinate are not consistent, determining a final mechanical coordinate according to the converted first mechanical coordinate and the converted second mechanical coordinate; and determining the shooting position according to the final mechanical coordinate.
9. An image processing apparatus based on data calibration, comprising:
a processing module to: determining the pixel coordinates of the current image to be shot; the pixel coordinates are used for representing coordinates of the current image to be shot in the corresponding whole image;
determining the shooting position of a camera according to the pixel coordinates and a first conversion matrix and a second conversion matrix which are determined in advance; the first conversion matrix is a coordinate conversion relation between a pixel coordinate system and a calibration plate coordinate system, the second conversion matrix is a coordinate conversion relation between the calibration plate coordinate system and a mechanical coordinate system, and the mechanical coordinate in the mechanical coordinate system is used for representing the shooting position of the camera;
the control module is used for controlling the camera to move to the shooting position so as to enable the camera to shoot the current image to be shot at the shooting position;
the processing module is further used for splicing the shot images into the current spliced image corresponding to the whole image based on the pixel coordinates.
10. An image processing apparatus characterized by comprising:
a processor; and a memory communicatively coupled to the processor;
wherein the memory stores instructions executable by the processor to enable the processor to perform the method of image processing based on data targeting as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210264411.6A CN114612447A (en) | 2022-03-17 | 2022-03-17 | Image processing method and device based on data calibration and image processing equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210264411.6A CN114612447A (en) | 2022-03-17 | 2022-03-17 | Image processing method and device based on data calibration and image processing equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114612447A true CN114612447A (en) | 2022-06-10 |
Family
ID=81864436
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210264411.6A Pending CN114612447A (en) | 2022-03-17 | 2022-03-17 | Image processing method and device based on data calibration and image processing equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114612447A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861782A (en) * | 2023-02-21 | 2023-03-28 | 济南邦德激光股份有限公司 | Excess material identification and typesetting system based on vision |
| CN115861429A (en) * | 2023-02-28 | 2023-03-28 | 深圳思谋信息科技有限公司 | Image acquisition equipment calibration method and device, computer equipment and storage medium |
| CN116067290A (en) * | 2023-03-07 | 2023-05-05 | 西安航天动力研究所 | Displacement testing method and displacement testing system for static test of engine |
| CN117140558A (en) * | 2023-10-25 | 2023-12-01 | 菲特（天津）检测技术有限公司 | Coordinate conversion method, system and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325980A (en) * | 2018-07-27 | 2019-02-12 | 深圳大学 | A kind of method, apparatus and manipulator for manipulator positioning target |
| CN109671122A (en) * | 2018-12-14 | 2019-04-23 | 四川长虹电器股份有限公司 | Trick camera calibration method and device |
| CN110599548A (en) * | 2019-09-02 | 2019-12-20 | Oppo广东移动通信有限公司 | Camera calibration method and device, camera and computer readable storage medium |
| CN110717943A (en) * | 2019-09-05 | 2020-01-21 | 中北大学 | Method and system for calibrating eyes of on-hand manipulator for two-dimensional plane |
| WO2020062434A1 (en) * | 2018-09-30 | 2020-04-02 | 初速度（苏州）科技有限公司 | Static calibration method for external parameters of camera |
| CN111738923A (en) * | 2020-06-19 | 2020-10-02 | 京东方科技集团股份有限公司 | Image processing method, apparatus and storage medium |
| CN111815719A (en) * | 2020-07-20 | 2020-10-23 | 北京百度网讯科技有限公司 | External parameter calibration method, device and equipment of image acquisition equipment and storage medium |
| CN112927306A (en) * | 2021-02-24 | 2021-06-08 | 深圳市优必选科技股份有限公司 | Calibration method and device of shooting device and terminal equipment |
-
2022
- 2022-03-17 CN CN202210264411.6A patent/CN114612447A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109325980A (en) * | 2018-07-27 | 2019-02-12 | 深圳大学 | A kind of method, apparatus and manipulator for manipulator positioning target |
| WO2020062434A1 (en) * | 2018-09-30 | 2020-04-02 | 初速度（苏州）科技有限公司 | Static calibration method for external parameters of camera |
| CN109671122A (en) * | 2018-12-14 | 2019-04-23 | 四川长虹电器股份有限公司 | Trick camera calibration method and device |
| CN110599548A (en) * | 2019-09-02 | 2019-12-20 | Oppo广东移动通信有限公司 | Camera calibration method and device, camera and computer readable storage medium |
| CN110717943A (en) * | 2019-09-05 | 2020-01-21 | 中北大学 | Method and system for calibrating eyes of on-hand manipulator for two-dimensional plane |
| CN111738923A (en) * | 2020-06-19 | 2020-10-02 | 京东方科技集团股份有限公司 | Image processing method, apparatus and storage medium |
| CN111815719A (en) * | 2020-07-20 | 2020-10-23 | 北京百度网讯科技有限公司 | External parameter calibration method, device and equipment of image acquisition equipment and storage medium |
| CN112927306A (en) * | 2021-02-24 | 2021-06-08 | 深圳市优必选科技股份有限公司 | Calibration method and device of shooting device and terminal equipment |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115861782A (en) * | 2023-02-21 | 2023-03-28 | 济南邦德激光股份有限公司 | Excess material identification and typesetting system based on vision |
| CN115861782B (en) * | 2023-02-21 | 2023-06-13 | 济南邦德激光股份有限公司 | Visual-based residue recognition and typesetting system |
| CN115861429A (en) * | 2023-02-28 | 2023-03-28 | 深圳思谋信息科技有限公司 | Image acquisition equipment calibration method and device, computer equipment and storage medium |
| CN115861429B (en) * | 2023-02-28 | 2023-06-16 | 深圳思谋信息科技有限公司 | Image acquisition equipment calibration method, device, computer equipment and storage medium |
| CN116067290A (en) * | 2023-03-07 | 2023-05-05 | 西安航天动力研究所 | Displacement testing method and displacement testing system for static test of engine |
| CN117140558A (en) * | 2023-10-25 | 2023-12-01 | 菲特（天津）检测技术有限公司 | Coordinate conversion method, system and electronic equipment |
| CN117140558B (en) * | 2023-10-25 | 2024-01-16 | 菲特（天津）检测技术有限公司 | Coordinate conversion method, system and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114612447A (en) | Image processing method and device based on data calibration and image processing equipment | |
| US20230345135A1 (en) | Method, apparatus, and device for processing images, and storage medium | |
| CN111612853A (en) | Camera parameter calibration method and device | |
| CN110910459B (en) | Camera device calibration method and device and calibration equipment | |
| JP2003254748A (en) | Stereo image characteristic inspection system | |
| CN109920004B (en) | Image processing method, device, calibration object combination, terminal equipment and calibration system | |
| CN107666546B (en) | Image shooting alignment method and system | |
| CN108805938B (en) | Detection method of optical anti-shake module, mobile terminal and storage medium | |
| CN112562014B (en) | Camera calibration method, system, medium and device | |
| JP3138080B2 (en) | Automatic calibration device for vision sensor | |
| CN111383264A (en) | Positioning method, positioning device, terminal and computer storage medium | |
| CN113989392A (en) | Color chessboard calibration method and device of splicing camera and camera | |
| CN110853102B (en) | Novel robot vision calibration and guide method and device and computer equipment | |
| CN113781576A (en) | Binocular vision detection system, method and device for multi-degree-of-freedom pose real-time adjustment | |
| CN117173254A (en) | Camera calibration method, system, device and electronic equipment | |
| CN112598751A (en) | Calibration method and device, terminal and storage medium | |
| CN115546016A (en) | Method for acquiring and processing 2D (two-dimensional) and 3D (three-dimensional) images of PCB (printed Circuit Board) and related device | |
| CN116012242A (en) | Camera distortion correction effect evaluation method, device, medium and equipment | |
| CN114295331A (en) | Multi-camera module optical axis testing method, device, equipment and medium | |
| CN115239816A (en) | Camera calibration method, system, electronic device and storage medium | |
| CN114466143A (en) | Shooting angle calibration method and device, terminal equipment and storage medium | |
| JP2021068929A (en) | Imaging apparatus and imaging method | |
| CN112734721A (en) | Optical axis deflection angle detection method, device, equipment and medium | |
| CN109379521A (en) | Camera calibration method, device, computer equipment and storage medium | |
| KR102280184B1 (en) | Active alignment system of lens module using phase autofocus pixel |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |