CN106570511B - Keyboard character defect online detection system and method based on computer vision - Google Patents
Keyboard character defect online detection system and method based on computer vision Download PDFInfo
- Publication number
- CN106570511B CN106570511B CN201610943910.2A CN201610943910A CN106570511B CN 106570511 B CN106570511 B CN 106570511B CN 201610943910 A CN201610943910 A CN 201610943910A CN 106570511 B CN106570511 B CN 106570511B
- Authority
- CN
- China
- Prior art keywords
- keyboard
- characters
- image
- character
- computer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/243—Aligning, centring, orientation detection or correction of the image by compensating for image skew or non-uniform image deformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/06—Recognition of objects for industrial automation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
- Character Input (AREA)
Abstract
The invention relates to a keyboard character online detection system and method based on computer vision. The system comprises an industrial camera, a computer, a single chip microcomputer system, a frequency converter, a motor and a conveyor belt. The method comprises the following operation steps: (1) the method comprises the steps of (1) improving image quality by using an image graying and median filtering method, (2) carrying out image matching by using surf feature points, (3) determining character positions, (4) carrying out book order labeling on characters, and (5) separating defective characters. The detection algorithm of the invention adopts an SVM classification method, marks normal characters as positive samples, marks wrong characters as negative samples, and respectively extracts the size, center and moment characteristics of the positive samples and the wrong characters for training. The invention has simple installation and high detection efficiency and can play a great role in the production process of the keyboard.
Description
Technical Field
The invention relates to a method based on a support vector machine, in particular to a keyboard character defect online detection system and method based on computer vision.
Background
In the production process of the keyboard, the keys on the keyboard need to be printed and installed, but in the printing and installing process, the phenomena of printing dirt, installation errors and the like easily occur. In order to strictly control the quality of keyboard production, the quality of printed matters on a keyboard production line needs to be monitored in real time, and unqualified products are removed in time.
In the past, printing enterprises finish the quality detection and control of products by workers, and a skilled worker with abundant experience is a detection system, but the quality stability cannot be ensured by manual detection, the phenomena of false detection and missing detection often occur, and the manual detection speed is too slow, the cost is too high, and the detection standards are different. In order to overcome the defects and shortcomings of the traditional detection, the method for searching an objective and efficient keyboard printing defect online detection technology is a great trend in the development of the printing industry.
At present, computer vision develops rapidly, and the defect detection is carried out by using a vision processing technology, so that great advantages are highlighted in the industry, the production cost is reduced, and the production efficiency is improved. Therefore, the computer vision technology can be applied to the keyboard defect detection to exert great advantages.
Disclosure of Invention
The invention aims to provide a keyboard defect online detection system and method based on computer vision, aiming at the defects in the prior art, and the system and method are combined with an automatic control system to build a set of complete keyboard defect online detection system and method, thereby improving the efficiency in the production process and ensuring the production quality.
In order to achieve the purpose, the invention has the following conception:
1. a detection system with a computer system as a core is established, a signal for controlling the running speed of the conveyor belt is sent to the single chip microcomputer through the computer, the single chip microcomputer is converted into an analog signal through D/A (digital/analog) and sent to the frequency converter, the running speed of the conveyor belt is further controlled, and meanwhile the computer controls the exposure time interval of the CCD camera.
2. And the image acquisition system acquires the image acquired by the industrial camera in a network connection mode.
3. And (3) image preprocessing, namely performing graying, binaryzation and noise reduction on the shot picture, removing irrelevant information and ensuring the subsequent processing time.
4. And (3) extracting the keyboard, wherein the position of the keyboard in the picture is found by the shot picture through a template matching method, so that the keyboard can be conveniently processed at the later stage.
5. The keycap is extracted, and characters on the keyboard are segmented through binarization processing of the keyboard, so that later-stage character comparison is facilitated.
6. And the keyboard is subjected to characteristic point matching with the template picture to obtain a transformation matrix, so that the keyboard can be transformed to the state with the same position as the template picture, and the accuracy of later-stage comparison is improved.
7. And (4) feature extraction, namely extracting the features of the perimeter, the area, the gravity center and the moment of the characters on the keyboard to serve as the features of the characters on the keyboard.
8. And defect comparison, namely inputting the extracted features into a support vector machine classifier for training to obtain a trained classifier, and inputting the picture to be detected into the classifier to detect the defective keyboard during detection.
According to the invention concept, the invention adopts the following technical scheme that the keyboard character defect online detection system based on computer vision comprises an industrial camera (1), a computer (2), a singlechip system (3), a frequency converter (4), a motor (5) and a conveyor belt (6), and is characterized in that: the industrial camera (1) is aligned with a keyboard (7) on the conveyor belt (6) for shooting, the output of the industrial camera is connected to the computer (2), the computer (2) is connected with the single chip microcomputer system (3), the output of the single chip microcomputer (3) is connected with the motor (5) through the frequency converter (4), and the motor (5) drives the conveyor belt (6) to carry out shooting; the method comprises the steps of acquiring a keyboard character image by using an industrial camera (1), detecting image defects by using a computer vision SVM classification method, and judging the production quality of a keyboard in the production process. The method comprises the following specific steps:
firstly, a computer (2) is used for communicating with a serial port of a singlechip msp430 to control the speed of a conveyor belt (6);
secondly, the image quality is improved by using the methods of image graying and median filtering;
thirdly, image matching is carried out by using surf characteristic points, and the inclined image is rotated to be in the same level with the template picture;
extracting the outlines of the characters by using an eight-adjacency method, calculating the minimum circumscribed rectangles of the outlines of the characters by acquiring the outlines of the characters, and determining the positions of the characters;
calculating the coordinates of the circumscribed rectangle, and carrying out sequential labeling on characters;
extracting the character size, center and moment characteristics, training by using an SVM classifier, and separating the normal keyboard characters from the defective characters.
According to the inventive concept, the invention adopts the following technical scheme:
the utility model provides a keyboard character defect on-line measuring system based on computer vision, includes industry camera, computer, single chip microcomputer system, converter, motor and conveyer belt, its characterized in that: the industrial camera is aligned with a keyboard on the conveyor belt for shooting, the output of the industrial camera is connected to a computer, the computer is connected with a single chip microcomputer system, the output of the single chip microcomputer is connected with a motor through a frequency converter, and the motor drives the conveyor belt to run; acquiring a keyboard character image by using an industrial camera, detecting image defects by adopting a computer vision SVM classification method, and judging the production quality of a keyboard in the production process; and a computer is used for communicating with the serial port of the singlechip msp430 to control the speed of the conveyor belt.
A keyboard character defect online detection method based on computer vision adopts the system for operation, and is characterized by comprising the following specific operation steps:
firstly, the image quality is improved by using an image graying and median filtering method;
secondly, image matching is carried out by using surf characteristic points, and the inclined image is rotated to be in the same level with the template picture;
extracting outlines of the characters by using an eight-adjacency method, calculating minimum circumscribed rectangles of the outlines of the characters by acquiring the outlines of the characters, and determining the positions of the characters;
fourthly, calculating the coordinates of the circumscribed rectangle, and carrying out sequential labeling on the characters;
extracting the character size, center and moment characteristics, training by using an SVM classifier, and separating the normal keyboard characters from the defective characters.
In the step (2), surf feature points are used for image matching, and in order to rotate the inclined picture on the conveyor belt into the angle of the template picture, the specific method is as follows:
the surf characteristics of the template image and the keyboard image shot on the conveyor belt are respectively extracted, and the obtained characteristic point is a 64-dimensional vector.
Secondly, traversing surf characteristic points on the two images respectively, calculating Euclidean distances of the surf characteristic points, finding out a point with the minimum distance, namely a point with the well matched two images, and solving a single mapping matrix H by using the well matched point.
And thirdly, affine change is carried out on the keyboard picture on the conveyor belt by using the single mapping matrix H to obtain a corrected image.
And (5) training by using an SVM classifier, respectively taking 2000 normal pictures as positive samples, respectively taking 2000 defective pictures as negative samples, respectively extracting the size, the gravity center, the perimeter and the moment characteristics of the positive samples and the negative samples, respectively taking the size, the gravity center, the perimeter and the moment characteristics as the input of the SVM, training, and detecting the defects by using the obtained SVM classifier.
Compared with the prior art, the invention has the following obvious and prominent substantive characteristics and remarkable technical progress:
1. when the image correction is carried out, the surf characteristics are adopted for carrying out characteristic matching, the mapping relation between the image to be detected and the template image is solved, the surf characteristics are used for guaranteeing the real-time performance under the condition of guaranteeing stability, and the correction of the image to be detected can be quickly and accurately completed.
2. In order to obtain the position of each character of the image to be detected and the template image, the text obtains a preliminarily segmented character region by a mathematical morphology method and an eight-neighborhood contour detection method. And a designed sorting algorithm is used to obtain an interested area, a certain index is arranged on a key, and the characters at the same position can be accurately compared in each detection.
3. During character comparison, the characteristics of keyboard characters can be effectively expressed by extracting the size, center and moment characteristics of the characters, and the defects of samples can be effectively overcome by training by utilizing an SVM classifier, so that the normal keyboard characters are separated from the defect characters.
Drawings
Fig. 1 is a schematic view of the overall structure of the present invention.
Fig. 2 is a block diagram of the overall process of the present invention.
Fig. 3 is a block diagram of an image correction process.
Fig. 4 is a block diagram of a character outline tracing program.
Fig. 5 is a block diagram of the software according to the third embodiment of the present invention.
FIG. 6 is a block diagram of a process for constructing an SVM classifier.
FIG. 7 is a schematic diagram of the calibration matching of the present invention.
Fig. 8 is a schematic diagram after correction.
Fig. 9 is a schematic of morphological processing.
FIG. 10 is a schematic view of character positioning.
FIG. 11 is a schematic diagram of character ordering.
Detailed Description
The preferred embodiments of the present invention are briefly described below with reference to the accompanying drawings:
the first embodiment is as follows:
referring to fig. 1, keyboard character defect on-line measuring system based on computer vision, including industry camera (1), computer (2), single chip microcomputer system (3), converter (4), motor (5) and conveyer belt (6), its characterized in that: the industrial camera (1) is aligned with a keyboard (7) on the conveyor belt (6) for shooting, the output of the industrial camera is connected to the computer (2), the computer (2) is connected with the single chip microcomputer system (3), the output of the single chip microcomputer (3) is connected with the motor (5) through the frequency converter (4), and the motor (5) drives the conveyor belt (6) to run; acquiring a keyboard character image by using an industrial camera (1), detecting image defects by adopting a computer vision SVM classification method, and judging the production quality of a keyboard in the production process; the computer (2) is used for communicating with the serial port of the singlechip msp430 to control the speed of the conveyor belt (6).
Example two:
referring to fig. 2 and 11, the online detection method for keyboard character defects based on computer vision, which is operated by the online detection system for keyboard character defects based on computer vision according to claim 1, is characterized by comprising the following specific operation steps:
improving the quality of the image by using a method of image graying and median filtering;
seventhly, matching images by using surf characteristic points, and rotating the inclined images to be at the same level with the template picture;
extracting outlines of the characters by using an eight-adjacency method, calculating minimum circumscribed rectangles of the outlines of the characters by acquiring the outlines of the characters, and determining the positions of the characters;
ninthly, calculating the coordinates of the circumscribed rectangle, and labeling the characters in sequence;
and extracting the characteristics of the size, the center and the moment of the characters, and training by using an SVM classifier to separate the normal keyboard characters from the defective characters.
The keyboard character defect online detection method based on computer vision is characterized in that in the step (2), surf feature points are used for image matching, and in order to rotate an inclined picture on a conveyor belt into an angle of a template picture, the specific method is as follows:
and fourthly, extracting surf characteristics of the template image and the keyboard image shot on the conveyor belt respectively, wherein the obtained characteristic point is a 64-dimensional vector.
Fifthly, traversing surf characteristic points on the two images respectively, calculating Euclidean distances of the surf characteristic points, finding out a point with the minimum distance, namely a point with the well matched two images, and solving a single mapping matrix H by using the well matched point.
And sixthly, carrying out affine change on the keyboard picture on the conveyor belt by using the single mapping matrix H to obtain a corrected image.
And (5) training by using an SVM classifier, respectively taking 2000 normal pictures as positive samples, respectively taking 2000 defective pictures as negative samples, respectively extracting the size, the gravity center, the perimeter and the moment characteristics of the positive samples and the negative samples, respectively taking the size, the gravity center, the perimeter and the moment characteristics as the input of the SVM, training, and detecting the defects by using the obtained SVM classifier.
Example three:
the method for detecting the keyboard defects on line based on computer vision not only has high detection accuracy, but also has high detection speed. The embodiment can be divided into two stages of hardware design and software design, and the detailed steps are as follows:
1. hardware design (see figure 1)
Fig. 1 is a hardware frame of the system, and the detection system mainly comprises a computer, an msp430 single chip microcomputer and a 500-ten-thousand-pixel industrial camera. Since the CCD camera and the conveyor belt are independently operated, the exposure time of the CCD camera must be determined so that the keyboard can be photographed by the camera each time, so that the exposure time must be calculated.
The distance between two adjacent keyboards is set to be delta s, the running speed of the conveyor belt is set to be v, the exposure time is set to be delta t, the exposure time of the camera can be obtained by delta s being delta t multiplied by v, and therefore the camera can completely shoot the keyboards every time only by means of serial port communication of a computer and the msp430 single-chip microcomputer and sending of a control command.
In order to control the running speed v of the conveyor belt, the single chip microcomputer converts the analog signals into analog signals through the D/A module and sends the analog signals to the frequency converter, and then the running speed v of the conveyor belt is controlled.
2. Software design (see FIG. 5)
2.1 image preprocessing
(1) Gray scale processing
The color image is converted into a gray image, the color image is composed of RGB channels, and the gray processing mainly calculates the RGB value of each pixel point of the color image into a gray value. The formula is as follows:
gray＝0.11R+0.59G+0.30B(2.1)
(2) filtering process
The purpose of image filtering is to reduce the noise of the keyboard image, which comes from various causes. In the process of keyboard production, because keyboard button depressed place and corner reflect light more strongly, the image in these places is easily whitened to form miscellaneous point, the normal segmentation process of interference image, need filter. We performed experiments using mean filtering and median filtering, respectively, with filter sizes of 3 x 3,5 x 5,7 x 7. Through experimental comparison, the method can effectively filter out the miscellaneous points in the image by using a 7-by-7 median filtering mode.
(3) Binarization method
The binary image can highlight the target pixel in the background, so that the contrast of the image is enhanced, and the complexity and time of subsequent processing can be reduced. Since the illumination condition is almost fixed during the keyboard detection, we only need to set a threshold value during the transformation, and the pixel at the point is greater than the threshold value by 1 and less than the threshold value by 0. The binarization formula is as follows:
2.2 image Tilt correction
When the keyboard is produced on a production line, the positions of the keyboard and the keyboard cannot be ensured to be always in the horizontal direction, so that inclination can be caused, if the inclination angle is large, the recognition of defective characters can be seriously influenced, and therefore, an image needs to be corrected according to the actual production condition. We used a surf-based matching algorithm for the correction, the matching is shown in fig. 2, and the corrected graph is shown in fig. 3.
Let p be (x)1,y1) The coordinates of a point in the image to be corrected, q ═ x2,y2) If the coordinates of the points in the corrected image are the coordinates, an affine transformation relationship exists:
then the affine matrix needs to be found, and let the affine matrix be:
in order to obtain the affine matrix, the positions of p and q in the images are required to be obtained, surf characteristics of the image to be rotated and the template image are respectively extracted, a matching pair of the two images is obtained by a nearest neighbor matching method, and the affine matrix between the images is obtained by combining RANSAC and a least square method.
2.3 keycap extraction
In order to compare whether the keyboard character has errors, the keyboard character needs to be extracted from the image. Because a large part of the keyboard image is irrelevant background, the method is not helpful for carrying out defect character recognition.
(1) Morphological treatment
In order to make a key have only one interesting area, we need to do the processing by morphological method. The basic idea is to use a morphological structure element to measure and extract the corresponding shape in the graph for image analysis and recognition purposes. Its basic operations are 4: expansion, corrosion, open and close operations, which are combined to achieve that only one region of interest exists on one key. We have designed such a process flow experimentally: expansion, closing, opening operation and expansion. The result is shown in fig. 4, which thus obtains the region of interest while ensuring that there is only one region of interest on the same key.
(2) Contour extraction
By the morphological method, the size of the character can be preliminarily determined, but the position of the character region cannot be determined yet. The position of the character can be determined by extracting the outlines of the regions and solving the minimum bounding rectangle of the regions, and the region surrounded by the minimum bounding rectangle is the region of interest.
Here we take an eight-adjacency approach to contour extraction. The algorithm is as follows: for any pixel P, the 8 surrounding pixels are P1, P2, P3, P4, P5, P6, P7, and P8, respectively. On the binary image, the point P is sent out, whether 8 pixels around the point P are 1 or not is searched, if all the pixels are 1, the point P is not a contour, if not all the pixels are 1, the point P is a contour, at the moment, the point P is started clockwise, a pixel point 1 is found, whether the pixel point is a contour or not is judged, recursion is continued until the whole image is searched, and therefore the contour of the target area can be extracted.
(3) Character sorting
By the contour extraction method, the position of each character of the keyboard in the picture is obtained, as shown in fig. 5. However, the order of these positions is not certain, such as the character 'a', and the character cannot be determined to be the fourth position, so we must sort the extracted character positions to determine the position of each character. The sequencing algorithm steps are described as follows:
a. traversing all the character outline sets, selecting the first character in the set, calculating the ordinate of the minimum circumscribed rectangle of the first character, continuously traversing the outlines of the rest characters, and judging whether the ordinates of the minimum circumscribed rectangles are intersected or not. If the coordinate ranges intersect, the characters are in the same row on the keyboard, and the row mark of the characters can be determined.
b. And for each character row, calculating the center position of the characters, sorting according to the abscissa of the center position of the characters, and finishing the sorting from left to right. Each character is identified by a label, and the result is shown in fig. 11.
2.4 character feature extraction
The image feature extraction is to extract the character of the letter from the result image after the image processing is finished, firstly, the establishment of the feature has a certain principle, and the feature is to well describe the letter. The difference of the characters can not be effectively seen by only using one feature for the keyboard characters, so that a plurality of features can be extracted to achieve the aim.
(1) Character size
In the character defect detection, when a character has a defect, such as a character is too long or short, the defect can be reflected by the length and height of the character. And obtaining the outline of the character by an outline extraction algorithm, and solving the minimum circumscribed rectangle of the outline, wherein the length and the width of the minimum circumscribed rectangle are the size of the character.
(2) Center of gravity of character
The letter barycenter is a statistical value of all '1' in the binary image, and the gray value of each character at the corresponding position is regarded as the quality of the character at the position, so that the overall quality and the barycenter position of the character can be calculated, and if the character is defective, the change of the barycenter position can be reflected. Assuming an image M × N binarized image, where f (x, y) is 0 or f (x, y) is 1, and the number of pixels in the region is k, the positions of all the points in the statistical set P are counted, and the average is the center of gravity, and the formula is as follows:
(3) character moment features
In consideration of the problem that the keyboard can cause pixel dislocation or different sizes of the image to be detected and the template image in the region of interest in the production process, the problem can be effectively solved by using the central moment of the character in order to obtain the translation, rotation and scale invariance of the character. For a digital image I (x, y), the central moment of p + q of region R is:
although the central moment features do not change after the image is translated and scaled, they change after rotation, and so must be done by combining the central moments. These central moments are combined as follows:
H1＝μ20+μ02 (2.7)
H2＝(μ20-μ02)2+4μ11 2 (2.8)
H3＝(μ30-3μ12)2+(3μ21-μ03)2 (2.9)
H4＝(μ30+μ12)2+(μ21+μ03)2 (2.10)
H5＝(μ30-3μ12)·(μ30+μ12)·[(μ30+μ12)2-3(μ21+μ03)2]
+(3μ21-μ03)·(μ21+μ03)·[3(μ30+μ12)2-(μ21+μ03)2] (2.11)
H6＝(μ20-μ02)·[(μ30+μ12)2-(μ21+μ03)2]+4μ11·(μ30+μ12)·(μ21+μ03) (2.12)
H7＝(3μ12-μ03)·(μ30+μ12)·[(μ30+μ12)2-3(μ21+μ03)2]
+(3μ21-μ30)·(μ21+μ03)·[3(μ30+μ12)2-(μ21+μ03)2] (2.13)
when the character is defective, the image to be detected and the template image will generate difference on Hu invariant moment value, and the defective character H is found through experiments1And H2The difference from normal characters is large, so we only use H1And H2As a feature.
2.5 support vector machine classifier design
The support vector machine is based on VC dimension theory of statistical learning theory and structure risk minimum principle, is an efficient machine learning method aiming at finding optimal classification surface, and has a plurality of advantages in solving small sample, nonlinear and high dimension pattern recognition. The SVM is selected to train the samples.
And marking the normal keyboard characters as positive samples, marking the defective keyboard characters as negative samples, respectively extracting the features of the normal keyboard characters, the defective keyboard characters, the. And finally obtaining a trained model, and using the trained model for detecting the keyboard defects.
Claims (4)
1. The utility model provides a keyboard character defect on-line measuring system based on computer vision, includes industry camera (1), computer (2), single chip microcomputer system (3), converter (4), motor (5) and conveyer belt (6), its characterized in that: the industrial camera (1) is aligned with a keyboard (7) on the conveyor belt (6) for shooting, the output of the industrial camera is connected to the computer (2), the computer (2) is connected with the single chip microcomputer system (3), the output of the single chip microcomputer system (3) is connected with the motor (5) through the frequency converter (4), and the motor (5) drives the conveyor belt (6) to run; acquiring a keyboard character image by using an industrial camera (1), detecting image defects by adopting a computer vision SVM classification method, and judging the production quality of a keyboard in the production process; the computer (2) is used for communicating with the msp430 serial port of the singlechip system (3) to control the speed of the conveyor belt (6); when image data processing is carried out, inclination correction is carried out on the keyboard, a transformation matrix is obtained by matching feature points with the template picture, the keyboard is transformed to a state with the same position as the template picture, and then the characteristic extraction of the keyboard image information is carried out; setting the distance between two adjacent keyboards as deltas, the running speed of a conveyor belt as v and the exposure time as deltat, obtaining the exposure time of the camera from the deltas as deltat multiplied by v, communicating with a msp430 serial port of a singlechip system (3) through a computer (2), and sending a control command to ensure that the camera can completely shoot the keyboards each time;
according to the keyboard character defect online detection system based on computer vision, the keyboard character defect online detection method based on computer vision specifically comprises the following operation steps:
firstly, the image quality is improved by using an image graying and median filtering method;
secondly, image matching is carried out by using surf characteristic points, and the inclined image is rotated to be in the same level with the template picture;
extracting outlines of the characters by using an eight-adjacency method, calculating minimum circumscribed rectangles of the outlines of the characters by acquiring the outlines of the characters, and determining the positions of the characters;
fourthly, calculating the coordinates of the circumscribed rectangle, and carrying out sequential labeling on the characters;
extracting the character size, the gravity center and the moment characteristics as the input of an SVM, training by using an SVM classifier, and separating the normal keyboard characters from the defective characters;
calculating the coordinates of the circumscribed rectangles, labeling the characters in sequence, selecting the first character in the set by traversing all character outline sets, calculating the ordinate of the minimum circumscribed rectangle in which the first character is located, continuously traversing the outlines of the rest characters, judging whether the ordinate of the minimum circumscribed rectangle is intersected or not, and if the coordinate ranges are intersected, the characters are located in the same line on the keyboard, so that the line marks of the characters can be determined.
2. The computer vision based keyboard character defect online detection system of claim 1, wherein
The step II is to use surf characteristic points to carry out image matching, and the purpose is to rotate the inclined picture on the conveyor belt into the angle of the template picture, and the specific method is as follows:
(2.1) extracting surf characteristics of the template image and the keyboard image shot on the conveying belt respectively, wherein the obtained characteristic point is a 64-dimensional vector;
(2.2) traversing surf characteristic points on the two images respectively, calculating Euclidean distances of the surf characteristic points, finding out a point with the minimum distance, namely a point with the matched two images, and solving a single mapping matrix H by using the matched point;
and (2.3) carrying out affine change on the keyboard picture on the conveyor belt by using the single mapping matrix H to obtain a corrected image.
3. The on-line detection system for keyboard character defects based on computer vision of claim 1, characterized in that said step (c) uses eight-adjacency method to extract the outline of the character, the method is as follows: each time a foreground pixel is hit, the scanning position returns to finding the last background pixel of the pixel, then 8 pixels around the foreground pixel are checked clockwise until the next foreground pixel is found, the algorithm ends when the starting point pixel is found for the second time, and the foreground pixels that pass through during tracking constitute the contour.
4. The system of claim 1, wherein the five steps are trained by using an SVM classifier, wherein 2000 normal pictures are respectively marked as positive samples, 2000 defective pictures are respectively marked as negative samples, and their sizes, center of gravity, perimeter, and moment characteristics are respectively extracted as input of SVM, training is performed, and the obtained SVM classifier is used for defect detection.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610943910.2A CN106570511B (en) | 2016-11-02 | 2016-11-02 | Keyboard character defect online detection system and method based on computer vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610943910.2A CN106570511B (en) | 2016-11-02 | 2016-11-02 | Keyboard character defect online detection system and method based on computer vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106570511A CN106570511A (en) | 2017-04-19 |
| CN106570511B true CN106570511B (en) | 2021-04-09 |
Family
ID=58536409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610943910.2A Active CN106570511B (en) | 2016-11-02 | 2016-11-02 | Keyboard character defect online detection system and method based on computer vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106570511B (en) |
Families Citing this family (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107328793B (en) * | 2017-06-30 | 2019-12-03 | 航天新长征大道科技有限公司 | A kind of jewelry surface word print flaw detection method and device based on machine vision |
| CN107367514A (en) * | 2017-08-28 | 2017-11-21 | 广东工业大学 | A kind of POS defect inspection method based on machine vision |
| CN108416765B (en) * | 2018-01-30 | 2020-08-18 | 华南理工大学 | Method and system for automatically detecting character defects |
| CN108548820B (en) * | 2018-03-28 | 2023-08-15 | 浙江理工大学 | Cosmetic paper label defect detection method |
| CN108647706B (en) * | 2018-04-24 | 2022-03-22 | 广州大学 | Article identification classification and flaw detection method based on machine vision |
| CN110706195A (en) * | 2018-06-22 | 2020-01-17 | 技嘉科技股份有限公司 | Keyboard detection method and system |
| CN109087286A (en) * | 2018-07-17 | 2018-12-25 | 江西财经大学 | A kind of detection method and application based on Computer Image Processing and pattern-recognition |
| CN109211136A (en) * | 2018-08-31 | 2019-01-15 | 广州大学 | A kind of Watch glass cover board profile tolerance detection method |
| CN109738450B (en) * | 2019-01-09 | 2021-06-29 | 合肥联宝信息技术有限公司 | Method and device for detecting notebook keyboard |
| CN109871895B (en) * | 2019-02-22 | 2021-03-16 | 北京百度网讯科技有限公司 | Method and device for detecting defects of circuit board |
| CN110363250A (en) * | 2019-07-23 | 2019-10-22 | 北京隆普智能科技有限公司 | A kind of method and its system of 3-D image intelligent Matching |
| CN110596121A (en) * | 2019-09-12 | 2019-12-20 | 南京旷云科技有限公司 | Keyboard appearance detection method and device and electronic system |
| CN110674876A (en) * | 2019-09-25 | 2020-01-10 | 北京猎户星空科技有限公司 | Character detection method and device, electronic equipment and computer readable medium |
| CN110658194A (en) * | 2019-09-30 | 2020-01-07 | 联想(北京)有限公司 | Keyboard detection method and keyboard detection equipment |
| WO2021109011A1 (en) * | 2019-12-04 | 2021-06-10 | 电子科技大学 | Intelligent capacitor internal defect detection method based on ultrasound image |
| CN111553360B (en) * | 2019-12-12 | 2020-12-18 | 电子科技大学中山学院 | Touch sliding pattern identification product function durability test self-learning method |
| CN112613337A (en) * | 2020-11-07 | 2021-04-06 | 泰州无印广告传媒有限公司 | Keyboard label missing detection platform and method |
| TWI779836B (en) * | 2021-09-15 | 2022-10-01 | 英業達股份有限公司 | Keyboard file verification method based on image processing |
| CN114972825B (en) * | 2022-07-19 | 2022-12-13 | 天津七一二移动通信有限公司 | Method for realizing automatic detection of MMI key icon matrix |
| CN116008294B (en) * | 2022-12-13 | 2024-03-08 | 无锡微准科技有限公司 | Key cap surface particle defect detection method based on machine vision |
| CN118505688B (en) * | 2024-07-17 | 2024-09-27 | 深圳市捷牛智能装备有限公司 | Method, device, equipment and storage medium for detecting images of keyboard screen printing |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103134803A (en) * | 2011-11-25 | 2013-06-05 | 西安中科麦特电子技术设备有限公司 | PCB optical detection system control circuit |
| CN103323457A (en) * | 2013-05-20 | 2013-09-25 | 中国农业大学 | Fruit appearance defect real-time on-line detection system and detection method |
| CN103592310A (en) * | 2013-11-22 | 2014-02-19 | 昆山视杰维光电科技有限公司 | System for detecting keyboard |
| CN104568986A (en) * | 2015-01-26 | 2015-04-29 | 中国科学院半导体研究所 | Method for automatically detecting printing defects of remote controller panel based on SURF (Speed-Up Robust Feature) algorithm |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20150086933A (en) * | 2014-01-21 | 2015-07-29 | 에스케이하이닉스 주식회사 | Semiconductor memory device |
-
2016
- 2016-11-02 CN CN201610943910.2A patent/CN106570511B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103134803A (en) * | 2011-11-25 | 2013-06-05 | 西安中科麦特电子技术设备有限公司 | PCB optical detection system control circuit |
| CN103323457A (en) * | 2013-05-20 | 2013-09-25 | 中国农业大学 | Fruit appearance defect real-time on-line detection system and detection method |
| CN103592310A (en) * | 2013-11-22 | 2014-02-19 | 昆山视杰维光电科技有限公司 | System for detecting keyboard |
| CN104568986A (en) * | 2015-01-26 | 2015-04-29 | 中国科学院半导体研究所 | Method for automatically detecting printing defects of remote controller panel based on SURF (Speed-Up Robust Feature) algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 自动光学检测中的键盘缺陷字符关键检测技术研究;管兴坚;《中国优秀硕士学位论文全文数据库 信息科技辑》;20150215;第1,9,13,22-26,33-34,40,44,56-59页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106570511A (en) | 2017-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106570511B (en) | Keyboard character defect online detection system and method based on computer vision | |
| CN109785316B (en) | Method for detecting apparent defects of chip | |
| CN109816644B (en) | Bearing defect automatic detection system based on multi-angle light source image | |
| CN108982508B (en) | Plastic package IC chip defect detection method based on feature template matching and deep learning | |
| CN104992449B (en) | Information identification and surface defect online test method based on machine vision | |
| CN105046196B (en) | Front truck information of vehicles structuring output method based on concatenated convolutional neutral net | |
| CN105046252B (en) | A kind of RMB prefix code recognition methods | |
| LeBourgeois | Robust multifont OCR system from gray level images | |
| CN111650220A (en) | Vision-based image-text defect detection method | |
| CN111242896A (en) | Color printing label defect detection and quality rating method | |
| CN111539330B (en) | Transformer substation digital display instrument identification method based on double-SVM multi-classifier | |
| KR20130105952A (en) | Method and apparatus for vehicle license plate recognition | |
| CN108647706A (en) | Article identification classification based on machine vision and flaw detection method | |
| CN109946304A (en) | Surface defects of parts on-line detecting system and detection method based on characteristic matching | |
| CN106874942B (en) | Regular expression semantic-based target model rapid construction method | |
| CN106097368A (en) | A kind of recognition methods in veneer crack | |
| CN118014933A (en) | Defect detection and identification method and device based on image detection | |
| CN113989308B (en) | Polygonal target segmentation method based on Hough transformation and template matching | |
| Budianto | Automatic License Plate Recognition: A Review with Indonesian Case Study | |
| Gui et al. | A fast caption detection method for low quality video images | |
| CN110046618B (en) | License plate recognition method based on machine learning and maximum extremum stable region | |
| CN116863463A (en) | Egg assembly line rapid identification and counting method | |
| CN108985294B (en) | Method, device and equipment for positioning tire mold picture and storage medium | |
| Gao et al. | Solder joint defect detection based on image segmentation and deep learning | |
| CN107122783B (en) | Method for quickly identifying assembly connector based on angular point detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |