CN111753692B - Target object extraction method, product detection method, device, computer and medium - Google Patents
Target object extraction method, product detection method, device, computer and medium Download PDFInfo
- Publication number
- CN111753692B CN111753692B CN202010542166.1A CN202010542166A CN111753692B CN 111753692 B CN111753692 B CN 111753692B CN 202010542166 A CN202010542166 A CN 202010542166A CN 111753692 B CN111753692 B CN 111753692B
- Authority
- CN
- China
- Prior art keywords
- image
- target object
- target
- contour
- training
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 62
- 238000000605 extraction Methods 0.000 title claims abstract description 44
- 238000012549 training Methods 0.000 claims abstract description 94
- 238000000034 method Methods 0.000 claims description 54
- 230000011218 segmentation Effects 0.000 claims description 37
- 238000012545 processing Methods 0.000 claims description 32
- 238000004590 computer program Methods 0.000 claims description 27
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 239000003086 colorant Substances 0.000 claims description 8
- 238000012216 screening Methods 0.000 claims description 8
- 230000007547 defect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 239000000284 extract Substances 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 238000003709 image segmentation Methods 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a target object extraction method, a product detection method, a device, a computer and a medium, wherein the target object extraction method comprises the steps of obtaining an image; inputting the image into a pre-trained training model for feature detection to obtain feature information of the image; obtaining a region of interest of the image from the image based on the feature information; obtaining a coverage area of the target object in the image according to the region of interest; acquiring a target outline of the target object in the image based on the coverage area of the target object in the image; and extracting the target object from the image according to the target outline in the image. And carrying out feature detection on the image through a training model to obtain feature information of the image, determining an interested region in the image, obtaining the position of the target object in the image, and obtaining the outline of the target object in the image so as to extract the target object from the image. The automatic extraction of the target object in the image is realized, so that the extraction of the refrigerator panel is more accurate, and the efficiency is improved.
Description
Technical Field
The present invention relates to the field of image recognition technologies, and in particular, to a target object extraction method, a product detection device, a computer, and a medium.
Background
In the field of modern manufacturing industry production, image registration is a key link in product quality detection based on computer vision, and the quality detection speed and effect are directly influenced by the image registration effect. The configuration effect is largely dependent on the choice of the feature region, so that it is particularly important to select a desired feature region. At present, in refrigerator panel quality detection, mostly select the characteristic region through manual or semi-automatic mode, the manual selection is wasted time and energy and inefficiency, and the manual selection characteristic region is different from person to person simultaneously, and is not objective enough, influences the efficiency and the detection precision of current detecting system.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a target object extraction method, a product detection method, a device, a computer, and a medium.
An image target object extraction method, comprising:
acquiring an image;
Inputting the image into a pre-trained training model for feature detection to obtain feature information of the image;
Selecting a preset number of regions of interest for each piece of characteristic information;
Performing binary classification of foreground and background on the image containing the region of interest to obtain pixel values of the foreground and the background, classifying the image based on a preset pixel threshold value, and obtaining a coverage area of a target object in the image from the region of interest;
acquiring a target contour of the target object in the image based on the coverage area of the target object in the image;
and extracting the target object from the image according to the target outline in the image.
In one embodiment, before the step of acquiring an image, further comprises:
Acquiring a training image;
Performing contour calibration on a target object of the training image to generate a training sample, wherein the training sample is an image file calibrated with the contour of the target object;
and inputting the training samples into a convolutional neural network for learning to obtain the training model containing the characteristic information of the target object of each training image.
In one embodiment, the step of selecting a preset number of regions of interest for each of the feature information includes:
Selecting a preset number of candidate interested areas for each piece of characteristic information;
And carrying out binary classification of foreground and background on the candidate regions of interest, and screening out the regions of interest of the image from a preset number of candidate regions of interest.
In one embodiment, the step of obtaining the target contour of the target object in the image based on the coverage area of the target object in the image comprises:
And extracting a target contour of the target object from the image by adopting a target detection method and a threshold segmentation method based on the coverage area of the target object in the image.
In one embodiment, the step of extracting the target object from the image according to a target contour in the image comprises:
performing edge processing on the extracted target contour to obtain edge position information of the target contour;
And extracting the target object from the image based on the edge position information of the target contour.
In one embodiment, the step of performing edge processing on the extracted target contour to obtain edge position information of the target contour includes:
Obtaining a mask image containing black and white two colors of the image based on the target contour;
And carrying out edge processing on the black-and-white mask image by adopting a Canny detection algorithm to obtain the edge position information of the target contour.
In one embodiment, the step of extracting the target object from the image based on edge position information of the target contour includes:
Acquiring gray values of all pixel points in the image by adopting a threshold segmentation method;
Comparing the gray value of each pixel point in the image with a preset gray threshold value, and performing binarization processing on the comparison result of the gray value of each pixel point in the image and the preset gray threshold value to obtain a binarized image;
and extracting the target object from the binarized image based on the edge position information of the target contour.
A method of product detection comprising:
the method for extracting an image target object according to any of the embodiments described above extracts a target object in a product image, obtains image information of the target object, and determines whether the product meets a preset requirement according to the image information of the target object.
An image target object extraction apparatus comprising:
The image acquisition module is used for acquiring images;
The feature information obtaining module is used for inputting the image into a pre-trained training model to perform feature detection and obtaining feature information of the image;
the interested region obtaining module is used for selecting a preset number of interested regions for each piece of characteristic information;
the target position information obtaining module is used for carrying out binary classification on the foreground and the background of the image containing the region of interest to obtain pixel values of the foreground and the background, classifying the image based on a preset pixel threshold value and obtaining a coverage area of a target object in the image from the region of interest;
a target contour obtaining module, configured to obtain a target contour of the target object in the image based on a coverage area of the target object in the image;
and the target object extraction module is used for extracting the target object from the image according to the target outline in the image.
The method comprises the steps of carrying out feature detection on an image based on a semantic segmentation technology through a pre-trained training model to obtain feature information of the image, further determining a region of interest in the image, positioning and obtaining the position of a target object in the image, obtaining the outline of the target object in the image based on the position in the image, and extracting the target object from the image according to the outline. The automatic extraction of the target object in the image is realized without manual selection, so that the extraction of the refrigerator panel is more accurate, more rapid and convenient, the time is effectively saved, the labor is reduced, and the efficiency is improved.
Drawings
FIG. 1A is a flow diagram of a method for extracting an image target object in one embodiment;
FIG. 1B is a flowchart of a method for extracting an image target object according to another embodiment;
FIG. 1C is a flow chart of a method for extracting an image target object in yet another embodiment;
FIG. 1D is a flow chart of a method for extracting an image target object in yet another embodiment;
FIG. 2 is a block diagram of an image target object extraction apparatus in one embodiment;
FIG. 3 is an internal block diagram of a computer device in one embodiment;
FIG. 4 is a flow diagram of a training process for training a model in one embodiment;
FIG. 5 is a flow diagram of an image target object extraction process in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
It should be understood that the method for extracting the image target object in the present application can be applied to the extraction of the panel of the refrigerator, and can also be applied to the extraction of the panel in other manufacturing products, or in other scenes where an image of an object needs to be extracted in an image, each scene is not described in detail in the present application, and in the following embodiments, the method for extracting the image target object is applied to the extraction of the panel of the refrigerator for further description. It should be understood that the target object extracted in the present application refers to an image or a graphic of the target object in the image.
In one embodiment, as shown in fig. 1A, an image target object extraction method is provided,
At step 110, an image is acquired.
Specifically, the image is an image of the target object to be extracted. For example, the image is an image of a refrigerator, and the image of the refrigerator includes a refrigerator panel, so the target object is a refrigerator panel, and the target object is an image of a panel of the refrigerator, and the panel may be a front panel, a rear panel or a side panel.
And 120, inputting the image into a pre-trained training model to perform feature detection, and obtaining feature information of the image.
Specifically, the training model is trained in advance, and can perform feature detection on the features of the image. In this embodiment, the training model uses a branch network of a segmentation deep learning Mask-RCNN framework to perform feature detection on an image, and the training model includes a feature information set of a target object trained in advance. In this embodiment, the training model performs feature detection on the image based on semantic segmentation, and further obtains feature information of the image, so that the feature information of the input image can be detected by inputting the image into the training model.
In one embodiment, step 120 includes preprocessing the image to obtain a preprocessed image; and inputting the preprocessed image into a pre-trained training model for learning, and obtaining the characteristic information of the image.
In the embodiment, irrelevant information in the image is eliminated through preprocessing the image, useful real information is reserved, denoising of the image is realized, the detectability of relevant information is enhanced, and data is simplified. It should be understood that when processing a color image, three channels are needed to process sequentially, which results in a high time consumption, and in this embodiment, only one byte is needed to store a gray value for each pixel of the gray image through the pre-processing graying, and the gray range is 0-255, so that the data amount of the pixels of the image is reduced, which is beneficial to improving the overall processing speed and reducing the data amount required to process.
And 130, selecting a preset number of regions of interest for each piece of characteristic information.
In this step, at least one region of interest is selected for each feature information in the image. The region of interest is a selected one of the image regions. The selection mode can be selected according to the number of the characteristic information, or according to the position of the characteristic information, or manually. It should be understood that, for selecting the region of interest for the feature information, the implementation may be performed by using the prior art, which is not described in detail in this embodiment.
And 140, performing binary classification of the foreground and the background on the image containing the region of interest to obtain pixel values of the foreground and the background, classifying the image based on a preset pixel threshold value, and obtaining the coverage area of the target object in the image from the region of interest.
Specifically, after the binary classification of the foreground and the background is performed on the image, obtaining pixel values of the foreground and the background of the image, obtaining a preset pixel threshold value, classifying each region of interest based on the pixel threshold value, and selecting a coverage area of the target object in the image from the region of interest by comparing the pixel values of the foreground and the pixel values of the background with the preset pixel threshold value, wherein the pixels of the foreground and the background are converted into gray values due to the binarization processing of the foreground and the background, and dividing the pixel values larger than the preset pixel threshold value into the target object and dividing the pixel values smaller than the preset pixel threshold value into the background based on the comparison of the gray values and the preset pixel threshold value. In the step, the region of interest is classified, and the position region of the target object in the image is located and searched, namely, the position information of the target object in the image is obtained. Specifically, according to the difference of pixel values of the foreground and the background of the image, classifying the region of interest according to a preset pixel threshold value to obtain the region of interest corresponding to the target object, so that the position region of the target object is positioned in the image, and the coverage region of the target object in the image is obtained.
And step 150, obtaining a target outline of the target object in the image based on the coverage area of the target object in the image.
Specifically, the target contour is a contour of the target object in the image, and in this embodiment, after the feature detection of semantic segmentation is performed on the image, the contour of the target object in the image is extracted by using a target detection method and a threshold segmentation method, so as to obtain the target contour of the target object.
And step 160, extracting the target object from the image according to the target outline in the image.
Specifically, after determining the target contour of the target object in the image, extracting the target object in the target contour from the image according to the target contour and then according to a threshold segmentation method, so as to automatically extract the target object in the image, wherein it is understood that the extracted target object is the image of the target object. For example, a target object is extracted, and image information of the target object is obtained. Therefore, the method and the device can automatically extract the images of the refrigerator panel from the images of the refrigerator without manual selection, so that the refrigerator panel is extracted more accurately, more rapidly and conveniently, the time and the labor are effectively saved, and the efficiency is improved.
In the above embodiment, feature detection is performed on the image based on the semantic segmentation technology through a pre-trained training model to obtain feature information of the image, and then a region of interest in the image is determined, so that the position of the target object in the image is obtained by positioning, and the contour of the target object in the image is obtained based on the position in the image, and the target object is extracted from the image according to the contour. The automatic extraction of the target object in the image is realized without manual selection, so that the extraction of the refrigerator panel is more accurate, more rapid and convenient, the time is effectively saved, the labor is reduced, and the efficiency is improved.
In one embodiment, as shown in fig. 1B, step 110 further includes:
Step 101, acquiring a training image.
And 102, performing contour calibration on the target object of the training image to generate a training sample, wherein the training sample is an image file calibrated with the contour of the target object.
And step 103, inputting the training samples into a convolutional neural network for learning to obtain the training model containing the characteristic information of the target object of each training image.
Specifically, the training image is the same type of image as the image in the above embodiment, and the training image includes a target object, for example, the training image is a refrigerator image including images of refrigerators of different types, and the image in the above embodiment is also an image of a refrigerator, where each refrigerator image includes a panel. Before the training image is input into the convolutional neural network for learning, the training image is subjected to standard reaching so as to generate an image file which marks the outline of the target object in the training image, and the image file is a training sample. The training sample is input into a convolutional neural network for learning to obtain a training model, and the training model comprises a set of characteristic information of a target object of a training image, so that the set of characteristic information of different refrigerator panels can be obtained through training of a large number of training images.
In one embodiment, the various types of refrigerator panel images are collected and fine-grained silhouette calibration of the target object in the training image is performed by LabelMe tools, which LabelMe tools may be used to create custom labeling tasks or perform image labeling. After the outline marking of the target object is completed, a JSON (JavaScript Object Notation, JS object numbered musical notation) file is generated. The JSON file is a training sample. Each JSON file was then converted DataSet, dataSet to include img. Png, info. Yaml, label. Png, label_names. Txt, label_viz. Png, generating the mask dataset. The convolutional neural network reads a mask dataset based on a mask-rcnn algorithm to perform model training, and the training is completed to generate a characteristic information set of the refrigerator panel, wherein the characteristic information set can also be called a characteristic dataset and comprises characteristic information of a plurality of refrigerator panels.
In one embodiment, as shown in FIG. 1C, step 130 includes:
Step 131, selecting a preset number of candidate interested areas for each feature information.
And 132, carrying out binary classification of foreground and background on the candidate regions of interest, and screening out the regions of interest of the image from a preset number of candidate regions of interest.
In this embodiment, a preset number of candidate regions of interest are set for each feature information. Specifically, a preset number of candidate regions of interest (ROIs, region Of Interest) that are preset are set separately for each point of the obtained set of the plurality of feature information. And sending the candidate interested areas into a regional network for foreground and background binary classification, filtering out a part of candidate interested areas, and screening out suitable interested areas. It should be appreciated that the area network is a candidate for performing area framing on an input image, and uses information such as edges, textures, colors, color changes, etc. of the image to select areas in the image that may contain objects. In this embodiment, in the process of sending the candidate interested areas into the area network to perform foreground and background binary classification, the candidate interested areas are classified according to the difference of pixel values of the foreground and the background of the image and a preset pixel threshold, so as to screen out the interested areas.
In one embodiment, the step of obtaining the target contour of the target object in the image based on the coverage area of the target object in the image comprises: and extracting a target contour of the target object from the image by adopting a target detection method and a threshold segmentation method based on the coverage area of the target object in the image.
Specifically, in this embodiment, by detecting the position information of the target object through semantic segmentation in the foregoing embodiment, based on the position information, a target detection method and a threshold segmentation method are adopted to extract the contour of the target object in the image in a complex background, thereby obtaining the target contour. It is noted that, the threshold segmentation algorithm may adopt fast rcnn, yolo, SSD, etc. By the target detection method and the threshold segmentation method, the target contour can be extracted efficiently and accurately.
In one embodiment, as shown in FIG. 1D, step 160 includes:
and 161, performing edge processing on the extracted target contour to obtain edge position information of the target contour.
And step 162, extracting the target object from the image based on the edge position information of the target contour.
In this embodiment, an optimal edge of the target contour is found through edge processing, so as to obtain accurate position information of the edge of the target contour, and further, based on the edge position information, the position and the range of the target contour can be accurately positioned, so that the accuracy of the target object extracted from the image is improved.
In one embodiment, the step of performing edge processing on the extracted target contour to obtain edge position information of the target contour includes: obtaining a mask image containing black and white two colors of the image based on the target contour; and carrying out edge processing on the black-and-white mask image by adopting a Canny detection algorithm to obtain the edge position information of the target contour.
In this embodiment, after extracting the target contour in the image, extracting a mask image only containing black and white colors, and performing edge processing on the mask image by adopting an image Canny detection algorithm to obtain an optimal edge, so that the edge of the target contour can be more accurate, and the extraction accuracy of the target object is improved.
In one embodiment, the step of extracting the target object from the image based on edge position information of the target contour includes: acquiring gray values of all pixel points in the image by adopting a threshold segmentation method; comparing the gray value of each pixel point in the image with a preset gray threshold value, and performing binarization processing on the comparison result of the gray value of each pixel point in the image and the preset gray threshold value to obtain a binarized image; and extracting the target object from the binarized image based on the edge position information of the target contour.
Specifically, a threshold segmentation method is adopted to determine a gray value of each pixel point in the image, the gray value of each pixel in the image is compared with the gray threshold value obtained by the threshold segmentation method in the previous step, a proper threshold value is selected for segmentation, binarization algorithm processing is carried out, and finally the extraction of the target object is realized. The suitable threshold can be set according to the pixel gray levels of the target object and the background, and the preset gray level threshold is obtained by setting the threshold. Through the above process, the influence of illumination due to the environment on the characteristics of the image can be reduced.
It should be understood that the threshold separation method is to select a suitable pixel value as a preset gray threshold, and the preset gray threshold may be set manually according to the pixel value of the analysis image. The image is processed into a high contrast, easily identifiable image by the preset gray threshold as a boundary. For example, the white color image is set to be larger than the preset gray threshold, the black color image is set to be smaller than or equal to the preset gray threshold, the white color image is the target object, and the black color image is the background outside the target object in the image. Thus, the target object can be separated from the image, and the extraction of the target object is realized.
The following is a specific example:
first, the refrigerator panel graphic marking training. Referring to fig. 4, a deep learning method of artificial intelligence is utilized to collect data of refrigerator panel feature images, then marking is carried out on each type of refrigerator panel feature, model training is carried out according to sample information by using a convolutional neural network algorithm, and a panel object of semantic recognition is obtained.
Next, a feature detection search. And performing feature detection on the candidate region through a branch network of an example segmentation deep learning Mask-RCNN framework. Referring to fig. 5, firstly, preprocessing an input picture, and then inputting the preprocessed picture into a pre-trained neural network to obtain corresponding data information; next, a predetermined number of ROIs are set for each point in this image data information, thereby obtaining a plurality of candidate ROIs; sending the candidate ROIs into a regional network for binary classification of foreground and background, and filtering out a part of candidate ROIs; and finally classifying the ROIs, and locating and searching the position area of the refrigerator panel.
Finally, extracting the target object. The method comprises the steps of detecting target features through semantic segmentation, extracting the general outline of a panel in a complex background by utilizing a target detection and threshold segmentation technology, extracting a mask only containing black and white, carrying out edge processing on the mask by using an image canny detection algorithm, finding an optimal edge, identifying the actual edge in an image as far as possible, determining a certain gray value of each pixel point in a refrigerator panel image in a gray range by adopting a threshold segmentation method, comparing the gray value of each pixel in the obtained image with a threshold determined before, carrying out binarization algorithm processing, selecting a proper threshold to reduce the influence of illumination caused by the environment on the image features, and finally extracting the characteristic refrigerator panel image automatically.
In this embodiment, the threshold segmentation algorithm may also be fast rcnn, yolo, SSD, or other algorithms.
According to the application, the area of the refrigerator panel area is detected by utilizing an artificial intelligent deep learning algorithm, a semantic segmentation technology and image edge detection, pixel-level classification is carried out, refrigerator panel information is automatically, rapidly and accurately extracted, and the working efficiency of quality inspection personnel is improved.
The application utilizes an artificial intelligent image segmentation algorithm to automatically extract the characteristics of the refrigerator panel in the manufacturing industry. Compared with the traditional manual or semi-automatic method, the method provided by the application can achieve the objectification, standardization and unification of the automatic extraction of the characteristics by training the refrigerator panel information data based on the deep learning algorithm, and the accuracy is greatly improved compared with the manual selection of the characteristic areas, so that the problem that the manual selection of the characteristic areas is different from person to person and is not objective enough is solved, and the efficiency and the detection accuracy of the detection system are improved.
The method solves the problems of manual calibration or semiautomatic photographing in a specific environment in the traditional sense, achieves automatic and accurate extraction, is quick and convenient, and effectively saves time and labor.
In one embodiment, a product detection method is characterized by comprising the steps of extracting a target object in a product image according to the image target object extraction method in any of the embodiments, obtaining image information of the target object, and judging whether a product meets a preset requirement according to the image information of the target object.
In this embodiment, the product detection method may also be referred to as a defect detection method of a target object of a product image, which is used to detect a defect of a certain portion on a product, for example, a defect of a panel of a refrigerator. The target object of the product is extracted through a target object extraction method, for example, a refrigerator panel is extracted from a refrigerator image at first, so that the refrigerator panel is accurately positioned, whether the product meets the preset requirement or not is judged according to the image information of the target object, and then defect detection is carried out on the refrigerator panel. The product detection method includes steps of defect detection of the target object besides the steps of the target object extraction method including the image, and it is worth mentioning that the step of defect detection of the target object can be implemented by adopting the prior art or adopting the conventional defect detection method in the field, and means of the defect detection can be known by those skilled in the art, and is not repeated in this embodiment.
In one embodiment, as shown in fig. 2, there is provided an image target object extraction apparatus including:
An image acquisition module 210 for acquiring an image;
The feature information obtaining module 220 is configured to input the image into a training model trained in advance to perform feature detection, so as to obtain feature information of the image;
A region of interest obtaining module 230, configured to select a preset number of regions of interest for each of the feature information;
the target position information obtaining module 240 is configured to perform binary classification of a foreground and a background on the image including the region of interest, obtain pixel values of the foreground and the background, perform classification based on a preset pixel threshold, and obtain a coverage area of a target object in the image from the region of interest;
a target contour obtaining module 250, configured to obtain a target contour of the target object in the image based on a coverage area of the target object in the image;
the target object extraction module 260 is configured to extract the target object from the image according to a target contour in the image.
In one embodiment, the image target object extraction apparatus further includes:
The training image acquisition module is used for acquiring training images;
the training image marking module is used for carrying out contour calibration on a target object of the training image to generate a training sample, wherein the training sample is an image file calibrated with the contour of the target object;
And the training model generation module is used for inputting the training samples into a convolutional neural network for learning to obtain the training model containing the characteristic information of the target object of each training image.
In one embodiment, the region of interest obtaining module comprises:
The candidate interested region obtaining unit is used for selecting a preset number of candidate interested regions for each piece of characteristic information;
And the interested region screening unit is used for carrying out binary classification on the foreground and the background of the candidate interested regions and screening out the interested regions of the image from the preset number of candidate interested regions.
In one embodiment, the target contour obtaining module is further configured to extract a target contour of the target object from the image by using a target detection method and a threshold segmentation method based on a coverage area of the target object in the image.
In one embodiment, the target object extraction module includes:
an edge position information obtaining unit, configured to perform edge processing on the extracted target contour to obtain edge position information of the target contour;
And the target object extraction unit is used for extracting the target object from the image based on the edge position information of the target contour.
In one embodiment, the edge position information obtaining unit includes:
A mask image obtaining subunit, configured to obtain a mask image including black and white two colors of the image based on the target contour;
and the edge position information obtaining subunit is used for carrying out edge processing on the black-white mask image by adopting a Canny detection algorithm to obtain the edge position information of the target contour.
In one embodiment, the target object extraction unit includes:
A gray value obtaining subunit, configured to obtain a gray value of each pixel point in the image by using a threshold segmentation method;
The binarization processing subunit is used for comparing the gray value of each pixel point in the image with a preset gray threshold value, and performing binarization processing on the comparison result of the gray value of each pixel point in the image with the preset gray threshold value to obtain a binarized image;
and the target object extraction subunit is used for extracting the target object from the binarized image based on the edge position information of the target contour.
The specific definition of the image target object extraction device may be referred to as the definition of the image target object extraction method hereinabove, and will not be described herein. The respective modules in the above-described image target object extraction apparatus may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided. The internal structure thereof can be shown in fig. 3. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for the communication of an external computer through a network connection. The computer program is executed by a processor to implement an image object extraction method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be keys, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in FIG. 3 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided comprising a memory storing a computer program and a processor that when executing the computer program performs the steps of:
acquiring an image;
Inputting the image into a pre-trained training model for feature detection to obtain feature information of the image;
Selecting a preset number of regions of interest for each piece of characteristic information;
Performing binary classification of foreground and background on the image containing the region of interest to obtain pixel values of the foreground and the background, classifying the image based on a preset pixel threshold value, and obtaining a coverage area of a target object in the image from the region of interest;
acquiring a target contour of the target object in the image based on the coverage area of the target object in the image;
and extracting the target object from the image according to the target outline in the image.
In one embodiment, the processor when executing the computer program further performs the steps of:
Acquiring a training image;
Performing contour calibration on a target object of the training image to generate a training sample, wherein the training sample is an image file calibrated with the contour of the target object;
and inputting the training samples into a convolutional neural network for learning to obtain the training model containing the characteristic information of the target object of each training image.
In one embodiment, the processor when executing the computer program further performs the steps of:
Selecting a preset number of candidate interested areas for each piece of characteristic information;
And carrying out binary classification of foreground and background on the candidate regions of interest, and screening out the regions of interest of the image from a preset number of candidate regions of interest.
In one embodiment, the processor when executing the computer program further performs the steps of:
And extracting a target contour of the target object from the image by adopting a target detection method and a threshold segmentation method based on the coverage area of the target object in the image.
In one embodiment, the processor when executing the computer program further performs the steps of:
performing edge processing on the extracted target contour to obtain edge position information of the target contour;
And extracting the target object from the image based on the edge position information of the target contour.
In one embodiment, the processor when executing the computer program further performs the steps of:
Obtaining a mask image containing black and white two colors of the image based on the target contour;
And carrying out edge processing on the black-and-white mask image by adopting a Canny detection algorithm to obtain the edge position information of the target contour.
In one embodiment, the processor when executing the computer program further performs the steps of:
Acquiring gray values of all pixel points in the image by adopting a threshold segmentation method;
Comparing the gray value of each pixel point in the image with a preset gray threshold value, and performing binarization processing on the comparison result of the gray value of each pixel point in the image and the preset gray threshold value to obtain a binarized image;
and extracting the target object from the binarized image based on the edge position information of the target contour.
In one embodiment, a computer device is provided that includes a memory storing a computer program and a processor that implements a product detection method when the computer program is executed.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring an image;
Inputting the image into a pre-trained training model for feature detection to obtain feature information of the image;
Selecting a preset number of regions of interest for each piece of characteristic information;
Performing binary classification of foreground and background on the image containing the region of interest to obtain pixel values of the foreground and the background, classifying the image based on a preset pixel threshold value, and obtaining a coverage area of a target object in the image from the region of interest;
acquiring a target contour of the target object in the image based on the coverage area of the target object in the image;
and extracting the target object from the image according to the target outline in the image.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Acquiring a training image;
Performing contour calibration on a target object of the training image to generate a training sample, wherein the training sample is an image file calibrated with the contour of the target object;
and inputting the training samples into a convolutional neural network for learning to obtain the training model containing the characteristic information of the target object of each training image.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Selecting a preset number of candidate interested areas for each piece of characteristic information;
And carrying out binary classification of foreground and background on the candidate regions of interest, and screening out the regions of interest of the image from a preset number of candidate regions of interest.
In one embodiment, the computer program when executed by the processor further performs the steps of:
And extracting a target contour of the target object from the image by adopting a target detection method and a threshold segmentation method based on the coverage area of the target object in the image.
In one embodiment, the computer program when executed by the processor further performs the steps of:
performing edge processing on the extracted target contour to obtain edge position information of the target contour;
And extracting the target object from the image based on the edge position information of the target contour.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Obtaining a mask image containing black and white two colors of the image based on the target contour;
And carrying out edge processing on the black-and-white mask image by adopting a Canny detection algorithm to obtain the edge position information of the target contour.
In one embodiment, the computer program when executed by the processor further performs the steps of:
Acquiring gray values of all pixel points in the image by adopting a threshold segmentation method;
Comparing the gray value of each pixel point in the image with a preset gray threshold value, and performing binarization processing on the comparison result of the gray value of each pixel point in the image and the preset gray threshold value to obtain a binarized image;
and extracting the target object from the binarized image based on the edge position information of the target contour.
In one embodiment, a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing a product detection method is provided.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples illustrate only a few embodiments of the application, which are described in detail and are not to be construed as limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.
Claims (11)
1. An image target object extraction method, characterized by comprising the following steps:
acquiring an image;
Inputting the image into a pre-trained training model for feature detection to obtain feature information of the image;
Selecting a preset number of regions of interest for each piece of characteristic information;
Performing binary classification of foreground and background on the image containing the region of interest to obtain pixel values of the foreground and the background, classifying the image based on a preset pixel threshold value, and obtaining a coverage area of a target object in the image from the region of interest;
acquiring a target contour of the target object in the image based on the coverage area of the target object in the image;
extracting the target object from the image according to a target contour in the image;
The step of obtaining a target contour of the target object in the image based on the coverage area of the target object in the image comprises the following steps:
Extracting a target contour of the target object from the image by adopting a target detection method and a threshold segmentation method based on the coverage area of the target object in the image;
the step of extracting the target object from the image according to the target contour in the image comprises the following steps:
performing edge processing on the extracted target contour to obtain edge position information of the target contour;
Extracting the target object from the image based on edge position information of the target contour;
the step of extracting the target object from the image based on the edge position information of the target contour includes:
Acquiring gray values of all pixel points in the image by adopting a threshold segmentation method;
Comparing the gray value of each pixel point in the image with a preset gray threshold value, and performing binarization processing on the comparison result of the gray value of each pixel point in the image and the preset gray threshold value to obtain a binarized image;
and extracting the target object from the binarized image based on the edge position information of the target contour.
2. The method of claim 1, further comprising, prior to the step of acquiring an image:
Acquiring a training image;
Performing contour calibration on a target object of the training image to generate a training sample, wherein the training sample is an image file calibrated with the contour of the target object;
and inputting the training samples into a convolutional neural network for learning to obtain the training model containing the characteristic information of the target object of each training image.
3. The method of claim 1, wherein the step of selecting a predetermined number of regions of interest for each of the characteristic information comprises:
Selecting a preset number of candidate interested areas for each piece of characteristic information;
And carrying out binary classification of foreground and background on the candidate regions of interest, and screening out the regions of interest of the image from a preset number of candidate regions of interest.
4. The method of claim 1, wherein the step of extracting the target contour of the target object from the image using a target detection method and a threshold segmentation method based on the coverage area of the target object in the image comprises:
and detecting the position information of the target object through semantic segmentation, and extracting the target outline of the target object from the image by adopting the target detection method and the threshold segmentation method based on the position information.
5. The method of claim 2, wherein the step of contouring the target object of the training image to generate training samples comprises:
and performing contour calibration on the target object of the training image through LabelMe tools to generate a training sample.
6. The method of claim 5, wherein the step of performing edge processing on the extracted target contour to obtain edge position information of the target contour comprises:
Obtaining a mask image containing black and white two colors of the image based on the target contour;
And carrying out edge processing on the black-and-white mask image by adopting a Canny detection algorithm to obtain the edge position information of the target contour.
7. The method according to claim 2, wherein the step of inputting the training samples into a convolutional neural network for learning to obtain the training model containing characteristic information of the target object of each training image comprises:
generating a mask data set according to the training sample;
And inputting the mask data set into the convolutional neural network for learning to obtain a characteristic information set of a target object containing each training image, wherein the characteristic information set is provided with a plurality of characteristic information.
8. A method of detecting a product, comprising:
The image object extraction method according to any one of claims 1 to 7, wherein the object in the product image is extracted, the image information of the object is obtained, and whether the product meets a preset requirement is determined based on the image information of the object.
9. An image target object extraction apparatus, characterized by comprising:
The image acquisition module is used for acquiring images;
The feature information obtaining module is used for inputting the image into a pre-trained training model to perform feature detection and obtaining feature information of the image;
the interested region obtaining module is used for selecting a preset number of interested regions for each piece of characteristic information;
the target position information obtaining module is used for carrying out binary classification on the foreground and the background of the image containing the region of interest to obtain pixel values of the foreground and the background, classifying the image based on a preset pixel threshold value and obtaining a coverage area of a target object in the image from the region of interest;
A target contour obtaining module, configured to obtain a target contour of the target object in the image based on a coverage area of the target object in the image; extracting a target contour of the target object from the image by adopting a target detection method and a threshold segmentation method based on the coverage area of the target object in the image;
The target object extraction module is used for extracting the target object from the image according to the target outline in the image; performing edge processing on the extracted target contour to obtain edge position information of the target contour; extracting the target object from the image based on edge position information of the target contour; acquiring gray values of all pixel points in the image by adopting a threshold segmentation method; comparing the gray value of each pixel point in the image with a preset gray threshold value, and performing binarization processing on the comparison result of the gray value of each pixel point in the image and the preset gray threshold value to obtain a binarized image; and extracting the target object from the binarized image based on the edge position information of the target contour.
10. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 8 when the computer program is executed.
11. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010542166.1A CN111753692B (en) | 2020-06-15 | 2020-06-15 | Target object extraction method, product detection method, device, computer and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010542166.1A CN111753692B (en) | 2020-06-15 | 2020-06-15 | Target object extraction method, product detection method, device, computer and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111753692A CN111753692A (en) | 2020-10-09 |
| CN111753692B true CN111753692B (en) | 2024-05-28 |
Family
ID=72675169
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010542166.1A Active CN111753692B (en) | 2020-06-15 | 2020-06-15 | Target object extraction method, product detection method, device, computer and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111753692B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112330598B (en) * | 2020-10-14 | 2023-07-25 | 浙江华睿科技股份有限公司 | Method, device and storage medium for detecting stiff yarn defects on chemical fiber surface |
| CN112712077B (en) * | 2020-12-30 | 2023-04-07 | 中冶赛迪信息技术(重庆)有限公司 | Steel flow contour determination method, system, terminal and medium |
| CN113076907B (en) * | 2021-04-16 | 2023-03-21 | 青岛海尔电冰箱有限公司 | Method for identifying information of articles in refrigerator, refrigerator and computer storage medium |
| CN113505781B (en) * | 2021-06-01 | 2024-06-25 | 北京旷视科技有限公司 | Target detection method, target detection device, electronic equipment and readable storage medium |
| CN113223041B (en) * | 2021-06-25 | 2024-01-12 | 上海添音生物科技有限公司 | Method, system and storage medium for automatically extracting target area in image |
| CN113361487A (en) * | 2021-07-09 | 2021-09-07 | 无锡时代天使医疗器械科技有限公司 | Foreign matter detection method, device, equipment and computer readable storage medium |
| CN113902910B (en) * | 2021-12-10 | 2022-04-08 | 中国科学院自动化研究所 | Vision measurement method and system |
| CN114789452B (en) * | 2022-06-21 | 2022-09-16 | 季华实验室 | Robot grabbing method and system based on machine vision |
| CN115170792B (en) * | 2022-09-07 | 2023-01-10 | 烟台艾睿光电科技有限公司 | Infrared image processing method, device and equipment and storage medium |
| CN115511834A (en) * | 2022-09-28 | 2022-12-23 | 广东利元亨智能装备股份有限公司 | Battery cell alignment degree detection method, controller, detection system and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105139415A (en) * | 2015-09-29 | 2015-12-09 | 小米科技有限责任公司 | Foreground and background segmentation method and apparatus of image, and terminal |
| CN109635733A (en) * | 2018-12-12 | 2019-04-16 | 哈尔滨工业大学 | View-based access control model conspicuousness and the modified parking lot of queue and vehicle target detection method |
| CN109741346A (en) * | 2018-12-30 | 2019-05-10 | 上海联影智能医疗科技有限公司 | Area-of-interest exacting method, device, equipment and storage medium |
| CN109993734A (en) * | 2019-03-29 | 2019-07-09 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
| CN110490212A (en) * | 2019-02-26 | 2019-11-22 | 腾讯科技（深圳）有限公司 | Molybdenum target image processing arrangement, method and apparatus |
-
2020
- 2020-06-15 CN CN202010542166.1A patent/CN111753692B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105139415A (en) * | 2015-09-29 | 2015-12-09 | 小米科技有限责任公司 | Foreground and background segmentation method and apparatus of image, and terminal |
| CN109635733A (en) * | 2018-12-12 | 2019-04-16 | 哈尔滨工业大学 | View-based access control model conspicuousness and the modified parking lot of queue and vehicle target detection method |
| CN109741346A (en) * | 2018-12-30 | 2019-05-10 | 上海联影智能医疗科技有限公司 | Area-of-interest exacting method, device, equipment and storage medium |
| CN110490212A (en) * | 2019-02-26 | 2019-11-22 | 腾讯科技（深圳）有限公司 | Molybdenum target image processing arrangement, method and apparatus |
| CN109993734A (en) * | 2019-03-29 | 2019-07-09 | 北京百度网讯科技有限公司 | Method and apparatus for output information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111753692A (en) | 2020-10-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111753692B (en) | Target object extraction method, product detection method, device, computer and medium | |
| CN110110799B (en) | Cell sorting method, cell sorting device, computer equipment and storage medium | |
| CN109492643B (en) | Certificate identification method and device based on OCR, computer equipment and storage medium | |
| CN113160192B (en) | Visual sense-based snow pressing vehicle appearance defect detection method and device under complex background | |
| CN111160301B (en) | Tunnel disease target intelligent identification and extraction method based on machine vision | |
| CN110838126B (en) | Cell image segmentation method, cell image segmentation device, computer equipment and storage medium | |
| CN110766017B (en) | Mobile terminal text recognition method and system based on deep learning | |
| CN111274926B (en) | Image data screening method, device, computer equipment and storage medium | |
| CN112750121B (en) | System and method for detecting digital image quality of pathological slide | |
| CN112036500A (en) | Garbage classification method and device based on depth image and multispectral image fusion | |
| CN109903272B (en) | Target detection method, device, equipment, computer equipment and storage medium | |
| CN108830197A (en) | Image processing method, device, computer equipment and storage medium | |
| CN112365497A (en) | High-speed target detection method and system based on Trident Net and Cascade-RCNN structures | |
| CN111126393A (en) | Vehicle appearance refitting judgment method and device, computer equipment and storage medium | |
| CN108460344A (en) | Dynamic area intelligent identifying system in screen and intelligent identification Method | |
| CN109389115A (en) | Text recognition method, device, storage medium and computer equipment | |
| CN115731220A (en) | Grey cloth defect positioning and classifying method, system, equipment and storage medium | |
| CN114040116B (en) | Plastic mould good product monitoring feedback system | |
| CN113256644A (en) | Bill image segmentation method, device, medium, and apparatus | |
| CN110751013A (en) | Scene recognition method, device and computer-readable storage medium | |
| CN111402185A (en) | Image detection method and device | |
| CN112861861B (en) | Method and device for recognizing nixie tube text and electronic equipment | |
| CN113139946A (en) | Shirt stain positioning device based on vision | |
| CN111027399A (en) | Remote sensing image surface submarine identification method based on deep learning | |
| CN117474916B (en) | Image detection method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |