WO2023098187A1 - 一种处理方法及处理装置、处理系统 - Google Patents

一种处理方法及处理装置、处理系统 Download PDF

Info

Publication number
WO2023098187A1
WO2023098187A1 PCT/CN2022/116130 CN2022116130W WO2023098187A1 WO 2023098187 A1 WO2023098187 A1 WO 2023098187A1 CN 2022116130 W CN2022116130 W CN 2022116130W WO 2023098187 A1 WO2023098187 A1 WO 2023098187A1
Authority
WO
WIPO (PCT)
Prior art keywords
spectral
image
different
detection
different image
Prior art date
Application number
PCT/CN2022/116130
Other languages
English (en)
French (fr)
Inventor
刘永华
靳玉茹
朱麟
Original Assignee
联想(北京)有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 联想(北京)有限公司 filed Critical 联想(北京)有限公司
Publication of WO2023098187A1 publication Critical patent/WO2023098187A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined

Definitions

  • the present disclosure relates to the field of machine vision inspection, and in particular to a processing method, processing device, and processing system.
  • the upper and lower covers of mobile phones and other portable terminals are usually realized by dispensing. After dispensing and before closing, the quality of the dispensing path needs to be inspected.
  • dispensing glue such as broken glue, overflowing glue, foreign matter and residual glue, etc.
  • visual inspection is time-consuming and inefficient, and eyes are prone to fatigue. Therefore, it is very important in this field to provide a solution based on machine vision.
  • a treatment method comprising:
  • the spectral features corresponding to different image regions identify the attribute parameters of the objects to be collected corresponding to the different image regions; the attribute parameters are related to the materials of the objects to be collected; the objects to be collected include detection objects and background objects;
  • Anomaly detection is performed on the detection object based on regional characteristics of different image regions in the second spectral image.
  • the determining spectral features respectively corresponding to different image regions in the first spectral image includes:
  • the spectral information of the sampling point determine the reflectance of the sampling point for light of different wavelengths, and obtain the spectral distribution characteristics of the sampling point;
  • the spectral distribution characteristics of the sampling points respectively corresponding to different image regions in the first spectral image are used as the spectral characteristics respectively corresponding to the different image regions.
  • the identifying attribute parameters of objects to be collected corresponding to different image regions according to spectral features corresponding to different image regions respectively includes:
  • the first identification rules include: the corresponding pairs of different attribute parameters The reflectance interval of each wavelength of light in different wavelengths of light;
  • the obtaining the second spectral image formed by the target image area includes:
  • the abnormal detection of the detection object based on the regional characteristics of different image regions in the second spectral image includes:
  • an abnormal type identification process is performed on the detection target areas corresponding to different image areas in the second spectrum image.
  • performing abnormal type identification processing on detection object areas corresponding to different image areas in the second spectral image includes:
  • the extracted visual features and the second recognition rule identify whether there is an abnormality in the detection target area corresponding to the different image areas in the second spectral image, and the corresponding abnormal type if there is an abnormality; wherein, the second identification rule Including: benchmark visual features corresponding to different abnormal types of detection objects;
  • the abnormal recognition result at least includes: when there is an abnormality in the detection object region corresponding to the different image region in the second spectral image the exception type.
  • the abnormal detection of the detected object based on the regional characteristics of different image regions in the second spectral image includes:
  • abnormal type identification processing is performed on the detection object regions corresponding to different image regions in the second spectral image.
  • performing abnormal type identification processing on detection object regions corresponding to different image regions in the second spectral image includes:
  • the extracted spectral features and the third identification rule identify whether there is abnormality in the detection object area corresponding to the different image areas in the second spectral image, and the corresponding abnormal type if there is an abnormality; wherein, the third identification rule Including: benchmark spectral features corresponding to different anomaly types of detected objects;
  • the abnormal recognition results at least include: when there is an abnormality in the detection object region corresponding to the different image region in the second spectral image the exception type.
  • a processing device comprising:
  • An acquisition module configured to acquire a spectral image of an object to be acquired to obtain a first spectral image
  • a determining module configured to determine spectral features corresponding to different image regions in the first spectral image
  • the recognition module is used to identify the attribute parameters of the objects to be collected corresponding to different image regions according to the spectral characteristics corresponding to the different image regions; the attribute parameters are related to the materials of the objects to be collected; the objects to be collected include detection objects and background object;
  • An acquisition module configured to determine the target area corresponding to the detection object in the first spectral image according to attribute parameters corresponding to different image areas in the first spectral image, and obtain a second target area formed by the target area.
  • a detection module configured to perform anomaly detection on the detection object based on regional characteristics of different image regions in the second spectral image.
  • a processing system comprising:
  • the light source component is used to perform illumination processing on the object to be collected;
  • the object to be collected includes a detection object and a background object;
  • a spectral image acquisition component configured to acquire a first spectral image of the object to be acquired
  • a processing device configured to implement the abnormality detection of the detection object by executing the processing method as described in any one of the above.
  • the processing method, processing device, and processing system perform spectral image collection on the object to be collected composed of the detection object and its background object, and according to the spectra of different image regions in the collected first spectral image Features, identify the attribute parameters of the object to be collected corresponding to different image areas, the identified attribute parameters are related to the material of the object to be collected, and then locate and extract the detection object according to the attribute parameters corresponding to different image areas in the first spectral image In the second spectral image corresponding to the first spectral image, and based on the regional characteristics of different image regions in the second spectral image, abnormality detection is performed on the detection object.
  • FIG. 1 is a schematic flow chart of the processing method provided by the present disclosure
  • Fig. 2 is a schematic diagram of 5-channel colors in a series of channels included in a sampling point provided by the present disclosure
  • 3 is a schematic diagram of comparison of different color channels corresponding to grayscale images, RGB images, multispectral images, and hyperspectral images provided by the present disclosure
  • Fig. 4 is a comparison chart of reflectance curves of different materials for light of various wavelengths in multispectral/hyperspectral provided by the present disclosure
  • Fig. 5 is another schematic flow chart of the processing method provided by the present disclosure.
  • Fig. 6 is a physical diagram of three types of glue path defects of the glue dispensing path provided by the present disclosure: broken glue, overflowing glue, and foreign matter residual glue;
  • Fig. 7 is another schematic flow chart of the processing method provided by the present disclosure.
  • FIG. 8 is a structural diagram of a processing device provided by the present disclosure.
  • FIG. 9 is a structural diagram of the processing system provided by the present disclosure.
  • the present disclosure provides a processing method, a processing device, and a processing system, which are used to provide a machine vision-based abnormality detection solution for detection objects such as glue dispensing roads, and overcome the existence of the above two possible solutions technical flaws.
  • the processing process of the processing method provided by the present disclosure specifically includes:
  • Step 101 Collect a spectral image of an object to be collected to obtain a first spectral image.
  • the objects to be collected include detection objects and background objects.
  • the detection object and its background object can be, but not limited to, the dispensing road on which the upper and lower lids of portable terminals such as mobile phones and tablet computers are based, and the background object of the dispensing road, such as the dispensing road on/under the mobile phone The corresponding base or surrounding other objects in the cover, etc.
  • the applicant's research has found that usually the material of the detection object and its background object is different, or even if the material is the same, there are differences between the detection object and its background object in terms of density or thickness and other attribute characteristics, and the applicant's research has found that spectral features are easier to distinguish different Different density, thickness and other attribute characteristics of the material or the same material, so this disclosure proposes to distinguish the detection object and its background object according to the spectral characteristics, so as to identify and extract the spectral image corresponding to the detection object from the spectral images of the detection object and its background object section, for anomaly detection of detection objects. Compared with the above two possible solutions, distinguishing the detection object and its background object through spectral features can identify and extract the image part corresponding to the detection object more accurately and quickly.
  • the spectral image of the object to be collected is collected to obtain the first spectral image of the object to be collected.
  • a multispectral camera or a hyperspectral camera may be used as a collection device, but not limited to, to collect spectral images of the object to be collected.
  • the preset light source components can be used to assist the acquisition equipment to achieve spectral image acquisition.
  • the object to be collected is irradiated by the light source assembly, and the illumination light of the light source assembly includes light of a wavelength that is sensitive to the object to be collected (such as the glue road and its periphery/base).
  • Spectral image collection is performed on the object to be collected under illumination to obtain a first spectral image.
  • an incandescent lamp capable of emitting full-band light can be preferably used as the light source component.
  • Step 102 Determine spectral features respectively corresponding to different image regions in the first spectral image.
  • Spectral features respectively corresponding to different image regions in the first spectral image include spectral distribution characteristics respectively corresponding to sampling points in different image regions in the first spectral image.
  • the spectrum of each sampling point includes a series of channels, such as more than 100 color channels, see Figure 2, which provides a certain sampling point
  • Figure 2 A schematic diagram of the 5-channel colors (specifically represented by different grayscales) in a series of channels included, reflecting the reflection of the sampling point to the light of the corresponding wavelength.
  • FIG. 3 it provides a schematic diagram of the comparison of different color channels corresponding to grayscale images, RGB (Red-Green-Blue, red-green-blue) images, multispectral images and hyperspectral images (also represented by different grayscales) Different color channels), compared with ordinary grayscale and RGB images, spectral images (multispectral images/hyperspectral images) have richer and more complex color channels, which can more accurately reflect the object’s response to different wavelengths of light reflective situation.
  • spectral images multispectral images/hyperspectral images
  • objects of different materials reflect different wavelengths of light
  • objects with different densities and thicknesses of the same material also reflect different wavelengths of light, which leads to different materials or different densities/thicknesses of the same material.
  • Spectral images of different objects present different spectral distribution characteristics.
  • the embodiment of the present disclosure extracts the spectral information of sampling points in different image regions in the first spectral image, such as the values of each color channel contained in different sampling points, and according to the Spectral information, determine the reflectivity of the sampling point to light of different wavelengths, use the reflectivity of the sampling point to light of different wavelengths to characterize the spectral distribution characteristics of the sampling point, and obtain the spectral characteristics corresponding to different image regions in the first spectral image, so as to use It is used as a basis for identifying the detection object and its background object (such as the glue dispensing road and its base/periphery) in the first spectral image.
  • the detection object and its background object such as the glue dispensing road and its base/periphery
  • Step 103 according to the spectral characteristics corresponding to different image regions, identify the attribute parameters of the objects to be collected corresponding to different image regions.
  • the above attribute parameters are parameters related to the material of the object to be collected, including but not limited to the material, density and/or thickness of the object to be collected.
  • this embodiment After determining the spectral characteristics corresponding to different image regions in the first spectral image, this embodiment continues to identify the material, material, and Attribute parameters such as density and/or thickness.
  • the pre-trained first recognition model is used to process the spectral distribution characteristics of the sampling points corresponding to different image regions to obtain attribute parameters of the objects to be collected corresponding to different image regions.
  • the first recognition model for extracting spectral features of different image regions in the first spectral image and identifying attribute parameters of different regions according to the extracted spectral features is pre-trained.
  • a series of spectral images of the glue road and materials beside the glue road can be collected in advance, and each In the spectral image, attribute information such as material, density, and/or thickness corresponding to the glue road and the material next to the glue road are obtained, and a series of spectral images marked with the attribute information of the glue road and the material next to the glue road are obtained, and used as model training samples.
  • different image areas can be identified according to the spectral distribution characteristics of the sampling points corresponding to different image areas in the first spectral image and the first identification rules.
  • the first identification rule is formulated in advance, and the first identification rule includes: reflectivity intervals for each wavelength of light in different wavelengths of light corresponding to different attribute parameters.
  • different materials correspond to reflectance intervals for each wavelength of light of different wavelengths, and/or different densities and thicknesses of the same material correspond to reflectance intervals for each of different wavelengths of light.
  • the attribute parameters of the object to be collected corresponding to different image areas according to the spectral distribution characteristics of the sampling points corresponding to the different image areas in the first spectral image and the first identification rule, it is possible to specifically identify different images in the first spectral image.
  • the reflectance of the sampling points corresponding to the regions for each wavelength of light in different wavelengths of light, the reflectance interval actually located in the first identification rule, and the reflectance interval actually located in the first identification rule The corresponding attribute parameters, such as material and/or density, thickness and other attribute parameters, are used as the attribute parameters of the objects to be collected corresponding to different image regions.
  • Step 104 according to attribute parameters corresponding to different image regions in the first spectral image, determine the target area corresponding to the detection object in the first spectral image, and obtain a second spectral image formed by the target area.
  • the detection object in the first spectral image corresponding to the target area For example, according to the material of the object to be collected corresponding to different image regions in the first spectral image and the actual material of the glue dispensing road, the target area corresponding to the glue dispensing road in the first spectral image is determined.
  • the second spectral image formed by extracting the target area from the first spectral image is extracted from the first spectral image by means of image segmentation, which is the spectral image of the detection object such as the glue dispensing road.
  • Step 105 based on the regional characteristics of different image regions in the second spectrum image, perform anomaly detection on the detection object.
  • anomaly detection is performed on the detection object, such as detecting whether there is a dispensing defect on the dispensing glue road, and the specific defect type if there is a dispensing defect.
  • the types of defects on the glue dispensing road include but are not limited to broken glue, overflow glue, foreign matter residue glue, etc.
  • the processing method provided by the present disclosure collects the spectral image of the object to be collected composed of the detection object and its background object, and identifies different image regions according to the spectral characteristics of different image regions in the collected first spectral image Corresponding to the attribute parameters of the object to be collected, the identified attribute parameters are related to the material of the object to be collected, and then, according to the attribute parameters corresponding to different image regions in the first spectral image, locate and extract the detection object in the first spectral image Corresponding to the second spectral image, and based on the regional characteristics of different image regions in the second spectral image, abnormality detection is performed on the detection object.
  • the present disclosure realizes a machine vision-based abnormality detection solution for detection objects such as glue dispensing roads.
  • the present disclosure distinguishes the detection object and its background object through spectral features, compared with existing solutions based on the use of 2D area array cameras or laser profilers to collect detection object data such as rubber roads to achieve abnormal detection, it can be more accurate and fast Recognition and extraction of the image part corresponding to the detection object is not affected by the change of the detection object. It has very strong adaptability to the different forms of the same type of detection object (such as rubber road adjustment) or the situation of changing the model and changing the production line. At the same time, it overcomes the The problem of small samples shortens the preparation time for changing models and lines, and improves the accuracy and efficiency of abnormal detection of detection objects.
  • an optional implementation manner provided by the present disclosure is to perform abnormal detection on the detection object according to the visual characteristics of different image regions in the second spectral image abnormal detection.
  • step 105 in the processing method shown in Figure 1 can be further implemented as:
  • Step 1051 extracting visual features of different image regions in the second spectrum image.
  • edge detection and/or image local feature point extraction based on technologies such as image edge detection and/or image local feature point extraction, but not limited to, by performing edge detection and/or feature point extraction on different image regions of the second spectral image, it can be identified that the second spectral image is different.
  • Various edge features and/or local features of image regions are used as visual features of different image regions.
  • the extracted visual features include but are not limited to the shape and direction of the edge lines of the dispensing road (such as regular straight/broken lines, fixed direction, etc.), corresponding types of points
  • the pattern and shape such as various irregular shapes) of glue defects, and/or depressions (such as broken glue) and protrusions (such as foreign matter residues, overflow glue), overflow (such as overflow glue) and other characteristics.
  • Step 1052 according to the extracted visual features, perform abnormal type identification processing on the detection target areas corresponding to different image areas in the second spectral image.
  • the extracted visual features are used as the basis for identification, and abnormal type identification processing is performed on the detection object areas corresponding to different image areas in the second spectral image.
  • the extracted visual features of the second spectral image and the second recognition rule identify whether there is anomaly in the detection target area corresponding to different image areas in the second spectral image, and if there is an abnormality, The specific corresponding exception type.
  • the second recognition rule for identifying the abnormal type of the detection object is formulated in advance, and the second recognition rule includes: reference visual features respectively corresponding to different abnormal types of the detection object.
  • the reference visual features corresponding to different types of abnormalities such as glue breakage, glue overflow, or foreign matter residue in the dispensing path.
  • the physical map of the type of rubber road defect in practical applications, can be combined with the actual visual features presented by different types of abnormalities of the detected object (preferably representative general features presented by defect samples with sufficient data volume) for the second identification
  • the setting of the benchmark visual features corresponding to different abnormal types in the rules can be combined with the actual visual features presented by different types of abnormalities of the detected object (preferably representative general features presented by defect samples with sufficient data volume) for the second identification.
  • a pre-trained second recognition model is used to process the extracted visual features of the second spectral image to obtain an abnormality recognition result.
  • the abnormality recognition result at least includes: the abnormality type when there is an abnormality in the detection object area corresponding to the different image regions in the second spectral image.
  • An annotated spectral image that has its abnormal region and is associated with a specific abnormal type (eg, broken glue, overflowed glue, or foreign substance residue) for the abnormal region.
  • the second recognition model for performing visual feature extraction on the second spectral image and realizing abnormal detection of the detection object based on the extracted visual features is pre-trained.
  • the detection object as the dispensing road as an example
  • a series of spectral images of abnormal and non-abnormal dispensing roads can be obtained in advance, and corresponding labels can be marked for each spectral image, for example, it is the spectral image of normal dispensing roads Mark the label representing "normal/non-abnormal glue road", outline the abnormal area for the spectral image of the abnormal glue dispensing road, and mark the specific abnormal type (such as broken glue, overflow glue or foreign body residual glue) for the abnormal area association ), etc., to obtain a series of rubber road spectral images marked with label information, and use them as model training samples.
  • the process continuously enables the model to learn the potential correlation between the visual features of different regions of the glue road in the sample image and whether the corresponding region is abnormal and the specific abnormal type in the abnormal situation, so that the trained model can extract different regions in the second spectral image According to the extracted visual features, identify whether the corresponding area is abnormal and which type of abnormality corresponds to the abnormal situation.
  • an abnormality detection result output by the second recognition model based on the visual features of different regions in the second spectral image for abnormal detection of the detection object can be obtained.
  • another optional implementation mode provided by the present disclosure is to detect the detection object according to the spectral characteristics of different image regions in the second spectral image Perform anomaly detection.
  • step 105 in the processing method shown in Figure 1 can be further implemented as:
  • Step 1053 extract spectral features of different image regions in the second spectral image.
  • the spectral characteristics of different image regions in the second spectral image may include spectral distribution characteristics respectively corresponding to sampling points in different image regions in the second spectral image.
  • the spectral information of sampling points in different image regions in the second spectral image can be extracted, such as the values of each color channel contained in different sampling points, and the reflectance of the sampling points to light of different wavelengths can be determined according to the spectral information of the sampling points , using the reflectance of the sampling point to light of different wavelengths to characterize the spectral distribution characteristics of the sampling point, and correspondingly obtain the spectral characteristics of different image regions in the second spectral image.
  • Step 1054 according to the extracted spectral features, perform abnormal type identification processing on the detection object regions corresponding to different image regions in the second spectral image.
  • the extracted spectral features are used as the basis for identification, and the detection object regions corresponding to different image regions in the second spectral image are subjected to abnormal type recognition processing, so as to identify whether there are abnormalities in different regions of the detection object, and the abnormal type in case of abnormality, Such as broken glue, overflow glue or foreign matter residual glue, etc.
  • an optional implementation is to identify different images in the second spectral image according to the extracted spectral features and the third recognition rule Whether there is an abnormality in the detection object area corresponding to the area, and the specific corresponding abnormal type if there is an abnormality.
  • the third identification rule for identifying the abnormal type of the detection object is formulated in advance, and the third identification rule includes: reference spectral features respectively corresponding to different abnormal types of the detection object.
  • the third identification rule includes: reference spectral features respectively corresponding to different abnormal types of the detection object.
  • the material of the broken glue area is the material of the glue road base, which is different from the material of the normal glue road area, and accordingly the broken glue area is different from the normal glue road area.
  • Different spectral distribution characteristics are shown in the second spectral image respectively; when the glue path defect is glue overflow, refer to Figure 6, the thickness of the glue material in the glue overflow area is generally different from that in the normal glue path area, which leads to overflow The spectral characteristics of the glue area and the normal glue path area are different; when the glue path defect is foreign matter residue, the material of the defect area is a mixed material of glue material and foreign matter. You can refer to the example in Figure 6.
  • the spectral features of the defect area are different from those of the normal glue path area, and different defect types usually correspond to different spectral features. Therefore, the reference spectral features corresponding to different glue path defects in the third identification rule can be set in combination with the actual spectral characteristics presented by different types of defects in the glue path.
  • the extracted spectral features of the second spectral image are processed by using a pre-trained third recognition model to obtain an abnormality recognition result.
  • the abnormality recognition result at least includes: the abnormality type when there is an abnormality in the detection object area corresponding to the different image regions in the second spectral image.
  • the third recognition model for performing spectral feature extraction on the second spectral image and realizing abnormal detection of the detection object based on the extracted spectral features is pre-trained.
  • the detection object as the dispensing road as an example
  • a series of spectral images of abnormal and non-abnormal dispensing roads can be obtained in advance, and corresponding labels can be marked for each spectral image, for example, it is the spectral image of normal dispensing roads Mark the label representing "normal/non-abnormal glue path", outline the abnormal area for the spectral image of the abnormal glue dispensing path, and associate specific abnormal types (such as broken glue, overflow glue or foreign body residual glue), etc., Obtain a series of rubber road spectral images marked with label information and use them as model training samples.
  • CNN Convolutional Neural Networks, Convolutional Neural Networks
  • an abnormality detection result output by the third recognition model based on the spectral features of different regions in the second spectral image for abnormal detection of the detection object can be obtained.
  • the disclosure distinguishes the detection object and its background object according to the spectral characteristics.
  • it can more accurately and quickly identify and extract the image part corresponding to the detection object, and is not affected by the change of the detection object. It has very strong adaptability to any situation, and at the same time overcomes the problem of small samples, shortens the preparation time for changing models and lines, and improves the accuracy and efficiency of abnormal detection of detection objects.
  • the present disclosure also provides a processing device, the composition and structure of which is shown in Figure 8, specifically including:
  • An acquisition module 801 configured to acquire a spectral image of an object to be acquired to obtain a first spectral image
  • a determining module 802 configured to determine spectral features corresponding to different image regions in the first spectral image
  • the identification module 803 is configured to identify the attribute parameters of the objects to be collected corresponding to different image regions according to the spectral characteristics corresponding to the different image regions; the attribute parameters are related to the materials of the objects to be collected; the objects to be collected include detection objects and the background object;
  • the acquisition module 804 is configured to determine the corresponding target area of the detection object in the first spectral image according to the attribute parameters corresponding to different image areas in the first spectral image, and obtain the second target area formed by the target area.
  • the detection module 805 is configured to perform anomaly detection on the detection object based on regional characteristics of different image regions in the second spectral image.
  • the determination module 802 is specifically used for:
  • the spectral information of the sampling point determine the reflectance of the sampling point for light of different wavelengths, and obtain the spectral distribution characteristics of the sampling point;
  • the spectral distribution characteristics of sampling points respectively corresponding to different image regions in the first spectral image are used as spectral characteristics respectively corresponding to the different image regions.
  • the identification module 803 is specifically used to:
  • the first identification rules include: the corresponding pairs of different attribute parameters The reflectance interval of each wavelength of light in different wavelengths of light;
  • the acquiring module 804 when acquiring the second spectral image formed by the target area, is specifically configured to:
  • the detection module 805 is specifically used for:
  • an abnormal type identification process is performed on the detection target areas corresponding to different image areas in the second spectrum image.
  • the detection module 805 is specifically used to:
  • the extracted visual features and the second recognition rule identify whether there is an abnormality in the detection target area corresponding to the different image areas in the second spectral image, and the corresponding abnormal type if there is an abnormality; wherein, the second identification rule Including: benchmark visual features corresponding to different abnormal types of detection objects;
  • the abnormal recognition result at least includes: when there is an abnormality in the detection object region corresponding to the different image region in the second spectral image the exception type.
  • the detection module 805 is specifically used for:
  • abnormal type identification processing is performed on the detection object regions corresponding to different image regions in the second spectral image.
  • the detection module 805 is specifically used to:
  • the extracted spectral features and the third identification rule identify whether there is abnormality in the detection object area corresponding to the different image areas in the second spectral image, and the corresponding abnormal type if there is an abnormality; wherein, the third identification rule Including: benchmark spectral features corresponding to different anomaly types of detected objects;
  • the abnormal recognition results at least include: when there is an abnormality in the detection object region corresponding to the different image region in the second spectral image the exception type.
  • an embodiment of the present disclosure also provides a processing system, referring to FIG. 9 , the system includes a light source component 901, a spectral image acquisition component 902, and a processing device 903 as provided in the previous embodiment.
  • the light source component 901 is configured to perform illumination processing on the object to be collected.
  • the irradiated light of the light source assembly 901 includes light of a wavelength sensitive to objects to be collected (eg, glue roads and their surroundings/substrates), and the objects to be collected include detection objects and background objects.
  • an incandescent lamp capable of emitting full-band light can be preferably used as the light source component.
  • Spectral image collection component 902 configured to collect a first spectral image of the object to be collected.
  • Spectral image acquisition component 902 may be, but not limited to, a multispectral camera or a hyperspectral camera.
  • the processing means 903 is configured to implement the abnormality detection of the detection object by executing the processing method provided in any one of the above method embodiments.
  • the present disclosure can be implemented by means of software plus a necessary general hardware platform.
  • the technical solution of the present disclosure can be embodied in the form of a software product in essence or the part that contributes to the prior art, and the computer software product can be stored in a storage medium, such as ROM/RAM, disk , CD, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present disclosure.

Abstract

一种处理方法及处理装置、处理系统，包括对检测对象及其背景对象构成的待采集对象进行光谱图像采集（101）；确定第一光谱图像中不同图像区域分别对应的光谱特征（102）；根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数，识别的属性参数与待采集对象的材料相关（103）；根据第一光谱图像中不同图像区域分别对应的属性参数，定位、提取检测对象在第一光谱图像中对应的第二光谱图像（104）；基于第二光谱图像中不同图像区域的区域特征，对检测对象进行异常检测（105）。

Description

一种处理方法及处理装置、处理系统
本公开要求于2021年11月30日提交中国专利局、申请号为202111452840.8、发明名称为“一种处理方法及处理装置、处理系统”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及机器视觉检测领域，尤其涉及一种处理方法及处理装置、处理系统。
背景技术
手机等便携终端的上下盖合盖，通常采用点胶工艺实现，在点胶后、合盖前，需对点胶胶路的质量问题进行检测。点胶胶路质量问题如断胶、溢胶、异物残胶等，问题较多，而肉眼检查较为费时、效率低，同时眼睛很容易疲劳。因此，提供一种基于机器视觉的解决方案，于本领域来说非常重要。
发明内容
为此，本公开提供如下技术方案：
一种处理方法，所述方法包括：
对待采集对象进行光谱图像采集得到第一光谱图像；
确定所述第一光谱图像中不同图像区域分别对应的光谱特征；
根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数；所述属性参数与待采集对象的材料相关；所述待采集对象包括检测对象和背景对象；
根据所述第一光谱图像中不同图像区域分别对应的属性参数，确定所述检测对象在所述第一光谱图像中对应的目标区域，并获得所述目标区域形成的第二光谱图像；
基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测。
可选的，所述确定所述第一光谱图像中不同图像区域分别对应的光谱特征，包括：
提取所述第一光谱图像中不同图像区域的采样点的光谱信息；
根据采样点的光谱信息，确定采样点对不同波长光的反射率，得到采样点的光谱分布特征；
其中，所述第一光谱图像中不同图像区域分别对应的采样点的光谱分布特征，作为所述不同图像区域分别对应的光谱特征。
可选的，所述根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数，包括：
根据不同图像区域分别对应的采样点的光谱分布特征及第一识别规则，识别不同图像区域分别对应的待采集对象的属性参数；其中，所述第一识别规则包括：不同属性参数分别对应的对不同波长光中每种波长光的反射率区间；
或，利用预先训练的第一识别模型对不同图像区域分别对应的采样点的光谱分布特征进行处理，得到不同图像区域分别对应的待采集对象的属性参数。
可选的，所述获得所述目标图像区域形成的第二光谱图像，包括：
从所述第一光谱图像中分割出所述目标区域的图像，得到所述第二光谱图像。
可选的，所述基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测，包括：
提取所述第二光谱图像中不同图像区域的视觉特征；
根据提取的视觉特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
可选的，所述根据提取的视觉特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理，包括：
根据提取的视觉特征及第二识别规则，识别所述第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，及存在异常情况下所对应的异常类型；其中，所述第二识别规则包括：检测对象的不同异常类型分别对应的基准视觉特征；
或，利用预先训练的第二识别模型对提取的视觉特征进行处理，得到异常识别结果；所述异常识别结果至少包括：所述第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型。
可选的，所述基于所述第二光谱图像中不同图像区域的区域特征，对所述 检测对象进行异常检测，包括：
提取所述第二光谱图像中不同图像区域的光谱特征；
根据提取的光谱特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
可选的，所述根据提取的光谱特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理，包括：
根据提取的光谱特征及第三识别规则，识别所述第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，及存在异常情况下所对应的异常类型；其中，所述第三识别规则包括：检测对象的不同异常类型分别对应的基准光谱特征；
或，利用预先训练的第三识别模型对提取的光谱特征进行处理，得到异常识别结果；所述异常识别结果至少包括：所述第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型。
一种处理装置，所述装置包括：
采集模块，用于对待采集对象进行光谱图像采集得到第一光谱图像；
确定模块，用于确定所述第一光谱图像中不同图像区域分别对应的光谱特征；
识别模块，用于根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数；所述属性参数与待采集对象的材料相关；所述待采集对象包括检测对象和背景对象；
获取模块，用于根据所述第一光谱图像中不同图像区域分别对应的属性参数，确定所述检测对象在所述第一光谱图像中对应的目标区域，并获得所述目标区域形成的第二光谱图像；
检测模块，用于基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测。
一种处理系统，所述系统包括：
光源组件，用于对待采集对象进行光照处理；所述待采集对象包括检测对象和背景对象；
光谱图像采集组件，用于采集所述待采集对象的第一光谱图像；
处理装置，用于通过执行如上文任一项所述的处理方法，实现对所述检测 对象的异常检测。
由以上方案可知，本公开提供的处理方法及处理装置、处理系统，对检测对象及其背景对象构成的待采集对象进行光谱图像采集，并根据采集得到的第一光谱图像中不同图像区域的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数，识别的属性参数与待采集对象的材料相关，之后，根据第一光谱图像中不同图像区域分别对应的属性参数，定位、提取检测对象在第一光谱图像中对应的第二光谱图像，并基于第二光谱图像中不同图像区域的区域特征，对检测对象进行异常检测。
附图说明
为了更清楚地说明本公开或相关技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1是本公开提供的处理方法的一种流程示意图；
图2是本公开提供的某采样点包含的一系列通道中的5通道色彩示意图；
图3是本公开提供的灰度图像、RGB图像、多光谱图像及高光谱图像分别对应的不同色彩通道的对比示意图；
图4是本公开提供的不同材质对多光谱/高光谱中各种波长光的反射率曲线对比图；
图5是本公开提供的处理方法的另一种流程示意图；
图6是本公开提供的点胶胶路的断胶、溢胶及异物残胶三种类型胶路缺陷的实物图；
图7是本公开提供的处理方法的又一种流程示意图；
图8是本公开提供的处理装置的组成结构图；
图9是本公开提供的处理系统的组成结构图。
具体实施方式
下面将结合本公开的实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分实施例，而不 是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
点胶胶路质量问题如断胶、溢胶、异物残胶等，问题较多，而肉眼检查费时费力、同时效率较低，针对该问题，申请人发现存在两种可能的解决方式，一种是，采用激光轮廓仪跟随点胶机路径扫描一圈的3D(3-Dimensions，三维)方案来采集胶路数据，进行胶路异常检测；另一种是，采用2D(2-Dimensions，二维)面阵相机拍摄胶路图像，之后用算法分割出胶路部分，进行胶路缺陷检测。
然而，上述两种解决方式存在以下缺陷：
1)胶路缺陷样本太少，上述两种解决方式必须积累足够多的样本才能训练得到所需的模型并随后在产线使用，需要很久的研发时间；
2)同一机型的胶路稍微改变(常有)，就会导致胶路异常、检测精度下降，从而需要重新收集样本、训练模型；
3)不同机型通常胶路都是不同的，导致胶路异常检测的精度下降较大，相应需要重新收集样本、训练模型。
鉴于此，本公开提供一种处理方法及处理装置、处理系统，用于实现为点胶胶路等检测对象提供一种基于机器视觉的异常检测解决方案，并克服上述两种可能的解决方式存在的技术缺陷。
参见图1，本公开提供的处理方法的处理过程，具体包括：
步骤101、对待采集对象进行光谱图像采集得到第一光谱图像。
其中，待采集对象包括检测对象及其背景对象。检测对象及其背景对象可以是但不限于手机、平板电脑等便携终端的上下盖合盖所基于的点胶胶路，及点胶胶路的背景对象，如点胶胶路在手机上/下盖中对应的基底或周边其他对象等。
申请人研究发现，通常检测对象与其背景对象的材质不同，或者即使材质相同，检测对象与其背景对象在密度或厚度等属性特征方面也存在区别，且，申请人研究发现，光谱特征更容易区分不同材质或同种材质的不同密度、厚度等属性特征，从而本公开提出根据光谱特征来区分检测对象及其背景对象，以从检测对象与其背景对象的光谱图像中识别、提取检测对象对应的光谱图像部 分，用于检测对象的异常检测。相比于上述两种可能的解决方式，通过光谱特征区分检测对象及其背景对象，能够更准确、快速的识别、提取检测对象对应的图像部分。
基于上述技术思路，对于检测对象及其背景对象构成的待采集对象，首先在本步骤101，对待采集对象进行光谱图像采集，得到待采集对象的第一光谱图像。
可以但不限于采用多光谱相机或高光谱相机作为采集设备，对待采集对象进行光谱图像采集。实际应用中，可利用预设光源组件来辅助采集设备实现光谱图像采集。
具体的，利用光源组件对待采集对象进行照射，光源组件的照射光中包含待采集对象(如，胶路和其周边/基底)敏感的波长的光，多光谱相机/高光谱相机对处于光源组件光照下的待采集对象进行光谱图像采集，得到第一光谱图像。
其中，可优选采用能用于发射全波段光的白炽灯作为光源组件。
步骤102、确定第一光谱图像中不同图像区域分别对应的光谱特征。
第一光谱图像中不同图像区域分别对应的光谱特征，包括第一光谱图像中不同图像区域的采样点分别对应的光谱分布特征。
多光谱相机/高光谱相机采集得到的待采集对象的第一光谱图像中，每个采样点的光谱包括一系列通道，如可以有100以上的色彩通道，参见图2，提供了某个采样点包含的一系列通道中的5通道色彩(具体以不同灰度表征)示意图，反映了该采样点对相应波长光的反射情况。
同时，结合参见图3，提供了灰度图像、RGB(Red-Green-Blue，红绿蓝)图像、多光谱图像及高光谱图像分别对应的不同色彩通道的对比示意图(同样以不同灰度表征不同的色彩通道)，与普通的灰度、RGB图像相比，光谱图像(多光谱图像/高光谱图像)拥有更丰富、复杂的色彩通道，能更精准地体现被拍摄物体对不同波长光的反射情况。申请人研究发现，不同材质的物体对各种波长光的反射情况不同，以及同一材质不同密度、厚度的物体对各波长光的反射情况也存在区别，相应导致不同材质或相同材质不同密度/厚度的物体的光谱图像分别呈现不同的光谱分布特征。参见图4中不同材质对多光谱/高光谱中各种波长光的反射率曲线，PVC、Acrylic、PET、PS这些不同材质，对各种 波长光分别有不同的反射率，相应在拍摄得到的光谱图像中分别呈现不同的光谱分布特征。
从而，针对待采集对象的第一光谱图像，本公开的实施例提取第一光谱图像中不同图像区域的采样点的光谱信息，如不同采样点包含的各色彩通道取值，并根据采样点的光谱信息，确定采样点对不同波长光的反射率，利用采样点对不同波长光的反射率表征采样点的光谱分布特征，相应得到第一光谱图像中不同图像区域分别对应的光谱特征，以用于作为识别第一光谱图像中检测对象及其背景对象(如点胶胶路及其基底/周边)的依据。
步骤103、根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数。
上述属性参数为与待采集对象的材料相关的参数，包括但不限于待采集对象的材质、密度和/或厚度。
在确定出第一光谱图像中不同图像区域分别对应的光谱特征后，本实施例继续根据第一光谱图像中不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的材质、密度和/或厚度等属性参数。
可选的，在一实施方式中，利用预先训练的第一识别模型对不同图像区域分别对应的采样点的光谱分布特征进行处理，得到不同图像区域分别对应的待采集对象的属性参数。
该实施方式中，预先训练用于对第一光谱图像中不同图像区域进行光谱特征提取并根据提取的光谱特征识别不同区域的属性参数的第一识别模型。以检测对象及其背景对象分别为点胶胶路及其基底/周边为例，可预先采集一系列胶路和胶路旁边材料(胶路的基底/周边)的光谱图像，并分别标注每一光谱图像中胶路和胶路旁边材料所对应的材质、密度和/或厚度等属性信息，得到标注有胶路和胶路旁边材料的属性信息的一系列光谱图像，并作为模型训练样本，在此基础上，利用训练样本对CNN(Convolutional Neural Networks，卷积神经网络)等网络模型进行训练，通过训练过程不断使模型学习样本图像中不同图像区域的光谱分布特征及其对应的属性信息(材质、密度和/或厚度)间的潜在关联，进而使训练所得的模型能够提取光谱图像中不同区域的光谱分布特征，并根据光谱图像中不同区域的光谱分布特征识别不同区域分别对应的材质、密度和/或厚度等属性信息。
但不限于上述的基于第一识别模型的实施方式，在其他实施方式中，可以根据第一光谱图像中不同图像区域分别对应的采样点的光谱分布特征及第一识别规则，识别不同图像区域分别对应的待采集对象的属性参数。
该实施方式中，预先制定第一识别规则，第一识别规则包括：不同属性参数分别对应的对不同波长光中每种波长光的反射率区间。如，不同材质分别对应的对不同波长光中每种波长光的反射率区间，和/或相同材质的不同密度、厚度分别对应的对不同波长光中每种波长光的反射率区间。
在根据第一光谱图像中不同图像区域分别对应的采样点的光谱分布特征及第一识别规则，识别不同图像区域分别对应的待采集对象的属性参数时，具体可识别第一光谱图像中不同图像区域分别对应的采样点对不同波长光中每种波长光的反射率，在所述第一识别规则中实际所处的反射率区间，并将实际所处的反射率区间在该第一识别规则中对应的属性参数，如材质和/或密度、厚度等属性参数，作为不同图像区域分别对应的待采集对象的属性参数。
步骤104、根据第一光谱图像中不同图像区域分别对应的属性参数，确定检测对象在第一光谱图像中对应的目标区域，并获得目标区域形成的第二光谱图像。
在识别出第一光谱图像中不同图像区域分别对应的待采集对象的属性参数后，进一步根据不同图像区域分别对应的属性参数，并结合检测对象的实际属性参数，确定检测对象在第一光谱图像中对应的目标区域。例如，根据第一光谱图像中不同图像区域分别对应的待采集对象的材质，以及点胶胶路的实际材质，确定点胶胶路在第一光谱图像中对应的目标区域。
之后，通过图像分割等手段，从第一光谱图像中提取目标区域形成的第二光谱图像，即为检测对象如点胶胶路的光谱图像。
步骤105、基于第二光谱图像中不同图像区域的区域特征，对检测对象进行异常检测。
最终，基于第二光谱图像中不同图像区域的区域特征，对检测对象进行异常检测，如检测点胶胶路上是否存在点胶缺陷，以及存在点胶缺陷情况下的具体缺陷类型等。
点胶胶路上的缺陷类型包括但不限于断胶、溢胶、异物残胶等多种类型。
由以上方案可知，本公开提供的处理方法，对检测对象及其背景对象构成 的待采集对象进行光谱图像采集，并根据采集得到的第一光谱图像中不同图像区域的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数，识别的属性参数与待采集对象的材料相关，之后，根据第一光谱图像中不同图像区域分别对应的属性参数，定位、提取检测对象在第一光谱图像中对应的第二光谱图像，并基于第二光谱图像中不同图像区域的区域特征，对检测对象进行异常检测。从而，本公开实现了一种对点胶胶路等检测对象的基于机器视觉的异常检测解决方案。
并且，本公开通过光谱特征区分检测对象及其背景对象，相比于已有基于采用2D面阵相机或激光轮廓仪采集胶路等检测对象数据以实现异常检测的方案，能够更准确、快速的识别提取检测对象对应的图像部分，不受检测对象的变化影响，对于同型号检测对象的不同形态(如胶路调整)或换型号换产线的情况均具有非常强的适应性，同时克服了小样本问题，缩短了换型换线的准备时间，提升了对检测对象异常检测的准确率及效率。
在基于第二光谱图像中不同图像区域的区域特征，对检测对象进行异常检测时，本公开提供的一种可选实施方式是，根据第二光谱图像中不同图像区域的视觉特征对检测对象进行异常检测。
参见图5提供的处理方法的流程示意图，该实施方式中，图1所示处理方法中的步骤105可进一步实现为：
步骤1051、提取第二光谱图像中不同图像区域的视觉特征。
具体的，可以但不限于基于图像边缘检测和/或图像局部特征点提取等技术，通过对第二光谱图像的不同图像区域进行边缘检测和/或特征点提取等处理，识别第二光谱图像不同图像区域的各种边缘特征和/或局部特征，作为不同图像区域的视觉特征。
以点胶胶路的第二光谱图像为例，提取的视觉特征包括但不限于点胶胶路的边缘线条形状、走向(如，规则的直线/折线线条、固定的走向等)，相应类型点胶缺陷呈现的式样、形状(如，各种不规则的异形形状)特征、和/或相比于周边较为规则的胶路所存在的凹陷(如断胶)、凸起(如异物残胶、溢胶)、外溢(如溢胶)等特征。
步骤1052、根据提取的视觉特征，对第二光谱图像中不同图像区域对应的 检测对象区域进行异常类型的识别处理。
之后以提取的视觉特征作为识别依据，对第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
可选的，在一种实施方式中，根据提取的第二光谱图像的视觉特征及第二识别规则，识别第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，以及存在异常情况下具体对应的异常类型。
该实施方式中，预先制定用于对检测对象进行异常类型识别的第二识别规则，第二识别规则包括：检测对象的不同异常类型分别对应的基准视觉特征。如，点胶胶路的断胶、溢胶或异物残胶等不同异常类型分别对应的基准视觉特征等，参考图6提供的点胶胶路的断胶、溢胶及异物残胶三种不同类型胶路缺陷的实物图，实际应用中，可结合检测对象不同异常类型分别呈现的实际视觉特征(优选的为足够数据量的缺陷样本呈现的具备代表性的通用特征)，来进行第二识别规则中不同异常类型分别对应的基准视觉特征的设定。
在另一种实施方式中，利用预先训练的第二识别模型对提取的第二光谱图像的视觉特征进行处理，得到异常识别结果。异常识别结果至少包括：第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型，可选的，异常识别结果具体可以为：第二识别模型输出的为第二光谱图像标注有其异常区域并为异常区域关联标注有具体异常类型(如，断胶、溢胶或异物残胶)的已标注光谱图像。
该实施方式中，预先训练用于对第二光谱图像进行视觉特征提取及基于提取的视觉特征实现对检测对象异常检测的第二识别模型。以检测对象为点胶胶路为例，可预先获得一系列异常及非异常点胶胶路的光谱图像，并为每一光谱图像标注对应的标签，如，为正常点胶胶路的光谱图像标注表征“胶路正常/非异常”的标签，为异常点胶胶路的光谱图像勾勒出其异常区域，并为异常区域关联标注具体的异常类型(如，断胶、溢胶或异物残胶)等，得到标注有标签信息的一系列胶路光谱图像，并作为模型训练样本，在此基础上，利用训练样本对CNN(Convolutional Neural Networks，卷积神经网络)等网络模型进行训练，通过训练过程，不断使模型学习样本图像中胶路不同区域的视觉特征与其对应的区域是否异常及异常情况下的具体异常类型间的潜在关联，进而使训练所得的模型能够提取第二光谱图像中不同区域的视觉特征，并根据提取的视 觉特征识别对应区域是否异常以及异常情况下具体对应哪种异常类型。
后续，通过将第二光谱图像输入第二识别模型，即可得到第二识别模型基于第二光谱图像中不同区域的视觉特征对检测对象进行异常检测所输出的异常检测结果。
在基于第二光谱图像中不同图像区域的区域特征，对检测对象进行异常检测时，本公开提供的另一种可选实施方式是，根据第二光谱图像中不同图像区域的光谱特征对检测对象进行异常检测。
参见图7提供的处理方法的流程示意图，该实施方式中，图1所示处理方法中的步骤105可进一步实现为：
步骤1053、提取第二光谱图像中不同图像区域的光谱特征。
第二光谱图像中不同图像区域的光谱特征，可以包括第二光谱图像中不同图像区域的采样点分别对应的光谱分布特征。
其中，具体可提取第二光谱图像中不同图像区域的采样点的光谱信息，如不同采样点包含的各色彩通道取值，并根据采样点的光谱信息，确定采样点对不同波长光的反射率，利用采样点对不同波长光的反射率表征采样点的光谱分布特征，相应得到第二光谱图像中不同图像区域的光谱特征。
步骤1054、根据提取的光谱特征，对第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
之后以提取的光谱特征作为识别依据，对第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理，以识别检测对象不同区域是否存在异常，以及存在异常情况下的异常类型，如断胶、溢胶或异物残胶等。
与基于视觉特征的异常检测方式相类似，在基于光谱特征对检测对象进行异常检测时，一种可选实施方式是，根据提取的光谱特征及第三识别规则，识别第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，及存在异常情况下具体对应的异常类型。
该实施方式中，预先制定用于对检测对象进行异常类型识别的第三识别规则，第三识别规则包括：检测对象的不同异常类型分别对应的基准光谱特征。实际应用中，可根据检测对象的不同异常类型分别呈现的实际光谱特征(优选的为足够数据量的缺陷样本呈现的具备代表性的通用特征)，进行第三识别规 则中不同异常类型分别对应的基准光谱特征的设定。
以检测对象为点胶胶路为例，胶路缺陷为断胶时，断胶区域的材质为胶路基底的材质，与正常胶路区域的材质不同，相应导致断胶区域与正常胶路区域在第二光谱图像中分别呈现不同的光谱分布特征；胶路缺陷为溢胶时，结合参见图6，溢胶区域的胶材厚度一般与正常胶路区域的胶材厚度存在差别，相应导致溢胶区域与正常胶路区域的光谱特征产生区别；胶路缺陷为异物残胶时，缺陷区域的材质为胶材与异物的混合材质，可结合参考图6的示例，该情况下，同样会导致缺陷区域与正常胶路区域的光谱特征存在区别，并且不同缺陷类型通常分别对应不同的光谱特征。从而，可结合胶路不同缺陷类型分别呈现的实际光谱特征，对第三识别规则中不同胶路缺陷分别对应的基准光谱特征进行设定。
在另一种实施方式中，利用预先训练的第三识别模型对提取的第二光谱图像的光谱特征进行处理，得到异常识别结果。异常识别结果至少包括：第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型，可选的，异常识别结果具体可以为：第三识别模型输出的为第二光谱图像标注有其异常区域并为异常区域关联标注有具体异常类型(如，断胶、溢胶或异物残胶)的已标注光谱图像。
该实施方式中，预先训练用于对第二光谱图像进行光谱特征提取及基于提取的光谱特征实现对检测对象异常检测的第三识别模型。以检测对象为点胶胶路为例，可预先获得一系列异常及非异常点胶胶路的光谱图像，并为每一光谱图像标注对应的标签，如，为正常点胶胶路的光谱图像标注表征“胶路正常/非异常”的标签，为异常点胶胶路的光谱图像勾勒出其异常区域，并关联标注具体的异常类型(如，断胶、溢胶或异物残胶)等，得到标注有标签信息的一系列胶路光谱图像，并作为模型训练样本，在此基础上，利用训练样本对CNN(Convolutional Neural Networks，卷积神经网络)等网络模型进行训练，通过训练过程，不断使模型学习样本图像中胶路不同区域的光谱特征与其对应的区域是否异常以及异常情况下的具体异常类型间的潜在关联，进而使训练所得的模型能够提取第二光谱图像中不同区域的光谱特征，并根据第二光谱图像中不同区域的光谱特征识别该区域是否异常以及异常情况下具体对应哪种异常类型。
后续，通过将第二光谱图像输入第三识别模型，即可得到第三识别模型基于第二光谱图像中不同区域的光谱特征对检测对象进行异常检测所输出的异常检测结果。
综上所述，由于光谱特征更容易区分不同的材料，本公开根据光谱特征区分检测对象及其背景对象，相比于已有基于采用2D面阵相机或激光轮廓仪采集胶路等检测对象数据以实现异常检测的方案，能够更准确、快速的识别提取检测对象对应的图像部分，不受检测对象的变化影响，对于同型号检测对象的不同形态(如胶路调整)或换型号换产线的情况均具有非常强的适应性，同时克服了小样本问题，缩短了换型换线的准备时间，提升了对检测对象异常检测的准确率及效率。
对应于上述的处理方法，本公开还提供一种处理装置，该装置的组成结构如图8所示，具体包括：
采集模块801，用于对待采集对象进行光谱图像采集得到第一光谱图像；
确定模块802，用于确定所述第一光谱图像中不同图像区域分别对应的光谱特征；
识别模块803，用于根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数；所述属性参数与待采集对象的材料相关；所述待采集对象包括检测对象和背景对象；
获取模块804，用于根据所述第一光谱图像中不同图像区域分别对应的属性参数，确定所述检测对象在所述第一光谱图像中对应的目标区域，并获得所述目标区域形成的第二光谱图像；
检测模块805，用于基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测。
在一实施方式中，确定模块802，具体用于：
提取所述第一光谱图像中不同图像区域的采样点的光谱信息；
根据采样点的光谱信息，确定采样点对不同波长光的反射率，得到采样点的光谱分布特征；
其中，所述第一光谱图像中不同图像区域分别对应的采样点的光谱分布特征，作为所述不同图像区域分别对应的光谱特征。
在一实施方式中，识别模块803，具体用于：
根据不同图像区域分别对应的采样点的光谱分布特征及第一识别规则，识别不同图像区域分别对应的待采集对象的属性参数；其中，所述第一识别规则包括：不同属性参数分别对应的对不同波长光中每种波长光的反射率区间；
或，利用预先训练的第一识别模型对不同图像区域分别对应的采样点的光谱分布特征进行处理，得到不同图像区域分别对应的待采集对象的属性参数。
在一实施方式中，获取模块804，在获得所述目标区域形成的第二光谱图像时，具体用于：
从所述第一光谱图像中分割出所述目标区域的图像，得到所述第二光谱图像。
在一实施方式中，检测模块805，具体用于：
提取所述第二光谱图像中不同图像区域的视觉特征；
根据提取的视觉特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
在一实施方式中，检测模块805，在根据提取的视觉特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理时，具体用于：
根据提取的视觉特征及第二识别规则，识别所述第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，及存在异常情况下所对应的异常类型；其中，所述第二识别规则包括：检测对象的不同异常类型分别对应的基准视觉特征；
或，利用预先训练的第二识别模型对提取的视觉特征进行处理，得到异常识别结果；所述异常识别结果至少包括：所述第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型。
在一实施方式中，检测模块805，具体用于：
提取所述第二光谱图像中不同图像区域的光谱特征；
根据提取的光谱特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
在一实施方式中，检测模块805，在根据提取的光谱特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理时，具体 用于：
根据提取的光谱特征及第三识别规则，识别所述第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，及存在异常情况下所对应的异常类型；其中，所述第三识别规则包括：检测对象的不同异常类型分别对应的基准光谱特征；
或，利用预先训练的第三识别模型对提取的光谱特征进行处理，得到异常识别结果；所述异常识别结果至少包括：所述第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型。
对于本实施例提供的数据的处理装置而言，由于其与上文各方法实施例提供的处理方法相对应，所以描述的比较简单，相关相似之处请参见上文相应方法实施例的说明即可，此处不再详述。
另外，本公开实施例还提供一种处理系统，参见图9，该系统包括光源组件901、光谱图像采集组件902和如上一实施例提供的处理装置903。
光源组件901，用于对待采集对象进行光照处理。光源组件901的照射光中包含待采集对象(如，胶路和其周边/基底)敏感的波长的光，待采集对象包括检测对象和背景对象。
其中，可优选采用能用于发射全波段光的白炽灯作为光源组件。
光谱图像采集组件902，用于采集待采集对象的第一光谱图像。光谱图像采集组件902可以是但不限于多光谱相机或高光谱相机。
处理装置903，用于通过执行如上文任一方法实施例提供的处理方法，实现对检测对象的异常检测。
关于该处理系统中各组成部分的功能，及通过各组成部分的协同实现对检测对象的异常检测的处理过程，具体可参见上文方法实施例的相关说明，不再详述。
需要说明的是，本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。
为了描述的方便，描述以上系统或装置时以功能分为各种模块或单元分别 描述。当然，在实施本公开时可以把各单元的功能在同一个或多个软件和/或硬件中实现。
通过以上的实施方式的描述可知，本领域的技术人员可以清楚地了解到本公开可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本公开各个实施例或者实施例的某些部分所述的方法。
最后，还需要说明的是，在本文中，诸如第一、第二、第三和第四等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
以上所述仅是本公开的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本公开原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本公开的保护范围。

Claims (10)

  1. 一种处理方法，所述方法包括：
    对待采集对象进行光谱图像采集得到第一光谱图像；
    确定所述第一光谱图像中不同图像区域分别对应的光谱特征；
    根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数；所述属性参数与待采集对象的材料相关；所述待采集对象包括检测对象和背景对象；
    根据所述第一光谱图像中不同图像区域分别对应的属性参数，确定所述检测对象在所述第一光谱图像中对应的目标区域，并获得所述目标区域形成的第二光谱图像；
    基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测。
  2. 根据权利要求1所述的方法，所述确定所述第一光谱图像中不同图像区域分别对应的光谱特征，包括：
    提取所述第一光谱图像中不同图像区域的采样点的光谱信息；
    根据采样点的光谱信息，确定采样点对不同波长光的反射率，得到采样点的光谱分布特征；
    其中，所述第一光谱图像中不同图像区域分别对应的采样点的光谱分布特征，作为所述不同图像区域分别对应的光谱特征。
  3. 根据权利要求2所述的方法，所述根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数，包括：
    根据不同图像区域分别对应的采样点的光谱分布特征及第一识别规则，识别不同图像区域分别对应的待采集对象的属性参数；其中，所述第一识别规则包括：不同属性参数分别对应的对不同波长光中每种波长光的反射率区间；
    或，利用预先训练的第一识别模型对不同图像区域分别对应的采样点的光谱分布特征进行处理，得到不同图像区域分别对应的待采集对象的属性参数。
  4. 根据权利要求1所述的方法，所述获得所述目标图像区域形成的第二光谱图像，包括：
    从所述第一光谱图像中分割出所述目标区域的图像，得到所述第二光谱图 像。
  5. 根据权利要求1所述的方法，所述基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测，包括：
    提取所述第二光谱图像中不同图像区域的视觉特征；
    根据提取的视觉特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
  6. 根据权利要求5所述的方法，所述根据提取的视觉特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理，包括：
    根据提取的视觉特征及第二识别规则，识别所述第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，及存在异常情况下所对应的异常类型；其中，所述第二识别规则包括：检测对象的不同异常类型分别对应的基准视觉特征；
    或，利用预先训练的第二识别模型对提取的视觉特征进行处理，得到异常识别结果；所述异常识别结果至少包括：所述第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型。
  7. 根据权利要求1所述的方法，所述基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测，包括：
    提取所述第二光谱图像中不同图像区域的光谱特征；
    根据提取的光谱特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理。
  8. 根据权利要求7所述的方法，所述根据提取的光谱特征，对所述第二光谱图像中不同图像区域对应的检测对象区域进行异常类型的识别处理，包括：
    根据提取的光谱特征及第三识别规则，识别所述第二光谱图像中不同图像区域对应的检测对象区域是否存在异常，及存在异常情况下所对应的异常类型；其中，所述第三识别规则包括：检测对象的不同异常类型分别对应的基准光谱特征；
    或，利用预先训练的第三识别模型对提取的光谱特征进行处理，得到异常识别结果；所述异常识别结果至少包括：所述第二光谱图像中不同图像区域对应的检测对象区域存在异常情况下的异常类型。
  9. 一种处理装置，所述装置包括：
    采集模块，用于对待采集对象进行光谱图像采集得到第一光谱图像；
    确定模块，用于确定所述第一光谱图像中不同图像区域分别对应的光谱特征；
    识别模块，用于根据不同图像区域分别对应的光谱特征，识别不同图像区域分别对应的待采集对象的属性参数；所述属性参数与待采集对象的材料相关；所述待采集对象包括检测对象和背景对象；
    获取模块，用于根据所述第一光谱图像中不同图像区域分别对应的属性参数，确定所述检测对象在所述第一光谱图像中对应的目标区域，并获得所述目标区域形成的第二光谱图像；
    检测模块，用于基于所述第二光谱图像中不同图像区域的区域特征，对所述检测对象进行异常检测。
  10. 一种处理系统，所述系统包括：
    光源组件，用于对待采集对象进行光照处理；所述待采集对象包括检测对象和背景对象；
    光谱图像采集组件，用于采集所述待采集对象的第一光谱图像；
    处理装置，用于通过执行如权利要求1-8任一项所述的处理方法，实现对所述检测对象的异常检测。
PCT/CN2022/116130 2021-11-30 2022-08-31 一种处理方法及处理装置、处理系统 WO2023098187A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111452840.8 2021-11-30
CN202111452840.8A CN114152621A (zh) 2021-11-30 2021-11-30 一种处理方法及处理装置、处理系统

Publications (1)

Publication Number Publication Date
WO2023098187A1 true WO2023098187A1 (zh) 2023-06-08

Family

ID=80455513

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/116130 WO2023098187A1 (zh) 2021-11-30 2022-08-31 一种处理方法及处理装置、处理系统

Country Status (2)

Country Link
CN (1) CN114152621A (zh)
WO (1) WO2023098187A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114152621A (zh) * 2021-11-30 2022-03-08 联想(北京)有限公司 一种处理方法及处理装置、处理系统

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107808115A (zh) * 2017-09-27 2018-03-16 联想(北京)有限公司 一种活体检测方法、装置及存储介质
US20180365822A1 (en) * 2017-06-19 2018-12-20 ImpactVision, Inc. System and method for hyperspectral image processing to identify foreign object
CN109187578A (zh) * 2018-09-03 2019-01-11 贵阳学院 基于高光谱成像技术的猕猴桃表面缺陷快速无损识别方法
CN110363186A (zh) * 2019-08-20 2019-10-22 四川九洲电器集团有限责任公司 一种异常检测方法、装置及计算机存储介质、电子设备
US20190347514A1 (en) * 2017-01-25 2019-11-14 Chengdu Zhongxin Huarui Technology Co., Ltd. Detection method, device, apparatus and computer storage medium
CN112418057A (zh) * 2020-11-19 2021-02-26 中国人民解放军空军军医大学 一种基于多光谱的地面伤员识别方法及系统
CN113260882A (zh) * 2020-07-20 2021-08-13 深圳大学 金属异物检测方法、装置及终端设备
CN114152621A (zh) * 2021-11-30 2022-03-08 联想(北京)有限公司 一种处理方法及处理装置、处理系统

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108956625B (zh) * 2018-07-20 2021-05-04 Oppo广东移动通信有限公司 胶路检测方法及胶路检测装置
WO2020082264A1 (zh) * 2018-10-24 2020-04-30 合刃科技（深圳）有限公司 基于高光谱光学传感器的涂层区域定位方法和装置、及除胶系统
CN109447977B (zh) * 2018-11-02 2021-05-28 河北工业大学 一种基于多光谱深度卷积神经网络的视觉缺陷检测方法
CN110598761A (zh) * 2019-08-26 2019-12-20 深圳大学 一种点胶检测方法、装置及计算机可读存储介质
CN112926702A (zh) * 2019-12-06 2021-06-08 李雯毓 主动光源式物体材质识别系统及方法
CN112666180A (zh) * 2020-12-23 2021-04-16 浙江大学 一种点胶自动化检测方法及系统
CN112950604B (zh) * 2021-03-12 2022-04-19 深圳市鑫路远电子设备有限公司 一种精密点胶的信息处理方法及系统
CN113252625B (zh) * 2021-04-27 2022-08-16 歌尔光学科技有限公司 一种具有荧光效应的胶水的胶路检测方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190347514A1 (en) * 2017-01-25 2019-11-14 Chengdu Zhongxin Huarui Technology Co., Ltd. Detection method, device, apparatus and computer storage medium
US20180365822A1 (en) * 2017-06-19 2018-12-20 ImpactVision, Inc. System and method for hyperspectral image processing to identify foreign object
CN107808115A (zh) * 2017-09-27 2018-03-16 联想(北京)有限公司 一种活体检测方法、装置及存储介质
CN109187578A (zh) * 2018-09-03 2019-01-11 贵阳学院 基于高光谱成像技术的猕猴桃表面缺陷快速无损识别方法
CN110363186A (zh) * 2019-08-20 2019-10-22 四川九洲电器集团有限责任公司 一种异常检测方法、装置及计算机存储介质、电子设备
CN113260882A (zh) * 2020-07-20 2021-08-13 深圳大学 金属异物检测方法、装置及终端设备
CN112418057A (zh) * 2020-11-19 2021-02-26 中国人民解放军空军军医大学 一种基于多光谱的地面伤员识别方法及系统
CN114152621A (zh) * 2021-11-30 2022-03-08 联想(北京)有限公司 一种处理方法及处理装置、处理系统

Also Published As

Publication number Publication date
CN114152621A (zh) 2022-03-08

Similar Documents

Publication Publication Date Title
US11774735B2 (en) System and method for performing automated analysis of air samples
CN108229561B (zh) 一种基于深度学习的颗粒产品缺陷检测方法
CN102288613B (zh) 一种灰度和深度信息融合的表面缺陷检测方法
CN107064170B (zh) 一种检测手机外壳轮廓度缺陷方法
KR102003781B1 (ko) 초분광영상화 기법을 이용한 글라스(Glass) 결함 검출 장치
CN111667455A (zh) 一种刷具多种缺陷的ai检测方法
CN112184648A (zh) 一种基于深度学习的活塞表面缺陷检测方法及系统
CN103366176B (zh) 光学元件缺陷批量自动识别装置和方法
Eshkevari et al. Automatic dimensional defect detection for glass vials based on machine vision: A heuristic segmentation method
WO2023098187A1 (zh) 一种处理方法及处理装置、处理系统
CN111126393A (zh) 车辆外观改装判断方法、装置、计算机设备及存储介质
WO2024051067A1 (zh) 红外图像处理方法、装置及设备、存储介质
CN111122590A (zh) 一种陶瓷表面缺陷检测装置及检测方法
CN109115719A (zh) 一种基于高光谱成像技术的柑橘黄龙病波段融合快速检测方法
WO2024002187A1 (zh) 缺陷检测方法、缺陷检测设备及存储介质
Kline et al. Automated hardwood lumber grading utilizing a multiple sensor machine vision technology
CN114359235A (zh) 一种基于改进YOLOv5l网络的木材表面缺陷检测方法
CN116596875A (zh) 晶圆缺陷检测方法、装置、电子设备及存储介质
CN114441452B (zh) 一种光纤尾纤检测方法
CN115359264A (zh) 一种密集型分布的粘连细胞深度学习识别方法
KR20190114241A (ko) 딥러닝 기반의 독성조류 판별 및 셀카운팅 장치 및 그 방법
CN113793322A (zh) 一种对磁性材料自动检测的方法、电子设备和存储介质
Kuo et al. Automated inspection of micro-defect recognition system for color filter
TW201601119A (zh) 物件辨識與定位方法
Bhutta et al. Smart-inspect: micro scale localization and classification of smartphone glass defects for industrial automation

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22900001

Country of ref document: EP

Kind code of ref document: A1