CN118505708B - Product repairing system based on image reconstruction technology - Google Patents
Product repairing system based on image reconstruction technology Download PDFInfo
- Publication number
- CN118505708B CN118505708B CN202410969790.8A CN202410969790A CN118505708B CN 118505708 B CN118505708 B CN 118505708B CN 202410969790 A CN202410969790 A CN 202410969790A CN 118505708 B CN118505708 B CN 118505708B
- Authority
- CN
- China
- Prior art keywords
- defect
- product
- image
- area
- reconstruction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005516 engineering process Methods 0.000 title claims abstract description 25
- 230000007547 defect Effects 0.000 claims abstract description 244
- 238000011156 evaluation Methods 0.000 claims abstract description 87
- 238000000034 method Methods 0.000 claims abstract description 29
- 230000011218 segmentation Effects 0.000 claims description 84
- 230000008439 repair process Effects 0.000 claims description 19
- 230000002159 abnormal effect Effects 0.000 claims description 13
- 238000007689 inspection Methods 0.000 claims description 10
- 238000013507 mapping Methods 0.000 claims description 9
- 238000004458 analytical method Methods 0.000 claims description 8
- 230000002950 deficient Effects 0.000 claims description 7
- 230000004807 localization Effects 0.000 claims description 6
- 238000005192 partition Methods 0.000 claims description 6
- 238000012216 screening Methods 0.000 claims description 4
- 230000001419 dependent effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 21
- 238000012545 processing Methods 0.000 abstract description 17
- 230000000694 effects Effects 0.000 abstract description 3
- 230000006870 function Effects 0.000 description 9
- 238000005259 measurement Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 3
- 238000005286 illumination Methods 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000003708 edge detection Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000012098 association analyses Methods 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007797 corrosion Effects 0.000 description 1
- 238000005260 corrosion Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001066 destructive effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000001125 extrusion Methods 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000000638 solvent extraction Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000001931 thermography Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a product repairing system based on an image reconstruction technology, which relates to the technical field of image processing, and comprises the following components: collecting product image information and carrying out defect identification; performing defect evaluation according to product defect positioning and defect characteristics; setting constraint coefficients according to defect influence, and dividing defect positioning areas according to defect area reconstruction accuracy; dividing the defect positioning of the product, and carrying out image reconstruction on the divided areas to obtain a reconstructed image of the divided areas; and performing splice smoothness evaluation, and performing self-adaptive reconstruction control on a splice smoothness evaluation result by using the constraint coefficient until the constraint coefficient is satisfied, and obtaining a reconstruction restoration strategy for feedback. The technical problems that the damage position and the damage degree of a product are difficult to accurately identify in the existing product repairing process, the product repairing process is not efficient enough, and the result is not accurate enough are solved, and the technical effects of improving the efficiency of the product repairing process and the accuracy of the repairing result are achieved.
Description
Technical Field
The application relates to the field of image processing technology, in particular to a product restoration system based on an image reconstruction technology.
Background
With the wear and aging of products in daily use, the appearance and functions of the products may be damaged to different degrees, which not only affects the service life of the products, but also may negatively affect the experience of users, and in the technological age of increasing development nowadays, the image reconstruction technology has become an important technology in the field of product repair, however, the traditional product repair technology generally depends on physical entity repair, and for product components with complex damage conditions, it is difficult to accurately identify and position the damage position and damage degree of the product components only by naked eyes, so that the product repair is difficult to meet the requirements of high precision of repair results and high efficiency of repair processes.
Therefore, in the related technology of product restoration at the present stage, the key parameters in the extrusion process are difficult to monitor in real time and accurately control, so that the technical problems of unstable product quality and low production efficiency are caused.
Disclosure of Invention
The product repairing system based on the image reconstruction technology solves the technical problems that the product repairing process is not efficient enough and the result is not accurate enough due to the fact that the damaged position and the damaged degree of the product are difficult to accurately identify in the existing product repairing process by adopting the technical means such as defect segmentation, image reconstruction and the like, and achieves the technical effects of improving the efficiency of the product repairing process and the accuracy of the repairing result.
The application provides a product restoration system based on an image reconstruction technology, which comprises: the product defect identification module is used for collecting product image information, carrying out defect identification and obtaining product defect positioning and defect characteristics; the defect evaluation information acquisition module is used for carrying out defect evaluation according to the product defect positioning and defect characteristics to acquire defect evaluation information, wherein the defect evaluation information comprises defect influence and defect area reconstruction accuracy; the defect positioning area segmentation module is used for setting constraint coefficients according to the defect influence and carrying out defect positioning area segmentation according to the defect area reconstruction accuracy; the product image reconstruction module is used for dividing the product defect positioning based on the defect positioning area dividing result, reconstructing the image of the divided area and obtaining a reconstructed image of the divided area; and the reconstruction restoration strategy obtaining module is used for evaluating the splicing smoothness according to the reconstruction images of the segmentation areas, and carrying out self-adaptive reconstruction control on the splicing smoothness evaluation result by utilizing the constraint coefficients until the constraint coefficients are met, so as to obtain a reconstruction restoration strategy for feedback.
In a possible implementation manner, the product defect identifying module further performs the following processing: obtaining standard image characteristics of a product; performing feature segmentation according to the product standard image features to construct a feature segmentation positioning set; correspondingly dividing the product image information based on the characteristic division positioning set, and constructing a convolution kernel based on the region division characteristics in the characteristic division positioning set; performing feature traversal on the segmentation results of the product image information by using the convolution check to obtain an abnormal feature recognition result; and carrying out segmentation region positioning on the product defects according to the abnormal characteristic recognition result, and obtaining defect characteristics, wherein the defect characteristics comprise difference information of the abnormal characteristic recognition result and region segmentation characteristics.
In a possible implementation manner, the product defect identifying module further performs the following processing: judging whether the product has standardized quality inspection characteristics, and acquiring the standardized quality inspection characteristics as the standard image characteristics of the product when the standardized quality inspection characteristics exist; when the product image information does not exist, binary data of the product image information is obtained; and carrying out cluster segmentation based on the binarized data, and determining a binary threshold value of each cluster segmentation area as the product standard image characteristic.
In a possible implementation manner, the defect-assessment-information obtaining module further performs the following processing: acquiring the functional requirement and the ornamental requirement of the product; analyzing based on the functional requirement and the ornamental requirement of the product, matching with the appearance structure of the product, and establishing a mapping relation between the structure of the product and the target requirement; according to the mapping relation, respectively carrying out influence analysis on the product appearance structure, the target requirement, the defect characteristic and the target requirement, and constructing a fuzzy evaluation list, wherein the fuzzy evaluation list comprises product structure partitions, defect characteristics and corresponding influences; performing influence configuration through the fuzzy evaluation list according to the product defect positioning and defect characteristics to obtain the defect influence; acquiring a historical reconstruction case, and according to the historical reconstruction case, fitting by taking the size of the defect area as an independent variable and the reconstruction accuracy as a dependent variable to determine a defect area reconstruction evaluation function; and carrying out accuracy evaluation on the area of the product defect positioning by using the defect area reconstruction evaluation function to obtain the defect area reconstruction accuracy.
In a possible implementation manner, the defect positioning area dividing module further performs the following processing: screening according to the historical reconstruction case and the reconstruction accuracy to obtain an area parameter set, wherein the area parameter set comprises a minimum segmentation area and the corresponding accuracy; when the defect area reconstruction accuracy meets a preset threshold value, skipping a defect positioning area segmentation step; when the defect area reconstruction accuracy does not meet a preset threshold value, starting a defect positioning area segmentation step; matching the accuracy of the area parameter set with the accuracy of the preset threshold value to obtain the minimum segmentation area; and according to the minimum area for segmentation, carrying out area segmentation on the defect positioning area with the positioning edge maximization.
In a possible implementation manner, the defect positioning area dividing module further performs the following processing: acquiring the edge length of the defect positioning area; dividing the edge length by utilizing the minimum side length of the minimum dividing area to obtain an edge dividing result; obtaining the residual defect area according to the edge segmentation result; and dividing the residual defect area from the edge to the inside by utilizing the dividing minimum area until the dividing of all defect positioning areas is completed.
In a possible implementation manner, the reconstruction repair policy obtaining module further performs the following processing: acquiring a defective edge connection image as a sample target image according to the product defect positioning; determining a countermeasure image relationship based on the position relationship of the segmentation area and the sample target image, wherein the countermeasure image relationship comprises the sample target image and a corresponding reconstructed image; respectively carrying out peak signal-to-noise ratio, structural similarity and integrity evaluation on the sample target image and the reconstructed image according to the countermeasure image relation; and configuring an evaluation weight, and calculating a peak signal-to-noise ratio, structural similarity and an integrity evaluation value based on the evaluation weight to obtain the splicing smoothness evaluation result.
In a possible implementation manner, the reconstruction repair policy obtaining module further performs the following processing: by the formula:
Calculating to obtain the splicing smoothness evaluation result, wherein, Respectively the evaluation weight values of peak signal-to-noise ratio, structural similarity and integrity evaluation values,The sum is 1,Is the maximum pixel value,A pixel for a sample target image,And (3) withPixels of the reconstructed image at corresponding positions,Mean value of pixels of sample target image,An average value of pixels of the reconstructed image,Pixel variance for the sample target image,Pixel variance of reconstructed image,Is covariance (covariance),、To avoid a constant with zero denominator, n is the total number of pixels, and m is the number of stitched pixel pairs.
The product repairing system based on the image reconstruction technology is used for collecting product image information and carrying out defect identification; performing defect evaluation according to product defect positioning and defect characteristics; setting constraint coefficients according to defect influence, and dividing defect positioning areas according to defect area reconstruction accuracy; dividing the defect positioning of the product, and carrying out image reconstruction on the divided areas to obtain a reconstructed image of the divided areas; and performing splice smoothness evaluation, and performing self-adaptive reconstruction control on a splice smoothness evaluation result by using the constraint coefficient until the constraint coefficient is satisfied, and obtaining a reconstruction restoration strategy for feedback. The technical problems that the damage position and the damage degree of a product are difficult to accurately identify in the existing product repairing process, the product repairing process is not efficient enough, and the result is not accurate enough are solved, and the technical effects of improving the efficiency of the product repairing process and the accuracy of the repairing result are achieved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly describe the drawings of the embodiments of the present disclosure, in which flowcharts are used to illustrate operations performed by a system according to embodiments of the present disclosure. It should be understood that the preceding or following operations are not necessarily performed in order precisely. Rather, the various steps may be processed in reverse order or simultaneously, as desired. Also, other operations may be added to or removed from these processes.
Fig. 1 is a schematic structural diagram of a product repairing system based on an image reconstruction technology according to an embodiment of the present application;
Fig. 2 is a schematic diagram illustrating an execution process of a product defect recognition module in a product repairing system based on an image reconstruction technology according to an embodiment of the present application.
Reference numerals illustrate: the system comprises a product defect identification module 10, a defect evaluation information acquisition module 20, a defect positioning area segmentation module 30, a product image reconstruction module 40 and a reconstruction restoration strategy acquisition module 50.
Detailed Description
The foregoing description is only an overview of the present application, and is intended to be implemented in accordance with the teachings of the present application in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present application more readily apparent.
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described in further detail with reference to the accompanying drawings, and the described embodiments should not be construed as limiting the present application, and all other embodiments obtained by those skilled in the art without making any inventive effort are within the scope of the present application.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is to be understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict, the term "first\second" being referred to merely as distinguishing between similar objects and not representing a particular ordering for the objects. The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, article, or apparatus, but may include other steps or modules not expressly listed or inherent to such process, article, or apparatus, and unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for the purpose of describing embodiments of the application only.
An embodiment of the present application provides a product repairing system based on an image reconstruction technology, as shown in fig. 1, the system includes:
The product defect identification module 10 is used for acquiring product image information, carrying out defect identification and acquiring product defect positioning and defect characteristics. The method comprises the steps of shooting or scanning a product by using image acquisition equipment (such as a camera, a scanner and the like), acquiring image information of the product, processing the acquired image of the product through image processing and analysis technology, identifying defects in the product, such as image enhancement, segmentation, feature extraction and the like, so as to accurately identify the defects of the product, determining specific positions of the defects on the product after identifying the defects of the product, and acquiring detailed defect feature information including the size, shape, color, texture and other features of the defects, wherein a plurality of methods can be used for identifying the defects of the product, such as a rule-based detection method, an infrared thermal imaging technology, a laser scanning technology and the like.
Next, the specific configuration of the product defect recognition module 10 will be described in detail. As shown in fig. 2, the product defect recognition module 10 may further include: and obtaining standard image characteristics of the product. Information representative of the product, which can describe the product attribute or characteristic, is extracted from the standard image of the product. And carrying out feature segmentation according to the product standard image features to construct a feature segmentation positioning set. Different areas or objects in the image are distinguished and separated according to the characteristics of the product standard image, for example, threshold-based segmentation, edge-based segmentation, region-based segmentation and the like, different parts in the image are divided according to the characteristic values (such as gray values, colors, textures and the like) of the different parts to form different segmented areas, a characteristic segmentation positioning set is constructed according to the segmentation result, the position of all the segmented areas and the characteristic information are contained, and the specific position (such as a coordinate range) of each segmented area in the image and the characteristic values (such as colors, textures and the like) of the area are recorded. And correspondingly dividing the product image information based on the characteristic division positioning set, and constructing a convolution kernel based on the region division characteristics in the characteristic division positioning set. According to the feature segmentation positioning set, carrying out corresponding accurate segmentation on the product image information to obtain image blocks corresponding to each region in the feature segmentation positioning set, further extracting the features of the image blocks, and constructing a convolution kernel according to the features, for example, if a certain region has a defect, a convolution kernel capable of highlighting the defect may be needed; if a region requires smoothing, a convolution kernel that can smooth the image may be required. And respectively performing feature traversal on the segmentation results of the product image information by utilizing the convolution check to obtain an abnormal feature recognition result. And carrying out convolution operation on each divided area of the product image information by utilizing convolution check to generate a corresponding feature map, and then carrying out feature traversal to find out an area which is inconsistent with the normal mode, thereby obtaining an abnormal feature recognition result. And carrying out segmentation region positioning on the product defects according to the abnormal characteristic recognition result, and obtaining defect characteristics, wherein the defect characteristics comprise difference information of the abnormal characteristic recognition result and region segmentation characteristics. Based on the abnormal feature recognition result, the product image is subjected to more accurate segmentation region positioning, namely, the position and the shape of the abnormal feature are compared with the information in the original image or region segmentation positioning set, the region where the defect matched with the original image or region is located is found, and after the segmentation region of the defect is determined, the feature information of the region, namely, the defect feature, including the difference information of the abnormal feature recognition result and the region segmentation feature, possibly the gray level difference, texture difference, color difference and the like of the defect region and the normal region, is further extracted.
Next, the specific configuration of the product defect recognition module 10 will be described in further detail. The product defect recognition module 10 may further include: judging whether the product has standardized quality inspection characteristics, and acquiring the standardized quality inspection characteristics as the standard image characteristics of the product when the standardized quality inspection characteristics exist. If standardized image requirements exist for mass-produced products, the standardized features are directly utilized for identification traversal, and the standardized quality inspection features are used as the standard image features of the products. When the product image information does not exist, binary data of the product image information is obtained; and carrying out cluster segmentation based on the binarized data, and determining a binary threshold value of each cluster segmentation area as the product standard image characteristic. For images that are not batched, nor are standard, such as solitary, cultural relics and the like, it is not known how the standard state before it is, threshold segmentation is performed according to binarization of a product image to obtain product standard image features, wherein binarization is a basic operation in image processing, pixel values of the image are simplified into two levels, usually 0 (black) and 255 (white), when cluster segmentation is performed on the binary image, a common method is based on connectivity, adjacent white (or black) pixels are regarded as the same cluster, for example, morphological operations (such as corrosion and expansion) can be used to remove noise and connect adjacent white pixels, and then a labeling algorithm (such as a connected component labeling algorithm) is used to identify and label different areas in the image, and the binary threshold of each cluster segmentation area is taken as the product standard image features.
The defect evaluation information obtaining module 20 is configured to perform defect evaluation according to the product defect positioning and defect characteristics, and obtain defect evaluation information including defect influence and defect area reconstruction accuracy. According to the obtained defect positioning and defect characteristics of the product, performing defect association analysis and evaluation by using association rule mining algorithm or machine learning and the like through sample data, determining the influence relation of each structure, obtaining defect evaluation information, including defect influence and defect area reconstruction accuracy, for example, performing comprehensive analysis and evaluation on the defect of the product, determining the nature, severity, influence range and repair necessity of the defect, wherein the defect influence refers to the influence degree of the defect on the aspects of product performance, safety, reliability or use experience and the like, and can be direct (such as product functional failure) or indirect (such as product service life reduction or maintenance cost increase), for example, a defect which can immediately cause the product to stop working is more serious than a defect which needs a period of time to appear at the edge or inside of the product, the defect has little influence on the appearance or function of the product, the constraint coefficient is low, the larger the defect area is, the accuracy of the reconstruction is reduced, and the reconstruction accuracy of the tiny defect area is high; defect area reconstruction accuracy refers to the ability to accurately identify and measure defect areas in a product image through image processing and analysis technologies, and when evaluating defect area reconstruction accuracy, measurement accuracy (error between a reconstructed defect area and an actual defect area should be as small as possible), measurement repeatability (similar or identical results should be obtained when the same defect area is measured for multiple times), measurement stability (measurement results should be kept relatively stable under different illumination, angles or shooting conditions), measurement integrity (the reconstructed defect area should cover the whole range of the actual defect to avoid omission or misjudgment), for example, accurate defect area measurement can evaluate the severity and influence of the defect more accurately.
Next, the specific configuration of the defect-evaluation-information obtaining module 20 will be described in detail. The defect-assessment-information obtaining module 20 may further include: the functional requirement and the ornamental requirement of the product are obtained. Functional requirements refer to a particular function or act that a product or system should have, describing how software or hardware should perform to meet a predetermined objective; the ornamental demands are more focused on the aspects of appearance, design, style and the like of products, and aim to meet the pursuit and aesthetic demands of users on beauty. Analyzing based on the functional requirement and the ornamental requirement of the product, matching with the appearance structure of the product, and establishing a mapping relation between the product structure and the target requirement. Analyzing the functional requirement and the ornamental requirement of the product, determining how each structural element meets the specific functional requirement or the ornamental requirement, and establishing a mapping relation between the product structure and the target requirement so as to ensure that the requirements can be effectively reflected in the appearance structure of the product. And respectively carrying out influence analysis on the product appearance structure, the target requirement, the defect characteristic and the target requirement according to the mapping relation, and constructing a fuzzy evaluation list, wherein the fuzzy evaluation list comprises product structure partitions, defect characteristics and corresponding influences. According to the established mapping relation, integrating the target requirement, the defect characteristic and the influence analysis result of the target requirement, and analyzing how each partition of the product appearance structure meets or does not meet the functional and ornamental requirements, for example, whether a button is positioned for convenient operation by a user, whether the resolution of a display screen is high enough to meet the visual requirements of the user, whether the design of a shell is attractive and meets the aesthetic requirements of the user, and the like; for the identified defect characteristics (such as insensitive buttons, dead spots on the display screen, scratches on the shell, etc.), how the defects affect the functionality and the ornamental requirement of the product is analyzed, for example, an insensitive button may cause a user to fail to use a certain function of the product, and the dead spots on the display screen may reduce the visual experience of the user; determining evaluation factors, assigning corresponding evaluation grades to each evaluation factor, recording the evaluation factors in a fuzzy evaluation list, and displaying different partitions of a product structure, defect characteristics and influences (namely, the evaluation grades) corresponding to the different partitions and the defect characteristics. And carrying out influence configuration through the fuzzy evaluation list according to the product defect positioning and defect characteristics to obtain the defect influence. And configuring corresponding influence grades, such as slight influence, general influence, serious influence and the like, for each defect in the fuzzy evaluation list according to the defect positioning and defect characteristics of the product, and finally obtaining the defect influence. And acquiring a historical reconstruction case, and according to the historical reconstruction case, fitting by taking the size of the defect area as an independent variable and the reconstruction accuracy as a dependent variable to determine a defect area reconstruction evaluation function. The defect area reconstruction evaluation function can predict the accuracy of reconstruction according to the given defect area size. And carrying out accuracy evaluation on the area of the product defect positioning by using the defect area reconstruction evaluation function to obtain the defect area reconstruction accuracy.
The defect positioning area segmentation module 30 is configured to set a constraint coefficient according to the defect influence, and segment the defect positioning area according to the defect area reconstruction accuracy. Setting a constraint coefficient according to defect influence, wherein the constraint coefficient is a reconstructed precision requirement coefficient, is a quantization index and is used for representing influence of the reconstructed precision requirement of the product defect, and the reconstructed precision requirement of the product defect can be determined through the constraint coefficient to guide subsequent defect treatment; in the defect identification and reconstruction process, the preliminary location and area information of the defect can be obtained through an image processing and analyzing technology, however, due to various factors (such as illumination, noise, shooting angles and the like) in the image acquisition and processing process, certain errors may exist in the preliminary defect location and area information, the defect location area is further divided and corrected according to the accuracy of defect area reconstruction, the preliminary identified defect area is subdivided into smaller areas so as to more accurately determine the position and size of the defect, for example, the product image is processed more finely by using edge detection, and more accurate defect position and area information is obtained.
Next, the specific configuration of the defect localization area splitting module 30 will be described in detail. The defect localization area splitting module 30 may further include: and screening according to the reconstruction accuracy according to the historical reconstruction case to obtain an area parameter set, wherein the area parameter set comprises a minimum segmentation area and corresponding accuracy. And screening all cases from the historical reconstruction case database, and sorting according to the reconstruction accuracy of the cases, namely by comparing reconstruction accuracy indexes (such as similarity or difference between the reconstructed product and the original product) in each case, so as to obtain an area parameter set, wherein the area parameter set comprises the minimum segmentation area and the corresponding reconstruction accuracy. And when the defect area reconstruction accuracy meets a preset threshold value, skipping a defect positioning area segmentation step. The preset threshold value refers to a reconstruction accuracy threshold value preset in the defect reconstruction process. And when the defect area reconstruction accuracy does not meet a preset threshold value, starting a defect positioning area segmentation step. The defect positioning area segmentation is a process for accurately identifying the defect position and the size range thereof, and more detailed and accurate defect information can be obtained through the defect positioning area segmentation, so that subsequent reconstruction or repair work is guided. And matching the accuracy of the area parameter set with the accuracy of the preset threshold value to obtain the minimum segmentation area. In the area parameter set, from the minimum dividing area, the area parameter set is compared with a preset threshold gradually. If the reconstruction accuracy corresponding to a certain segmented area reaches or exceeds a preset threshold, the segmented area is considered as a segmented minimum area meeting the requirements. And according to the minimum area for segmentation, carrying out area segmentation on the defect positioning area with the positioning edge maximization. After determining the minimum area for segmentation, the area of the defective area is re-segmented, while ensuring that the integrity and maximization of the positioning edge is maintained as much as possible during the segmentation process, because of the defect of the edge, because of the connected product features, reconstruction, such as texture, can be performed relatively easily, and the reconstruction can be performed with reference to the connected part without defect, but for the middle area, reconstruction is required according to the reconstructed image because of the non-aligned product artwork, and the error is relatively large.
Next, the specific configuration of the defect localization area splitting module 30 will be described in further detail. The defect localization area splitting module 30 may further include: and acquiring the edge length of the defect positioning area. And carrying out edge detection on the defect area which is positioned, and calculating the total length of the edge of the defect area. And dividing the edge length by utilizing the minimum side length of the dividing minimum area to obtain an edge dividing result. The use of the minimum edge length as a reference to segment the already detected defect edge length means that the edge length is divided into a plurality of parts, each of which has a length close to or equal to the minimum edge length, and finally an edge segmentation result is obtained. And obtaining the residual defect area according to the edge segmentation result. The edge segmentation result is given, and the remaining defect area is determined by calculating the area of the area surrounded by the segments. And dividing the residual defect area from the edge to the inside by utilizing the dividing minimum area until the dividing of all defect positioning areas is completed.
And the product image reconstruction module 40 is used for dividing the product defect positioning based on the defect positioning area dividing result, reconstructing the image of the divided area and obtaining a reconstructed image of the divided area. Further fine segmentation is performed on the product defect positioning according to the defect positioning area segmentation result to obtain a specific position and shape of the defect, after the segmentation area of the defect is obtained, an image reconstruction technology is used to reconstruct a part of the defect area, wherein the image reconstruction is a computer vision technology, a High Resolution (HR) image can be generated from a Low Resolution (LR) image, or information of an original image can be recovered from a damaged image, for example, a super resolution technology generally uses a deep learning model, such as a Convolutional Neural Network (CNN) or a generating countermeasure network (GAN), a super resolution model is trained by learning mapping relations between a large number of high and low resolution image pairs, after model training is completed, the low resolution image is input into the model, an output of the super resolution model is obtained to predict and generate the high resolution image, and the segmentation area is reconstructed in order to recover the original state of the segmented area, and finally a reconstructed image of the segmentation area is obtained.
And the reconstruction restoration strategy obtaining module 50 is configured to perform a stitching smoothness evaluation according to the reconstructed image of the segmented area, and perform adaptive reconstruction control on a stitching smoothness evaluation result by using the constraint coefficient until the constraint coefficient is satisfied, so as to obtain a reconstruction restoration strategy for feedback. After the image is reconstructed, the reconstructed image with the segmentation area is spliced with the original image or other reconstructed images to form a complete image, the smoothness evaluation is mainly to observe and analyze the edge part of the spliced image and check whether obvious discontinuous, misplacement or blurring phenomenon exists, if the smoothness evaluation result shows that the smoothness of the spliced image does not meet the requirement (namely, does not meet the set constraint coefficient), the adaptive reconstruction control is carried out on the spliced image by utilizing the constraint coefficient, namely, the parameters of the image reconstruction are adjusted according to the smoothness evaluation result, the image reconstruction and splicing are carried out again, the reconstruction restoration strategy is generated and fed back until the smoothness of the spliced image meets the set constraint coefficient, and the method can comprise the scheme of product restoration and the reminding of products which cannot be restored or are not high in restoration reliability.
Next, the specific configuration of the reconstruction restoration policy obtaining module 50 will be described in detail. The reconstruction repair policy obtaining module 50 may further include: and acquiring a defective edge connection image as a sample target image according to the product defect positioning. And carrying out finer processing analysis on the defect positioning of the product to obtain a defect edge connection image as a sample target image, particularly, carrying out connection processing on the broken edges to form a complete and continuous defect edge contour due to the fact that the extracted edges are broken or discontinuous due to the fact that noise, illumination and other factors possibly influence in the actual detection process, and finally marking the position and the shape of the defect in the original image according to the connected defect edge contour to generate an image containing the complete defect edge. Based on the positional relationship of the segmented area and the sample target image, a countermeasure image relationship is determined, the countermeasure image relationship including a sample target image and a corresponding reconstructed image. The contrast image relationship refers to a set of image pairs including a sample target image (i.e., an original image containing a defect) and a corresponding reconstructed image (i.e., an image that has been processed or transformed in an attempt to simulate or repair the defect). And respectively carrying out peak signal-to-noise ratio, structural similarity and integrity evaluation on the sample target image and the reconstructed image according to the countermeasure image relation. The peak signal-to-noise ratio, the structural similarity and the integrity of the sample target image and the reconstructed image are evaluated according to the contrast image relationship, namely, the quality of the two images is evaluated, specifically, the peak signal-to-noise ratio is calculated to be the ratio between the maximum power of a signal and the destructive noise power affecting the representation precision of the peak signal-to-noise ratio, and the ratio is used for quantifying the difference between the sample target image (original image) and the reconstructed image, and a higher peak signal-to-noise ratio value indicates that the difference between the two images is smaller, namely, the quality of the reconstructed image is higher; The structural similarity is calculated based on the mean value, variance and covariance of the sample, reflects the similarity of the images on the structure, is used for evaluating the similarity of a target image of the sample and a reconstructed image on the structure, and the higher structural similarity indicates that the two images are similar on the structure, namely the reconstructed image retains the structural information of the original image; the integrity assessment is a comparison of defective areas in the sample target image and the reconstructed image to assess whether the reconstructed image successfully recovers defective areas in the original image, for example, by calculating pixel differences or structural differences of the defective areas. And configuring an evaluation weight, and calculating a peak signal-to-noise ratio, structural similarity and an integrity evaluation value based on the evaluation weight to obtain the splicing smoothness evaluation result. Respectively configuring a weight for the peak signal-to-noise ratio, the structural similarity and the integrity evaluation value to represent the importance degree of each index in comprehensive evaluation, and performing weighted operation on the peak signal-to-noise ratio, the structural similarity and the integrity evaluation value by using the configured weight, for example, if the weight of the peak signal-to-noise ratio is 0.3, the weight of the structural similarity is 0.5 and the weight of the integrity evaluation is 0.2, the comprehensive evaluation result can be calculated by the following formula: comprehensive evaluation result=0.3×peak signal-to-noise value+0.5×structural similarity value+0.2×integrity evaluation value, the obtained comprehensive evaluation result is the splice smoothness evaluation result, Reflecting the overall quality of the stitched image in several respects.
Next, the specific configuration of the reconstruction restoration policy obtaining module 50 will be described in further detail. The reconstruction repair policy obtaining module 50 may further include: the calculation formula of the splicing smoothness evaluation result is as follows:
；
wherein, Respectively the evaluation weight values of peak signal-to-noise ratio, structural similarity and integrity evaluation values,The sum is 1,Is the maximum pixel value,A pixel for a sample target image,And (3) withPixels of the reconstructed image at corresponding positions,Mean value of pixels of sample target image,An average value of pixels of the reconstructed image,Pixel variance for the sample target image,Pixel variance of reconstructed image,Is covariance (covariance),、To avoid a constant with zero denominator, n is the total number of pixels, and m is the number of stitched pixel pairs.
Although the present application makes various references to certain modules in a system according to embodiments of the present application, any number of different modules may be used and run on a user terminal and/or server, including units and modules that are merely partitioned by functional logic, but are not limited to the above-described partitioning, so long as the corresponding functionality is enabled; in addition, the specific names of the functional units are also only for distinguishing from each other, and are not used to limit the protection scope of the present application.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.
Claims (6)
1. A product repair system based on image reconstruction techniques, the system comprising:
The product defect identification module is used for collecting product image information, carrying out defect identification and obtaining product defect positioning and defect characteristics;
The defect evaluation information acquisition module is used for carrying out defect evaluation according to the product defect positioning and defect characteristics to acquire defect evaluation information, wherein the defect evaluation information comprises defect influence and defect area reconstruction accuracy;
the defect positioning area segmentation module is used for setting constraint coefficients according to the defect influence and carrying out defect positioning area segmentation according to the defect area reconstruction accuracy;
The product image reconstruction module is used for dividing the product defect positioning based on the defect positioning area dividing result, reconstructing the image of the divided area and obtaining a reconstructed image of the divided area;
The reconstruction restoration strategy obtaining module is used for evaluating the splicing smoothness according to the reconstruction images of the segmentation areas, and performing self-adaptive reconstruction control on the splicing smoothness evaluation result by utilizing the constraint coefficients until the constraint coefficients are met, so as to obtain a reconstruction restoration strategy for feedback;
The reconstruction restoration strategy obtaining module performs the steps of:
acquiring a defective edge connection image as a sample target image according to the product defect positioning;
Determining a countermeasure image relationship based on the position relationship of the segmentation area and the sample target image, wherein the countermeasure image relationship comprises the sample target image and a corresponding reconstructed image;
Respectively carrying out peak signal-to-noise ratio, structural similarity and integrity evaluation on the sample target image and the reconstructed image according to the countermeasure image relation;
configuring an evaluation weight, and calculating a peak signal-to-noise ratio, a structural similarity and an integrity evaluation value based on the evaluation weight to obtain the splicing smoothness evaluation result;
The reconstruction restoration strategy obtaining module performs the steps of:
by the formula:
Calculating to obtain the splicing smoothness evaluation result, wherein, Respectively the evaluation weight values of peak signal-to-noise ratio, structural similarity and integrity evaluation values,The sum is 1,Is the maximum pixel value,A pixel for a sample target image,And (3) withPixels of the reconstructed image at corresponding positions,Mean value of pixels of sample target image,An average value of pixels of the reconstructed image,Pixel variance for the sample target image,Pixel variance of reconstructed image,Is covariance (covariance),、To avoid a constant with zero denominator, n is the total number of pixels, and m is the number of stitched pixel pairs.
2. The image reconstruction technology based product repair system of claim 1, wherein the product defect identification module performs the steps comprising:
Obtaining standard image characteristics of a product;
Performing feature segmentation according to the product standard image features to construct a feature segmentation positioning set;
correspondingly dividing the product image information based on the characteristic division positioning set, and constructing a convolution kernel based on the region division characteristics in the characteristic division positioning set;
Performing feature traversal on the segmentation results of the product image information by using the convolution check to obtain an abnormal feature recognition result;
And carrying out segmentation region positioning on the product defects according to the abnormal characteristic recognition result, and obtaining defect characteristics, wherein the defect characteristics comprise difference information of the abnormal characteristic recognition result and region segmentation characteristics.
3. The image reconstruction technology based product repair system of claim 2, wherein the product defect identification module performs the steps comprising:
Judging whether the product has standardized quality inspection characteristics, and acquiring the standardized quality inspection characteristics as the standard image characteristics of the product when the standardized quality inspection characteristics exist;
When the product image information does not exist, binary data of the product image information is obtained;
And carrying out cluster segmentation based on the binarized data, and determining a binary threshold value of each cluster segmentation area as the product standard image characteristic.
4. The image reconstruction technique-based product repair system of claim 1, wherein the defect-assessment-information obtaining module performs the steps comprising:
Acquiring the functional requirement and the ornamental requirement of the product;
Analyzing based on the functional requirement and the ornamental requirement of the product, matching with the appearance structure of the product, and establishing a mapping relation between the structure of the product and the target requirement;
according to the mapping relation, respectively carrying out influence analysis on the product appearance structure, the target requirement, the defect characteristic and the target requirement, and constructing a fuzzy evaluation list, wherein the fuzzy evaluation list comprises product structure partitions, defect characteristics and corresponding influences;
Performing influence configuration through the fuzzy evaluation list according to the product defect positioning and defect characteristics to obtain the defect influence;
Acquiring a historical reconstruction case, and according to the historical reconstruction case, fitting by taking the size of the defect area as an independent variable and the reconstruction accuracy as a dependent variable to determine a defect area reconstruction evaluation function;
And carrying out accuracy evaluation on the area of the product defect positioning by using the defect area reconstruction evaluation function to obtain the defect area reconstruction accuracy.
5. The image reconstruction technique based product repair system of claim 4 wherein said defect localization area segmentation module performs the steps comprising:
screening according to the historical reconstruction case and the reconstruction accuracy to obtain an area parameter set, wherein the area parameter set comprises a minimum segmentation area and the corresponding accuracy;
When the defect area reconstruction accuracy meets a preset threshold value, skipping a defect positioning area segmentation step;
When the defect area reconstruction accuracy does not meet a preset threshold value, starting a defect positioning area segmentation step;
matching the accuracy of the area parameter set with the accuracy of the preset threshold value to obtain the minimum segmentation area;
and according to the minimum area for segmentation, carrying out area segmentation on the defect positioning area with the positioning edge maximization.
6. The image reconstruction technique based product repair system of claim 5 wherein said defect localization area segmentation module performs the steps comprising:
acquiring the edge length of the defect positioning area;
Dividing the edge length by utilizing the minimum side length of the minimum dividing area to obtain an edge dividing result;
obtaining the residual defect area according to the edge segmentation result;
And dividing the residual defect area from the edge to the inside by utilizing the dividing minimum area until the dividing of all defect positioning areas is completed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410969790.8A CN118505708B (en) | 2024-07-19 | 2024-07-19 | Product repairing system based on image reconstruction technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410969790.8A CN118505708B (en) | 2024-07-19 | 2024-07-19 | Product repairing system based on image reconstruction technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN118505708A CN118505708A (en) | 2024-08-16 |
| CN118505708B true CN118505708B (en) | 2024-09-10 |
Family
ID=92241931
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410969790.8A Active CN118505708B (en) | 2024-07-19 | 2024-07-19 | Product repairing system based on image reconstruction technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118505708B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108932699A (en) * | 2018-04-24 | 2018-12-04 | 南京信息工程大学 | Three-dimensional matching reconciliation filtering image denoising method based on transform domain |
| CN114882016A (en) * | 2022-06-30 | 2022-08-09 | 中国矿业大学(北京) | Method and system for identifying concrete defect area based on infrared temperature field time sequence information |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112907528B (en) * | 2021-02-09 | 2021-11-09 | 南京航空航天大学 | Point cloud-to-image-based composite material laying wire surface defect detection and identification method |
| CN116953196B (en) * | 2023-09-20 | 2023-12-05 | 上海飞挺管业制造有限公司 | Defect detection and safety state assessment method for steel tee joint |
| CN118134809A (en) * | 2024-04-07 | 2024-06-04 | 成都信息工程大学 | Self-adaptive face restoration method and device based on face attribute information prediction |
-
2024
- 2024-07-19 CN CN202410969790.8A patent/CN118505708B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108932699A (en) * | 2018-04-24 | 2018-12-04 | 南京信息工程大学 | Three-dimensional matching reconciliation filtering image denoising method based on transform domain |
| CN114882016A (en) * | 2022-06-30 | 2022-08-09 | 中国矿业大学(北京) | Method and system for identifying concrete defect area based on infrared temperature field time sequence information |
Also Published As
| Publication number | Publication date |
|---|---|
| CN118505708A (en) | 2024-08-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111833306B (en) | Defect detection method and model training method for defect detection | |
| CN110148130B (en) | Method and device for detecting part defects | |
| CN116188475B (en) | Intelligent control method, system and medium for automatic optical detection of appearance defects | |
| CN111862064A (en) | Silver wire surface flaw identification method based on deep learning | |
| CN109444169B (en) | Bearing defect detection method and system | |
| CN110473201A (en) | A kind of automatic testing method and device of disc surface defect | |
| Wang et al. | Surface defects detection using non-convex total variation regularized RPCA with kernelization | |
| CN114170184A (en) | Product image anomaly detection method and device based on embedded feature vector | |
| CN117392042A (en) | Defect detection method, defect detection apparatus, and storage medium | |
| CN114494780A (en) | Semi-supervised industrial defect detection method and system based on feature comparison | |
| CN118279304B (en) | Abnormal recognition method, device and medium for special-shaped metal piece based on image processing | |
| CN118037091A (en) | Intelligent report working quality detection method and system based on computer vision technology | |
| CN117333492A (en) | Optical film quality detection method and related device based on image processing | |
| CN117798087A (en) | Visual sorting system and terminal for lithium battery based on appearance defect detection | |
| CN114549513A (en) | Part identification method, part identification device, quality inspection method, electronic device, and storage medium | |
| CN110458231B (en) | Ceramic product detection method, device and equipment | |
| CN118505708B (en) | Product repairing system based on image reconstruction technology | |
| CN116664540A (en) | Rubber sealing ring surface defect detection method based on Gaussian line detection | |
| CN116188445A (en) | Product surface defect detection and positioning method and device and terminal equipment | |
| CN115937095A (en) | Printing defect detection method and system integrating image processing algorithm and deep learning | |
| CN117011216A (en) | Defect detection method and device, electronic equipment and storage medium | |
| CN118537339B (en) | Method and system for evaluating surface quality of part based on sander | |
| CN117593301B (en) | Machine vision-based memory bank damage rapid detection method and system | |
| CN117934453B (en) | Method and system for diagnosing defects of backlight foreign matters of mobile phone screen | |
| CN117474916B (en) | Image detection method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |