CN118134219B

CN118134219B - Production early warning method, system, device and nonvolatile storage medium

Info

Publication number: CN118134219B
Application number: CN202410560719.4A
Authority: CN
Inventors: 施大龙; 付茂胜; 李宇昂; 郑熠
Original assignee: Avic International Golden Net Beijing Technology Co ltd
Current assignee: Avic International Golden Net Beijing Technology Co ltd
Priority date: 2024-05-08
Filing date: 2024-05-08
Publication date: 2024-08-06
Anticipated expiration: 2044-05-08
Also published as: CN118134219A

Abstract

The application discloses a production early warning method, a production early warning system, a production early warning device and a nonvolatile storage medium, and relates to the field of industrial production control. Wherein the method comprises the following steps: acquiring a first image of a target production line through first monitoring equipment; collecting a second image of a target product produced by a target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is an unqualified product according to the second image and the brightness of the image corresponding to the second image; and adjusting the production plan under the condition that abnormal production behaviors exist in the production process or unqualified products exist in the target products. The application solves the technical problem that whether the product is qualified cannot be accurately judged because the influence of the brightness of the product image on the verification of whether the product is qualified is not considered in the related technology.

Description

Production early warning method, system, device and nonvolatile storage medium

Technical Field

The application relates to the field of industrial production control, in particular to a production early warning method, a production early warning system, a production early warning device and a nonvolatile storage medium.

Background

In the aerospace industry, a plurality of manufacturers are generally required to use a plurality of production lines to produce different products, and the products produced by the different manufacturers are assembled into final products. Because of the large number of manufacturers and product types involved, and the inability of any one of the products to be delivered in sufficient quantities on a set schedule, there is a potential for negative impact on the lead time of the final product. In order to avoid the problem that the product cannot be delivered on time in the production link as far as possible, whether the product is qualified or not and whether the product can be delivered on time or not need to be timely found in the production process.

In the related art, when checking whether a product is acceptable, it is generally determined whether the product is acceptable by using a manual spot check or by comparing a photographed product image with a previously photographed acceptable product image. However, in the related art, differences in environments of the shot product image and the qualified product image during shooting are not considered, so that differences in brightness of images of different images exist, and accuracy of a judgment result is affected. Particularly in the aerospace industry, a lot of products with precise structures are often involved, and in this case, the influence of different image brightness on the judgment result may be further increased.

In view of the above problems, no effective solution has been proposed at present.

Disclosure of Invention

The embodiment of the application provides a production early warning method, a system, a device and a nonvolatile storage medium, which at least solve the technical problem that whether a product is qualified or not cannot be accurately judged due to the fact that the influence of brightness of a product image on whether the product is qualified or not is not considered in the related technology.

According to an aspect of the embodiment of the present application, there is provided a production pre-warning method, including: receiving production manufacturing data sent by production equipment of a target production line, and acquiring a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines; collecting a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; and under the condition that abnormal production behaviors exist in the production process or unqualified products exist in target products, adjusting production plans of a plurality of production lines.

Optionally, determining whether the target product is a defective product according to the second image and the brightness of the image corresponding to the second image includes: acquiring a reference image of a target product and the image brightness of the reference image; respectively carrying out image segmentation processing on the reference image and the second image to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image, wherein the first segmented image comprises a product image area in the reference image, and the second segmented image comprises a product image area in the second image; determining the image brightness of the first divided image according to the image brightness of the reference image, and determining the image brightness of the second divided image according to the image brightness of the second image; adjusting the image brightness of the first divided image and the second divided image to make the image brightness of the first divided image and the second divided image consistent; and determining whether the target product is a defective product according to the adjusted first segmentation image and second segmentation image.

Optionally, adjusting the image brightness of the first and second segmented images such that the image brightness of the first and second segmented images is consistent comprises: converting the first segmented image into a first HSV image according to the brightness of the first segmented image, and converting the second segmented image into a second HSV image according to the brightness of the second segmented image; determining a first image brightness matrix, a first eigenvalue and a first eigenvector of the first HSV image, and a second image brightness matrix, a second eigenvalue and a second eigenvector of the second HSV image, wherein the multiplication of the first image brightness matrix and the first eigenvector is equal to the multiplication of the first eigenvalue and the first eigenvector, and the multiplication of the second image brightness matrix and the second eigenvector is equal to the multiplication of the second eigenvalue and the second eigenvector; determining a first brightness quality classification statistical result of the first HSV image according to the first feature vector, and determining a second brightness quality classification statistical result of the second HSV image according to the second feature vector, wherein the first brightness quality classification statistical result comprises the duty ratio of each brightness quality grade in the first HSV image and the duty ratio of each brightness quality grade in the second HSV image; determining a target brightness quality level according to the first brightness quality grading statistical result and the second brightness quality grading statistical result, and determining a characteristic value corresponding to the target brightness quality level as a target characteristic value; determining a first brightness adjustment matrix according to the target characteristic value, the first characteristic value and the first image brightness matrix, and determining a second brightness adjustment matrix according to the target characteristic value, the second characteristic value and the second image brightness matrix; and adjusting the brightness of the first divided image according to the first brightness adjusting matrix and the brightness of the second divided image according to the second brightness adjusting matrix so that the brightness of the first divided image is consistent with the brightness of the second divided image.

Optionally, determining whether the target product is a defective product according to the adjusted first and second divided images includes: extracting first image features of the adjusted first segmented image through a residual error module, and extracting second image features of the adjusted first segmented image through an inverse residual error module with expansion coefficients; extracting third image features of the adjusted second divided image through a residual error module, and extracting fourth image features of the second divided image through an inverse residual error module; tiling the first image feature, the second image feature, the third image feature and the fourth image feature to one dimension respectively to obtain a first one-dimensional feature corresponding to the first image feature, a second one-dimensional feature corresponding to the second image feature, a third one-dimensional feature corresponding to the third image feature and a fourth one-dimensional feature corresponding to the fourth image feature; carrying out maximum pooling treatment and average pooling treatment on the first one-dimensional feature, the second one-dimensional feature, the third one-dimensional feature and the fourth one-dimensional feature to obtain a similarity evaluation score between the first segmentation image and the second segmentation image; and determining that the target product is qualified when the similarity evaluation score is larger than the preset score, and determining that the target product is unqualified when the similarity evaluation score is not larger than the preset score.

Optionally, the number of the second images is plural, and the photographing angle of each second image is different; after the target product is qualified, the method further comprises the following steps: under the condition that the production process of the target production line is abnormal according to the production and manufacturing data or the first image, a product model to be judged of the target product is obtained by splicing the second segmentation images corresponding to each second image; determining the similarity between a product model to be judged and a preset product model of a target product; and determining that the target product is qualified under the condition that the similarity is larger than a preset similarity threshold value, and determining that the target product is unqualified under the condition that the similarity is not larger than the preset similarity threshold value.

Optionally, acquiring the reference image of the target product includes: determining the relative position relation between the second monitoring equipment and the target product when the second image is shot; and setting a virtual camera according to the relative position relation, and shooting a preset product model through the virtual camera to obtain a reference image.

Optionally, performing image segmentation processing on the reference image and the second image respectively, to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image includes: determining target pixels and background pixels in an image to be processed, wherein the image to be processed is a reference image or a second image, the target pixels are pixels in a product image area in the image to be processed, and the background pixels are pixels in a background image area in the image to be processed; clustering the target pixel and the background pixel respectively, and determining Gaussian model parameters according to a clustering result, wherein the Gaussian model parameters comprise at least one of the following: the number of mixing coefficients, mean vectors, covariance matrices and Gaussian distributions; determining a Gaussian model corresponding to the image to be processed according to Gaussian model parameters, and determining a segmented image of the image to be processed according to energy terms of the Gaussian model, wherein the segmented image of the image to be processed is a first segmented image when the image to be processed is a reference image, and the segmented image of the image to be processed is a second segmented image when the image to be processed is a second image.

Optionally, adjusting the production schedule of the plurality of production lines includes: determining the priority of products as a target layer, processing constraint conditions, time constraint conditions and the importance degree of each product as a standard layer, wherein the processing difficulty and the number of procedures of each product under the processing constraint conditions, the promised delivery period and the processing time of the target products under the time constraint conditions, and the number, the complexity and the value of the products under the constraint of the importance degree of the products are used as sub-standard layers, wherein each product comprises the target products, and each product is produced by a plurality of production lines; constructing an evaluation index system scheme layer of the target product according to the target layer, the criterion layer and the sub-criterion layer; constructing a judgment matrix, and determining the production priority of each product according to the judgment matrix, the target layer, the criterion layer, the sub-criterion layer and the evaluation index system scheme layer; and adjusting the production plans of the multiple production lines according to the production priority of each product.

Optionally, the production pre-warning method further comprises: determining a plurality of groups of first images, second images and production manufacturing data corresponding to the unqualified products; extracting fault characteristics from a plurality of groups of first images, second images and production and manufacturing data, wherein the fault characteristics are characteristics affecting the quality of products produced by a production line; training a preset fault discrimination model according to the fault characteristics to obtain a target fault discrimination model; and determining whether the target product is qualified or not according to the target fault discrimination model.

According to another aspect of the embodiment of the present application, there is also provided a production pre-warning system, including: the system comprises a data management background, a first monitoring device and a second monitoring device, wherein the data management background is used for receiving production and manufacturing data sent by production devices of a target production line and collecting a first image of the target production line through the first monitoring device, and the target production line is any one of a plurality of production lines; collecting a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; and under the condition that abnormal production behaviors exist in the production process or unqualified products exist in target products, adjusting production plans of a plurality of production lines.

According to another aspect of the embodiment of the present application, there is also provided a production pre-warning device, including: the first processing module is used for receiving production manufacturing data sent by production equipment of a target production line and collecting a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines; the second processing module is used for acquiring a second image of the target product produced by the target production line through second monitoring equipment and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the shot target product is fixed; the third processing module is configured to determine whether a production process of the target production line is abnormal according to the production manufacturing data or the first image, and determine whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, where determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image includes: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; and the fourth processing module is used for adjusting the production plans of the multiple production lines under the condition that abnormal production behaviors exist in the production process or unqualified products exist in the target products.

According to another aspect of the embodiment of the present application, there is further provided a nonvolatile storage medium, in which a program is stored, wherein when the program runs, the apparatus in which the nonvolatile storage medium is controlled to execute the production pre-warning method.

According to another aspect of the embodiment of the present application, there is also provided an electronic device, including: the system comprises a memory and a processor, wherein the processor is used for running a program stored in the memory, and the program executes a production early warning method when running.

According to another aspect of the embodiments of the present application, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements a production pre-warning method.

In the embodiment of the application, production manufacturing data sent by production equipment of a target production line is received, and a first image of the target production line is acquired through first monitoring equipment, wherein the target production line is any one of a plurality of production lines; collecting a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; under the condition that abnormal production behaviors exist in the production process or unqualified products exist in target products, the production plans of a plurality of production lines are adjusted, brightness corresponding to each pixel point in the second image is adjusted to be preset brightness through adjusting brightness of the second image, the purpose of eliminating influences of brightness of the image on the verification process is achieved, and therefore the technical effect that whether products are qualified or not is affected due to the fact that brightness of different product images are inconsistent due to different environments when the product images are shot is achieved, and the technical problem that whether the products are qualified or not cannot be accurately judged due to the fact that whether the influence of brightness of the product images on the verification products is qualified or not is solved in the related technology.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:

fig. 1 is a schematic structural view of a computer terminal (mobile terminal) according to an embodiment of the present application;

FIG. 2 is a flow chart of a method for early warning of production according to an embodiment of the present application;

FIG. 3 is a schematic diagram of a production pre-warning system according to an embodiment of the present application;

FIG. 4 is a schematic structural view of a fixing device for monitoring equipment according to an embodiment of the present application;

FIG. 5 is a schematic diagram of another production pre-warning system according to an embodiment of the present application;

FIG. 6 is a schematic diagram of data transmission in a production pre-warning system according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a production early warning device according to an embodiment of the present application.

Detailed Description

In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In order to better understand the embodiments of the present application, technical terms related to the embodiments of the present application are explained as follows:

Brightness: brightness refers to brightness of an image or gray level of an image, and specifically includes the following parts:

① Brightness: refers to the light intensity information of each pixel point in the image.

② Shading: dark area information generated due to the object shielding the light source, high light: bright area information generated by direct illumination of the object by the light source.

At present, aviation equipment products are complex and various, matched levels are multiple, manufacturing processes are complex, the related materials are various, the quantity is large, and effective monitoring and guaranteeing of the production process are important manifestations of influencing and determining basic capability of aviation equipment product manufacturing industry. The high on-time supply of each intermediate product involved in the production process has important guarantee for improving the on-time delivery rate of the final product, improving the market competitiveness and improving the material safety. Whether the intermediate product can meet the complete set of requirements and further meet the production, manufacture and processing of the product is the basic guarantee of the on-time delivery of the final product. Therefore, the production and manufacturing processes of aviation equipment products are effectively monitored, the production and manufacturing complete set requirements of the products are met, and the production quality of the products is guaranteed.

In the prior art, when checking whether a product is qualified, manual sampling inspection is generally adopted or whether the product is qualified is determined by comparing a shot product image with a pre-shot qualified product image. However, in the related art, differences in environments of the shot product image and the qualified product image during shooting are not considered, so that differences in brightness of images of different images exist, and accuracy of a judgment result is affected. Particularly in the aerospace industry, a lot of products with precise structures are often involved, and in this case, the influence of different image brightness on the judgment result may be further increased.

In order to solve the above problems, related solutions are provided in the embodiments of the present application, and are described in detail below.

According to an embodiment of the present application, there is provided a method embodiment of a method of producing an early warning, it being noted that the steps shown in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that herein.

The method embodiments provided by the embodiments of the present application may be performed in a mobile terminal, a computer terminal, or similar computing device. Fig. 1 shows a block diagram of a hardware architecture of a computer terminal (or mobile device) for implementing a production pre-warning method. As shown in fig. 1, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b, … …,102 n) processors 102 (the processors 102 may include, but are not limited to, a microprocessor MCU, a programmable logic device FPGA, etc. processing means), a memory 104 for storing data, and a transmission means 106 for communication functions. In addition, the method may further include: a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS), a network interface, a power supply, and/or a camera. It will be appreciated by those of ordinary skill in the art that the configuration shown in fig. 1 is merely illustrative and is not intended to limit the configuration of the electronic device described above. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors 102 and/or other data processing circuits described above may be referred to generally herein as "data processing circuits. The data processing circuit may be embodied in whole or in part in software, hardware, firmware, or any other combination. Furthermore, the data processing circuitry may be a single stand-alone processing module, or incorporated, in whole or in part, into any of the other elements in the computer terminal 10 (or mobile device). As referred to in embodiments of the application, the data processing circuit acts as a processor control (e.g., selection of the path of the variable resistor termination connected to the interface).

The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the production pre-warning method in the embodiment of the present application, and the processor 102 executes the software programs and modules stored in the memory 104, thereby executing various functional applications and data processing, that is, implementing the production pre-warning method described above. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission means 106 is arranged to receive or transmit data via a network. The specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC) that can connect to other network devices through a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module for communicating with the internet wirelessly.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

In the above operating environment, the embodiment of the present application provides a production early warning method, as shown in fig. 2, which includes the following steps:

Step S202, receiving production manufacturing data sent by production equipment of a target production line, and acquiring a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines;

In the scheme provided in step S202, the device data or the image data may be transmitted by using a wired communication or a wireless communication, which is not limited herein.

Step S204, acquiring a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed;

In the technical scheme provided in step S204, the relative position between the second monitoring device and the target product can be fixed by controlling the position and the orientation of the second monitoring device through the fixing device and enabling the second camera to shoot the target product reaching the designated area.

As an optional implementation manner, the second monitoring device may be calibrated by a zhang calibration method, so as to complete calibration of parameters in the camera. The method can extract characteristic points of the target object based on a fusion algorithm combining a SIFT algorithm and an ORB algorithm, match the characteristic points among images by using a similarity measure, calibrate parameters outside a camera by using a random sampling consistency algorithm (RANSAC), and take the best result of a model with the largest support set/the RANSAC algorithm.

Step S206, determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is an unqualified product according to the second image and the brightness of the image corresponding to the second image, wherein determining whether the target product is an unqualified product according to the second image and the brightness of the image corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image;

As an alternative embodiment, the determination of whether the production manufacturing process is abnormal or not may be implemented according to the production manufacturing data or the first image data by:

the method comprises the steps that firstly, pre-stored reference manufacturing data are obtained through a data management background, whether production manufacturing data are in a range specified by the reference manufacturing data or not is judged, if yes, the production process is judged to be normal, and if not, the production manufacturing process is judged to be abnormal;

and secondly, judging whether equipment which stops running exists in production equipment in the production process according to the first image data by the data management background, if so, judging that the production process is abnormal, and otherwise, judging that the production process is normal.

The first image collected in the embodiment of the application is used for monitoring the production and manufacturing process, and judging whether the production and manufacturing equipment is out of service, whether the equipment is out of service (whether the appearance is deformed and broken and the functions are normal), on-duty identification (whether working procedure personnel are on duty and the number of on duty), material shortage (whether the material tray placement position is correct and the materials are sufficient), beat abnormality (whether the processing time of different working procedures is abnormal) and other problems according to the first image, so as to judge whether the production and manufacturing process is abnormal or not, thereby determining whether the production and manufacturing process is abnormal.

In the technical solution provided in step S206, determining whether the target product is a defective product according to the second image and the brightness of the image corresponding to the second image includes: acquiring a reference image of a target product and the image brightness of the reference image; respectively carrying out image segmentation processing on the reference image and the second image to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image, wherein the first segmented image comprises a product image area in the reference image, and the second segmented image comprises a product image area in the second image; determining the image brightness of the first divided image according to the image brightness of the reference image, and determining the image brightness of the second divided image according to the image brightness of the second image; adjusting the image brightness of the first divided image and the second divided image to make the image brightness of the first divided image and the second divided image consistent; and determining whether the target product is a defective product according to the adjusted first segmentation image and second segmentation image.

As an alternative embodiment, adjusting the image brightness of the first and second divided images such that the image brightness of the first and second divided images is identical includes: converting the first segmented image into a first HSV image according to the brightness of the first segmented image, and converting the second segmented image into a second HSV image according to the brightness of the second segmented image; determining a first image brightness matrix of the first HSV image, a first eigenvalue and a first eigenvector, and a second image brightness matrix of the second HSV image, a second eigenvalue and a second eigenvector, wherein the multiplication of the first image brightness matrix and the first eigenvector is equal to the multiplication of the first eigenvalue and the first eigenvector, and the multiplication of the second image brightness matrix and the second eigenvector is equal to the multiplication of the second eigenvalue and the second eigenvector; determining a first brightness quality classification statistical result of the first HSV image according to the first feature vector, and determining a second brightness quality classification statistical result of the second HSV image according to the second feature vector, wherein the first brightness quality classification statistical result comprises the duty ratio of each brightness quality grade in the first HSV image and the duty ratio of each brightness quality grade in the second HSV image; determining a target brightness quality level according to the first brightness quality grading statistical result and the second brightness quality grading statistical result, and determining a characteristic value corresponding to the target brightness quality level as a target characteristic value; determining a first brightness adjustment matrix according to the target characteristic value, the first characteristic value and the first image brightness matrix, and determining a second brightness adjustment matrix according to the target characteristic value, the second characteristic value and the second image brightness matrix; and adjusting the brightness of the first divided image according to the first brightness adjusting matrix and the brightness of the second divided image according to the second brightness adjusting matrix so that the brightness of the first divided image is consistent with the brightness of the second divided image.

The duty ratio of the brightness quality level refers to the ratio of the number of pixels with brightness of a certain brightness level corresponding to the image to the total number of pixels in the image. When determining the target brightness quality level according to the first brightness quality classification statistical result and the second brightness quality classification statistical result, the target brightness quality level may be determined according to the principle that the total adjustment amount is minimum after determining the brightness distribution conditions in the first divided image and the second divided image according to the first brightness quality classification statistical result and the second brightness quality classification statistical result. Or determining the brightness quality level with the largest sum of the corresponding proportions in the first divided image and the second divided image as the target brightness quality level.

By adjusting the brightness of the first divided image and the second divided image to be uniform brightness, that is, normalizing the brightness of the first divided image and the second divided image, the influence of illumination can be eliminated when comparing the similarity of the first divided image and the second divided image, and the stability can be enhanced, and the structure and texture information in the images can be more highlighted. In particular, different shooting environments may cause significant differences in brightness of images, and if brightness is not adjusted, image similarity calculation may be inaccurate due to differences in illumination rather than the product itself. Through brightness normalization, misjudgment caused by uneven light intensity can be reduced, and comparison is focused on the structure and texture consistency of the product.

In addition, when the brightness difference is eliminated, the gray distribution of the image reflects the detail characteristics of the product surface, such as texture, shape, contour and the like, which is important for similarity evaluation based on the characteristics. That is, by eliminating the difference of brightness between the two images, the influence of the characteristics such as texture, shape, outline and the like of the product in the images on the similarity is increased, so that the size of the similarity of the images can show the probability of whether the product is qualified or not.

In some embodiments of the present application, determining whether the target product is a defective product based on the adjusted first and second segmented images includes: extracting first image features of the adjusted first segmented image through a residual error module, and extracting second image features of the adjusted first segmented image through an inverse residual error module with expansion coefficients; extracting third image features of the adjusted second divided image through a residual error module, and extracting fourth image features of the second divided image through an inverse residual error module; tiling the first image feature, the second image feature, the third image feature and the fourth image feature to one dimension respectively to obtain a first one-dimensional feature corresponding to the first image feature, a second one-dimensional feature corresponding to the second image feature, a third one-dimensional feature corresponding to the third image feature and a fourth one-dimensional feature corresponding to the fourth image feature; carrying out maximum pooling treatment and average pooling treatment on the first one-dimensional feature, the second one-dimensional feature, the third one-dimensional feature and the fourth one-dimensional feature to obtain a similarity evaluation score between the first segmentation image and the second segmentation image; and determining that the target product is qualified when the similarity evaluation score is larger than the preset score, and determining that the target product is unqualified when the similarity evaluation score is not larger than the preset score.

As an alternative embodiment, the step of adjusting the brightness of the first segmented image and the second segmented image and determining the adjusted image similarity comprises:

the method comprises the steps that a first segmentation image and a second segmentation image are used as two inputs of a preset neural network;

The second step, adjust the brightness of the first divided image and the second divided image, includes the following steps:

(1) Converting the first segmentation image and the second segmentation image into HSV images, and calculating an image brightness matrix and a characteristic value;

(2) Calculating the characteristic vectors of the first divided image and the second divided image respectively, and determining brightness quality grading distribution results of the first divided image and the second divided image based on the characteristic vectors;

Specifically, the above-described brightness quality gradation distribution result includes a ratio between a pixel corresponding to each brightness quality gradation in the first divided image and the second divided image and a total pixel. The brightness matrix, the eigenvalues and the eigenvectors of the image satisfy the following relation:

A*v=λ*v

In the above formula, a represents a brightness matrix, v represents a feature vector, and λ represents a feature value. Each element in the brightness matrix is brightness of each pixel point in the HSV image, and the position of each pixel point in the image is the same as the position of the corresponding element in the matrix. That is, the row and column numbers of a pixel in the image are identical to the row and column numbers of the element in the matrix corresponding to the pixel. The eigenvalues and eigenvectors of the image, i.e., the eigenvalues and eigenvectors of the brightness matrix.

The eigenvalues represent the scaling factors of the matrix transformation. In image processing, the feature values may reflect the significance of the image brightness variation. The magnitude of the eigenvalues indicates the importance of the corresponding eigenvectors in the transformation. A larger eigenvalue means that the brightness of the image varies more significantly in the direction of the eigenvector. The eigenvectors represent the direction of the matrix transformation. In image processing, feature vectors may reveal the dominant direction or pattern of image brightness variation. The feature vectors define the invariant directions in which the brightness of the image changes, i.e. in these directions the brightness of the image changes.

(3) And respectively differencing the target characteristic value with the characteristic values of the first divided image and the second divided image according to the target characteristic value, and superposing brightness matrixes corresponding to the difference values to obtain an adjusted matrix, so as to realize the consistency of the brightness of the images of the first divided image and the second divided image.

When the target characteristic value is determined, according to the principle that the total adjustment amplitude is minimum, determining which brightness quality grade the first divided image and the second divided image are uniformly adjusted to according to brightness quality grade distribution results of the first divided image and the second divided image, and determining the corresponding target characteristic value according to the finally determined brightness quality grade.

Thirdly, carrying out convolution and batch normalization on the first divided image and the second divided image with consistent brightness;

Extracting general feature information of the first divided image and the second divided image respectively by using residual blocks, and extracting important features of the first divided image and the second divided image respectively by using inverse residual blocks with expansion coefficients;

Fifthly, tiling the high-dimensional characteristic information obtained in the fourth step on one dimension, and carrying out maximum pooling and average pooling on the tiled characteristics;

And step six, outputting a similarity score of 0-1, comparing the similarity score with a set threshold value, judging that the similarity score is similar when the similarity score is larger than the threshold value, and judging that the similarity score is dissimilar when the similarity score is smaller than the threshold value.

Specifically, the embodiment of the application also provides a method for measuring the similarity of images based on the twin neural network. When the similarity between images is determined, an end-to-end twin neural network model based on the network weight extracted by the shared lightweight expansion attention main features can be constructed, the original convolution is replaced by hole expansion convolution for part of residual blocks in the lightweight expansion attention expansion network, similarity of two adjacent vertexes is estimated by adopting a similarity calculation rule, larger feature contrast is obtained on the premise of reducing calculation amount, an IAM (Improved Attention Module, improved attention mechanism) method is utilized for carrying out average pooling and maximum pooling on the input feature images, important areas in the images are highlighted, the similarity of the images is further obtained, the judgment result of the images is judged to be the same or different by setting a threshold value, and therefore whether a target product is faulty or not is determined, and early warning is further realized.

In some embodiments of the present application, after normalizing the brightness of the first and second segmented images, the similarity of the images may also be measured based on a perceptual hash algorithm. Specifically, whether two pictures are similar or not can be measured by acquiring the hash value of the picture and comparing the Hamming distance of the hash values of the two pictures, and the smaller the Hamming distance of the hash number of the two pictures is, the more similar the two pictures are. The method comprises the following steps:

First, the picture is scaled, and the picture size is scaled uniformly to a standard size, such as 32×32. Thus, 1024 pixel points are obtained in total; secondly, converting the gray level image, unifying a next input standard, and converting the non-single-channel images into single-channel gray level images; then, calculating DCT, and calculating the corresponding 32X 32 data matrix after discrete cosine transformation of the 32X 32 data matrix; then, reducing DCT, getting the upper left corner 8 x 8 subarea of the 32 x 32 data matrix; then, calculating an average value, namely obtaining an integer matrix G of 8 x 8 through the last step, calculating the average value of all elements in the matrix, and assuming the value of the average value to be a; finally, the fingerprint is calculated and phash = ", which is the input picture, is initialized. Traversing each pixel of the matrix G from left to right, row to row, if the i-th row j column element G (i, j) > = a, then phash + = "1"; if the i-th row and j-th column element G (i, j) < a, phash + = "0", after phash values of the pictures are obtained, a hamming distance of phash values of the two pictures is compared, and a group of pictures with hamming distance less than 10 can be generally considered as similar pictures.

As an alternative embodiment, the number of the second images is plural, and the photographing angle of each second image is different; after the target product is qualified, the method further comprises the following steps: under the condition that the production process of the target production line is abnormal according to the production and manufacturing data or the first image, a product model to be judged of the target product is obtained by splicing the second segmentation images corresponding to each second image; determining the similarity between a product model to be judged and a preset product model of a target product; and determining that the target product is qualified under the condition that the similarity is larger than a preset similarity threshold value, and determining that the target product is unqualified under the condition that the similarity is not larger than the preset similarity threshold value.

Specifically, when a product model is constructed according to a plurality of second images obtained through shooting, as an optional implementation manner, a Grab cut algorithm can be adopted to segment a monitoring image collected at a limited visual angle, the algorithm can obtain a good segmentation effect by utilizing texture information and boundary information of the image and combining a small amount of user interaction, then, a findContours function in OpenCV is utilized to extract a contour line, finally, a full-dimension entity model of the product is constructed by utilizing a Delaunay triangulation method, and whether an abnormality occurs or not is found by comparing the constructed entity model with a preset design model in a system.

In some embodiments of the present application, acquiring a reference image of a target product includes: determining the relative position relation between the second monitoring equipment and the target product when the second image is shot; and setting a virtual camera according to the relative position relation, and shooting a preset product model through the virtual camera to obtain a reference image.

As an optional implementation manner, performing image segmentation processing on the reference image and the second image respectively to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image includes: determining target pixels and background pixels in an image to be processed, wherein the image to be processed is a reference image or a second image, the target pixels are pixels in a product image area in the image to be processed, and the background pixels are pixels in a background image area in the image to be processed; clustering the target pixel and the background pixel respectively, and determining Gaussian model parameters according to a clustering result, wherein the Gaussian model parameters comprise at least one of the following: the number of mixing coefficients, mean vectors, covariance matrices and Gaussian distributions; determining a Gaussian model corresponding to the image to be processed according to Gaussian model parameters, and determining a segmented image of the image to be processed according to energy terms of the Gaussian model, wherein the segmented image of the image to be processed is a first segmented image when the image to be processed is a reference image, and the segmented image of the image to be processed is a second segmented image when the image to be processed is a second image.

As an alternative embodiment, the specific process of image segmentation of the image to be processed includes the following steps:

The method comprises the steps of firstly, determining background pixels belonging to a background area and target pixels in a product image area in an image to be processed, setting a label 0 for the background pixels and setting a label 1 for the target pixels;

Secondly, clustering background pixels and target pixels based on a K-means clustering algorithm, and calculating model parameters of a Gaussian model corresponding to an image to be processed according to values on RGB three channels of each pixel in a clustering result, wherein the model parameters comprise mixing coefficients, mean vectors, covariance matrixes and Gaussian distribution numbers of the Gaussian model;

Thirdly, determining a Gaussian model according to model parameters, distributing Gaussian components of the Gaussian model to each pixel, and optimizing the Gaussian model parameters according to image information;

Specifically, each gaussian component is defined by three main parameters: mean, covariance, and mixing coefficients. These parameters determine the shape, location and weight of each gaussian component. The correspondence of gaussian components to pixels is determined during model training. In image processing, the value of each pixel can be considered as an observed value, while GMM is used to model the distribution of these observed values. During training, the model parameters are optimized using a Expectation Maximization (EM) algorithm to better fit the data. The EM algorithm updates the parameters of each gaussian component through an iterative process while assigning a gaussian component to each pixel, i.e., determining from which gaussian distribution each pixel is most likely to be generated. The assignment of a gaussian component to each pixel is accomplished by calculating the posterior probability that each pixel belongs to each gaussian component.

Step four, building an ST image (Segmentation Map) through Gibbs energy items of a Gaussian model, and processing the ST image by adopting a maximum flow minimum Segmentation algorithm;

and fifthly, determining whether the objective function is converged, jumping to the third step if the objective function is determined not to be converged, and outputting a segmentation result if the objective function is determined to be converged. The objective function may be used to measure the quality of the segmented result, and when the objective function converges, it may be considered that the best segmented result has been obtained at this time, without continuing the iteration.

Step S208, when it is determined that abnormal production behaviors exist in the production process or unqualified products exist in the target products, the production plans of the multiple production lines are adjusted.

In the technical solution provided in step S208, the adjusting the production schedule of the multiple production lines includes: determining the priority of products as a target layer, processing constraint conditions, time constraint conditions and the importance degree of each product as a standard layer, wherein the processing difficulty and the number of procedures of each product under the processing constraint conditions, the promised delivery period and the processing time of the target products under the time constraint conditions, and the number, the complexity and the value of the products under the constraint of the importance degree of the products are used as sub-standard layers, wherein each product comprises the target products, and each product is produced by a plurality of production lines; constructing an evaluation index system scheme layer of the target product according to the target layer, the criterion layer and the sub-criterion layer; constructing a judgment matrix, and determining the production priority of each product according to the judgment matrix, the target layer, the criterion layer, the sub-criterion layer and the evaluation index system scheme layer; and adjusting the production plans of the multiple production lines according to the production priority of each product.

As an alternative embodiment, the plurality of production lines are flexible, that is to say each production line can be used for producing any of the individual products described above.

Specifically, after determining that a product has a fault or that a production process has an abnormality, the cause of the fault and the repair time can be estimated first. If the estimated repair time exceeds a preset threshold, the production deviation correction can be performed in the following manner, so that the production plans of a plurality of production lines can be adjusted:

firstly, constructing a target layer as a product priority according to a complete set rule based on an AHP method, wherein a criterion layer is processing constraint, time constraint and product importance, a sub-criterion layer is processing difficulty and procedure number under the processing constraint, and a product promised delivery period and processing time under the time constraint and the number, complexity and value of products under the product importance constraint, so as to construct an evaluation index system scheme layer as a result of sequencing products according to the priority;

Secondly, constructing a judgment matrix and carrying out consistency test, thereby determining the weight of the sub-criterion layer index and scoring the product by using a fuzzy judgment method, and further determining the production priority of the product;

And thirdly, rescheduling the processing sequence of the parts by using an NSGA-II algorithm based on the priority sequence of the parts, so as to meet the complete delivery requirement to the greatest extent. Wherein meeting the package delivery requirements to the greatest extent means that the number of end products that can be assembled for each type of product that is ultimately produced is as great as possible.

When the estimated repair time does not exceed the preset threshold, the production plan may be adjusted as follows:

Firstly, constructing an objective function, namely that the maximum working time of a workshop under machine faults is minimum, and constructing a dynamic scheduling problem model under the machine faults according to the minimum maximum working time;

secondly, constructing a two-section coding and decoding method based on procedure ordering and machine selection;

thirdly, a complete rescheduling scheme is provided for the problem of faults of a production line of aviation equipment products based on an ICA algorithm, and then production requirements are met to the greatest extent.

As an optional embodiment, the production pre-warning method further includes: determining a plurality of groups of first images, second images and production manufacturing data corresponding to the unqualified products; extracting fault characteristics from a plurality of groups of first images, second images and production and manufacturing data, wherein the fault characteristics are characteristics affecting the quality of products produced by a production line; training a preset fault discrimination model according to the fault characteristics to obtain a target fault discrimination model; and determining whether the target product is qualified or not according to the target fault discrimination model.

Specifically, in the embodiment of the application, the first image data, the second image data and the production manufacturing data can be subjected to joint analysis, so that the fault judgment standard of the production equipment can be corrected, and the product fault can be found in time. The method comprises the following steps:

First, data preprocessing: cleaning production and manufacturing data, processing missing values and abnormal values, and unifying data formats into a format suitable for a machine learning algorithm;

Secondly, extracting features: extracting features affecting the production quality of the product from time series, quality, production and the like based on the first image data, the second image data and the production manufacturing data;

Thirdly, constructing a model: constructing a Convolutional Neural Network (CNN) model, determining the number of layers of the network, the number of neurons in each layer, an activation function, a loss function and an optimizer, training the neural network by using training set data, and continuously adjusting network parameters to achieve a target result;

Fourth, two-stage self-correcting method: stage-E: the classical supervised training mode is adopted to carry out forward propagation, loss calculation, backward propagation and parameter updating from a complete input sequence to a complete output sequence, and the prediction of each step is carried out based on unbiased real input and output. Stage-S: training gradually and correcting errors. After Stage-E training is completed, starting from a certain initial time step M, developing a model training step K; when the X-th training (M < X is less than or equal to M+K), the output of the 1 st to M-1 st time steps is a true value, the output of the M to X-1 st time steps is a predicted value with errors, and the input sequence is always the true value;

fifthly, updating fault judgment standards: according to the result of the model method, the fault cause can be found, the fault standard is continuously updated, and the deviation is continuously corrected.

The method comprises the steps of acquiring production manufacturing data sent by production equipment of a target production line through a first monitoring device, wherein the target production line is any one of a plurality of production lines; collecting a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; under the condition that abnormal production behaviors exist in the production process or unqualified products exist in target products, the production plans of a plurality of production lines are adjusted, brightness corresponding to each pixel point in the second image is adjusted to be preset brightness through adjusting brightness of the second image, the purpose of eliminating influences of brightness of the image on the verification process is achieved, and therefore the technical effect that whether products are qualified or not is affected due to the fact that brightness of different product images are inconsistent due to different environments when the product images are shot is achieved, and the technical problem that whether the products are qualified or not cannot be accurately judged due to the fact that whether the influence of brightness of the product images on the verification products is qualified or not is solved in the related technology.

In addition, in the production and manufacture, the related intermediate products are numerous and huge in quantity, and the fact that the intermediate products cannot be packaged in time directly influences the start time of each link of product assembly and production and the assembly period of the whole product. The intermediate product complete set state-oriented early warning in the production monitoring method provided by the embodiment of the application can ensure that the complete set processing production of the intermediate product is the target, monitor the production process in real time, know the production progress in time and control the quality, correct the production according to the complete set rule under the condition of stopping production and waiting, reschedule the production and processing sequence of parts so as to meet the complete set requirement, thereby ensuring the production and delivery of the product. The state early warning for production equipment and products aims at ensuring normal operation of the equipment and production quality of the products, equipment working conditions are known in real time through monitoring the equipment state, data support is provided for equipment maintenance planning and production scheduling of enterprises, and the production quality of the products is monitored to ensure that the products reach preset quality standards and requirements in the production and delivery processes, so that customer requirements are met, customer satisfaction is improved, and market competitiveness is enhanced.

According to an embodiment of the present application, there is also provided a production early warning system as shown in fig. 3. As can be seen from fig. 3, the system comprises: the system comprises a data management background 30, a first monitoring device 32 and a second monitoring device 34, wherein the data management background 30 is used for receiving production and manufacturing data sent by production equipment of a target production line, and acquiring a first image of the target production line through the first monitoring device 32, wherein the target production line is any one of a plurality of production lines; acquiring a second image of a target product produced by the target production line through a second monitoring device 34, and determining brightness corresponding to the second image through an image sensor in the second monitoring device 34, wherein the relative position between the second monitoring device 34 and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the brightness of the image corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the brightness of the image corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; and adjusting the production plans of the multiple production lines under the condition that abnormal production behaviors exist in the production process or unqualified products exist in the target products.

As an alternative embodiment, the first monitoring device 32 monitors the production process of the preset production line to obtain first image data, the second monitoring device 34 photographs the target product to obtain second image data, and sends the first image data and the second image data to the data management background 30; the data management background 30 receives the first image data and the second image data, and the production manufacturing data transmitted by the production equipment on the preset production line; the data management background 30 judges whether the production manufacturing process is abnormal according to the production manufacturing data or the first image data, and judges whether the target product is qualified according to the second image data; if the data management background 30 judges that the production and manufacturing process is abnormal or the data management background 30 judges that the target product is not qualified, prompt information is sent to the alarm equipment, and after the alarm equipment receives the prompt information, alarm information corresponding to the prompt information is sent. The data management background 30 may be a terminal device, or an electronic device such as a server, which has a certain computing capability and a certain communication capability, so long as the electronic device can receive the first image data sent by the first monitoring device 32, the production manufacturing data sent by the production device on the preset production line, and the second image data sent by the second monitoring device 34, and determine whether an abnormality occurs in the preset production line.

In some embodiments of the application, the production line may be a flexible production line that can process a variety of products. The first monitoring device 32 and the second monitoring device 34 may each be a video camera or a still camera as long as they can take a picture or take a photograph; the second monitoring device 34 is typically disposed at a product output port or a semi-finished product output port of the production line, and photographs the product or semi-finished product output from the product output port or the semi-finished product output port of the production line to make a judgment as to whether the product or semi-finished product is acceptable; the target product is a product or a semi-finished product; the alarm device may be a functional module integrated in the data management background 30, and send out alarm information by popping up a small screen on the data management background 30, or may be a separate device capable of performing information interaction with the data management background 30, for example, an alarm lamp or an alarm capable of sending out alarm sound, where the alarm device sends out different alarm information according to the received prompt information, so as to prompt a staff to process the corresponding alarm information as soon as possible.

Through increasing alarm device in the system, realized the diversification and the comprehension to the control of production manufacturing process, through alarm device's cooperation, when production manufacturing process appears unusual, can in time discover and solve the problem, realized the management and control to the complete process of production and intermediate product complete set state. The system is based on a network, integrates the technologies of video monitoring, abnormal alarming, production progress tracking and quality information acquisition, realizes comprehensive and real-time monitoring and management of manufacturing information, and is beneficial to real-time monitoring of complete set states of intermediate products.

As an alternative implementation manner, the structure of the fixing device for monitoring equipment provided by the embodiment of the application is shown in fig. 4. As can be seen in fig. 4, the monitoring device fixture includes a product holding tray 40, a bracket 42, and a photographing angle acquisition device. The fixing device is arranged at a product output port or a semi-finished product output port of the production line, the product fixing plate 40 is used for fixing a target product output from the product output end or the semi-finished product output end, the support 42 is used for fixing the second monitoring device 34, the photographing angle acquisition device is used for acquiring an angle between a straight line L where a marking position on the product fixing plate 40 and a fixing position on the support 42 are located and a plane where a lens of the second monitoring device 34 is located, namely, a photographing angle, and the photographing angle is changed by rotating the product fixing plate 40. By arranging the monitoring equipment fixing device, the second monitoring equipment 34 can be ensured to shoot and collect according to a preset shooting angle when collecting the product image.

It should be noted that, in the related art, an MES system (Manufacturing Execution System, factory manufacturing execution system is abbreviated as MES) is generally used for production monitoring and early warning. However, at present, manufacturing enterprises adopting an MES system mostly take common personal computers or industrial control computers as main bodies to serve as field terminals, and the field terminals have the advantages of large power consumption, large volume, single function, low industrial protection level, poor reliability, incapability of well adapting to dynamic changes of environments and incapability of providing stable services for enterprises of aviation equipment manufacturing and supply chains.

Compared with the MES system in the related technology, the production early warning system provided by the embodiment of the application fully utilizes the communication network, realizes real-time processing and transmission of various data, solves the problem of lower calculation power of local equipment, and can realize high-efficiency analysis and processing of the data by means of a cloud server and the like.

The embodiment of the application also provides a production early warning system shown in fig. 5. As can be seen from fig. 5, a plurality of cameras may be included in the system as either the first monitoring device or the second monitoring device. Wherein the cameras include web cameras and conventional cameras. The network camera can send the shot pictures to a monitoring center or other electronic equipment with a browser for browsing and viewing directly according to the TCP/IP protocol. Conventional cameras require the transmission of captured picture video data to a monitoring center or other electronic device via a hard disk recorder or video server. In addition, the production early warning system also comprises a data acquisition and control module arranged in the production equipment and used for acquiring production data of each equipment in the production line.

In some embodiments of the present application, the data transmission manner in the production pre-warning system may be as shown in fig. 6. The video acquisition lens (first monitoring equipment and second monitoring equipment) on the production line and relevant parameters input by operators are sent to a data management background through a PC end MES system on the production line by the Internet, and the production and manufacturing process is monitored through the data management background.

The embodiment of the application provides a production early warning device, and fig. 7 is a schematic structural diagram of the device. As can be seen in fig. 7, the device comprises: a first processing module 70, configured to receive production manufacturing data sent by a production device of a target production line, and collect, by a first monitoring device, a first image of the target production line, where the target production line is any one of a plurality of production lines; a second processing module 72, configured to collect, by using a second monitoring device, a second image of the target product produced by the target production line, and determine, by using an image sensor in the second monitoring device, brightness corresponding to the second image, where a relative position between the second monitoring device and the photographed target product is fixed; a third processing module 74, configured to determine whether the production process of the target production line is abnormal according to the production manufacturing data or the first image, and determine whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, where determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image includes: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; a fourth processing module 76 for adjusting the production schedule of the plurality of production lines in the event that it is determined that there is abnormal production behavior during the production process or that there is a defective product in the target product.

In some embodiments of the present application, the second processing module 72 determining whether the target product is a defective product according to the second image and the brightness of the image corresponding to the second image includes: acquiring a reference image of a target product and the image brightness of the reference image; respectively carrying out image segmentation processing on the reference image and the second image to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image, wherein the first segmented image comprises a product image area in the reference image, and the second segmented image comprises a product image area in the second image; determining the image brightness of the first divided image according to the image brightness of the reference image, and determining the image brightness of the second divided image according to the image brightness of the second image; adjusting the image brightness of the first divided image and the second divided image to make the image brightness of the first divided image and the second divided image consistent; and determining whether the target product is a defective product according to the adjusted first segmentation image and second segmentation image.

In some embodiments of the present application, the second processing module 72 adjusts the image brightness of the first and second segmented images such that the image brightness of the first and second segmented images is consistent comprises: converting the first segmented image into a first HSV image according to the brightness of the first segmented image, and converting the second segmented image into a second HSV image according to the brightness of the second segmented image; determining a first image brightness matrix, a first eigenvalue and a first eigenvector of the first HSV image, and a second image brightness matrix, a second eigenvalue and a second eigenvector of the second HSV image, wherein the multiplication of the first image brightness matrix and the first eigenvector is equal to the multiplication of the first eigenvalue and the first eigenvector, and the multiplication of the second image brightness matrix and the second eigenvector is equal to the multiplication of the second eigenvalue and the second eigenvector; determining a first brightness quality classification statistical result of the first HSV image according to the first feature vector, and determining a second brightness quality classification statistical result of the second HSV image according to the second feature vector, wherein the first brightness quality classification statistical result comprises the duty ratio of each brightness quality grade in the first HSV image and the duty ratio of each brightness quality grade in the second HSV image; determining a target brightness quality level according to the first brightness quality grading statistical result and the second brightness quality grading statistical result, and determining a characteristic value corresponding to the target brightness quality level as a target characteristic value; determining a first brightness adjustment matrix according to the target characteristic value, the first characteristic value and the first image brightness matrix, and determining a second brightness adjustment matrix according to the target characteristic value, the second characteristic value and the second image brightness matrix; and adjusting the brightness of the first divided image according to the first brightness adjusting matrix and the brightness of the second divided image according to the second brightness adjusting matrix so that the brightness of the first divided image is consistent with the brightness of the second divided image.

In some embodiments of the present application, the second processing module 72 determining whether the target product is a defective product according to the adjusted first segmented image and the second segmented image includes: extracting first image features of the adjusted first segmented image through a residual error module, and extracting second image features of the adjusted first segmented image through an inverse residual error module with expansion coefficients; extracting third image features of the adjusted second divided image through a residual error module, and extracting fourth image features of the second divided image through an inverse residual error module; tiling the first image feature, the second image feature, the third image feature and the fourth image feature to one dimension respectively to obtain a first one-dimensional feature corresponding to the first image feature, a second one-dimensional feature corresponding to the second image feature, a third one-dimensional feature corresponding to the third image feature and a fourth one-dimensional feature corresponding to the fourth image feature; carrying out maximum pooling treatment and average pooling treatment on the first one-dimensional feature, the second one-dimensional feature, the third one-dimensional feature and the fourth one-dimensional feature to obtain a similarity evaluation score between the first segmentation image and the second segmentation image; and determining that the target product is qualified when the similarity evaluation score is larger than the preset score, and determining that the target product is unqualified when the similarity evaluation score is not larger than the preset score.

In some embodiments of the present application, the number of the second images is plural, and the photographing angle of each second image is different; after the second processing module 72 determines that the target product is qualified, the production early warning device is further configured to: under the condition that the production process of the target production line is abnormal according to the production and manufacturing data or the first image, a product model to be judged of the target product is obtained by splicing the second segmentation images corresponding to each second image; determining the similarity between a product model to be judged and a preset product model of a target product; and determining that the target product is qualified under the condition that the similarity is larger than a preset similarity threshold value, and determining that the target product is unqualified under the condition that the similarity is not larger than the preset similarity threshold value.

In some embodiments of the present application, the second processing module 72 acquiring the reference image of the target product includes: determining the relative position relation between the second monitoring equipment and the target product when the second image is shot; and setting a virtual camera according to the relative position relation, and shooting a preset product model through the virtual camera to obtain a reference image.

In some embodiments of the present application, the second processing module 72 performs image segmentation processing on the reference image and the second image, to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image, respectively, including: determining target pixels and background pixels in an image to be processed, wherein the image to be processed is a reference image or a second image, the target pixels are pixels in a product image area in the image to be processed, and the background pixels are pixels in a background image area in the image to be processed; clustering the target pixel and the background pixel respectively, and determining Gaussian model parameters according to a clustering result, wherein the Gaussian model parameters comprise at least one of the following: the number of mixing coefficients, mean vectors, covariance matrices and Gaussian distributions; determining a Gaussian model corresponding to the image to be processed according to Gaussian model parameters, and determining a segmented image of the image to be processed according to energy terms of the Gaussian model, wherein the segmented image of the image to be processed is a first segmented image when the image to be processed is a reference image, and the segmented image of the image to be processed is a second segmented image when the image to be processed is a second image.

In some embodiments of the application, the fourth processing module 76 adjusts the production plan for the plurality of production lines includes: determining the priority of products as a target layer, processing constraint conditions, time constraint conditions and the importance degree of each product as a standard layer, wherein the processing difficulty and the number of procedures of each product under the processing constraint conditions, the promised delivery period and the processing time of the target products under the time constraint conditions, and the number, the complexity and the value of the products under the constraint of the importance degree of the products are used as sub-standard layers, wherein each product comprises the target products, and each product is produced by a plurality of production lines; constructing an evaluation index system scheme layer of the target product according to the target layer, the criterion layer and the sub-criterion layer; constructing a judgment matrix, and determining the production priority of each product according to the judgment matrix, the target layer, the criterion layer, the sub-criterion layer and the evaluation index system scheme layer; and adjusting the production plans of the multiple production lines according to the production priority of each product.

In some embodiments of the application, the production pre-warning device is further configured to: determining a plurality of groups of first images, second images and production manufacturing data corresponding to the unqualified products; extracting fault characteristics from a plurality of groups of first images, second images and production and manufacturing data, wherein the fault characteristics are characteristics affecting the quality of products produced by a production line; training a preset fault discrimination model according to the fault characteristics to obtain a target fault discrimination model; and determining whether the target product is qualified or not according to the target fault discrimination model.

Note that each module in the production early warning device may be a program module (for example, a set of program instructions for implementing a specific function), or may be a hardware module, and for the latter, it may be represented by the following form, but is not limited thereto: the expression forms of the modules are all a processor, or the functions of the modules are realized by one processor.

According to an embodiment of the present application, there is also provided a nonvolatile storage medium in which a program is stored, wherein the nonvolatile storage medium is controlled to execute the following production pre-warning method when the program runs: receiving production manufacturing data sent by production equipment of a target production line, and acquiring a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines; collecting a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; and under the condition that abnormal production behaviors exist in the production process or unqualified products exist in target products, adjusting production plans of a plurality of production lines.

According to an embodiment of the present application, there is also provided an electronic device including a memory and a processor, where the processor is configured to execute a program stored in the memory, and the program executes the following production pre-warning method: receiving production manufacturing data sent by production equipment of a target production line, and acquiring a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines; collecting a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; and under the condition that abnormal production behaviors exist in the production process or unqualified products exist in target products, adjusting production plans of a plurality of production lines.

According to an embodiment of the present application, there is also provided a computer program product comprising a computer program which, when executed by a processor, implements a production pre-warning method of: receiving production manufacturing data sent by production equipment of a target production line, and acquiring a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines; collecting a second image of a target product produced by the target production line through second monitoring equipment, and determining brightness corresponding to the second image through an image sensor in the second monitoring equipment, wherein the relative position between the second monitoring equipment and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product according to the adjusted second image; and under the condition that abnormal production behaviors exist in the production process or unqualified products exist in target products, adjusting production plans of a plurality of production lines.

In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.

In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the related art or all or part of the technical solution, in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a usb disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.

The foregoing is merely a preferred embodiment of the present application and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present application, which are intended to be comprehended within the scope of the present application.

Claims

1. A method of producing an early warning, comprising:

Receiving production manufacturing data sent by production equipment of a target production line, and acquiring a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines;

Acquiring a second image of the target product produced by the target production line through a second monitoring device, and determining brightness corresponding to the second image through an image sensor in the second monitoring device, wherein the relative position between the second monitoring device and the photographed target product is fixed;

Determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product or not according to the adjusted second image;

Adjusting production plans of the plurality of production lines under the condition that abnormal production behaviors exist in the production process or unqualified products exist in the target products are determined;

Determining whether the target product is a defective product according to the second image and the brightness of the image corresponding to the second image comprises: acquiring a reference image of the target product and the image brightness of the reference image; respectively carrying out image segmentation processing on the reference image and the second image to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image, wherein the first segmented image comprises a product image area in the reference image, and the second segmented image comprises a product image area in the second image; determining the image brightness of the first divided image according to the image brightness of the reference image, and determining the image brightness of the second divided image according to the image brightness of the second image; adjusting the image brightness of the first divided image and the second divided image so that the image brightness of the first divided image and the image brightness of the second divided image are consistent; determining whether the target product is an unqualified product according to the adjusted first segmentation image and second segmentation image;

Determining whether the target product is a defective product according to the adjusted first segmentation image and second segmentation image comprises: extracting first image features of the adjusted first segmented image through a residual error module, and extracting second image features of the adjusted first segmented image through an inverse residual error module with expansion coefficients; extracting third image features of the adjusted second divided image through the residual error module, and extracting fourth image features of the second divided image through the inverse residual error module; tiling the first image feature, the second image feature, the third image feature and the fourth image feature to one dimension respectively to obtain a first one-dimensional feature corresponding to the first image feature, a second one-dimensional feature corresponding to the second image feature, a third one-dimensional feature corresponding to the third image feature and a fourth one-dimensional feature corresponding to the fourth image feature; performing maximum pooling treatment and average pooling treatment on the first one-dimensional feature, the second one-dimensional feature, the third one-dimensional feature and the fourth one-dimensional feature to obtain a similarity evaluation score between the first segmentation image and the second segmentation image; and determining that the target product is qualified when the similarity evaluation score is larger than a preset score, and determining that the target product is unqualified when the similarity evaluation score is not larger than the preset score.

2. The production pre-warning method of claim 1, wherein adjusting the image brightness of the first and second divided images such that the image brightness of the first and second divided images is consistent comprises:

Converting the first segmented image into a first HSV image according to the brightness of the first segmented image, and converting the second segmented image into a second HSV image according to the brightness of the second segmented image;

Determining a first image brightness matrix, a first eigenvalue and a first eigenvector of the first HSV image, and a second image brightness matrix, a second eigenvalue and a second eigenvector of the second HSV image, wherein the multiplication of the first image brightness matrix and the first eigenvector equals the multiplication of the first eigenvalue and the first eigenvector, and the multiplication of the second image brightness matrix and the second eigenvector equals the multiplication of the second eigenvalue and the second eigenvector;

Determining a first brightness quality classification statistical result of the first HSV image according to the first feature vector, and determining a second brightness quality classification statistical result of the second HSV image according to the second feature vector, wherein the first brightness quality classification statistical result comprises the duty ratio of each brightness quality grade in the first HSV image and the duty ratio of each brightness quality grade in the second HSV image;

Determining a target brightness quality level according to the first brightness quality grading statistical result and the second brightness quality grading statistical result, and determining a characteristic value corresponding to the target brightness quality level as a target characteristic value;

Determining a first brightness adjustment matrix according to the target characteristic value, the first characteristic value and the first image brightness matrix, and determining a second brightness adjustment matrix according to the second characteristic value and the second image brightness matrix;

adjusting the brightness of the first divided image according to the first brightness adjusting matrix, and adjusting the brightness of the second divided image according to the second brightness adjusting matrix, so that the brightness of the first divided image is consistent with the brightness of the second divided image.

3. The production pre-warning method according to claim 1, wherein the number of the second images is plural, and the photographing angle of each of the second images is different; after determining that the target product is qualified, the method further comprises:

Under the condition that the production process of the target production line is abnormal according to the production and manufacturing data or the first image, a product model to be judged of the target product is obtained by splicing the second segmentation images corresponding to each second image;

Determining the similarity between the product model to be judged and a preset product model of the target product;

And determining that the target product is qualified when the similarity is greater than a preset similarity threshold, and determining that the target product is unqualified when the similarity is not greater than the preset similarity threshold.

4. The production pre-warning method of claim 3, wherein obtaining a reference image of the target product comprises:

determining the relative position relation between the second monitoring equipment and the target product when the second image is shot;

setting a virtual camera according to the relative position relation, and shooting the preset product model through the virtual camera to obtain the reference image.

5. The production pre-warning method according to claim 1, wherein performing image segmentation processing on the reference image and the second image, respectively, to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image includes:

Determining target pixels and background pixels in an image to be processed, wherein the image to be processed is the reference image or the second image, the target pixels are pixels in a product image area in the image to be processed, and the background pixels are pixels in a background image area in the image to be processed;

Clustering the target pixel and the background pixel respectively, and determining Gaussian model parameters according to a clustering result, wherein the Gaussian model parameters comprise at least one of the following: the number of mixing coefficients, mean vectors, covariance matrices and Gaussian distributions;

Determining a Gaussian model corresponding to the image to be processed according to the Gaussian model parameters, and determining a segmented image of the image to be processed according to an energy term of the Gaussian model, wherein the segmented image of the image to be processed is a first segmented image when the image to be processed is the reference image, and the segmented image of the image to be processed is a second segmented image when the image to be processed is the second image.

6. The production pre-warning method of claim 1, wherein adjusting the production schedule of the plurality of production lines comprises:

determining a product priority as a target layer, a processing constraint condition, a time constraint condition and the importance degree of each product as a criterion layer based on an analytic hierarchy process, wherein the processing difficulty and the number of procedures of each product under the processing constraint condition, the target product promised delivery period and the processing time under the time constraint condition, and the number of products, the product complexity and the product value under the constraint of the importance degree are used as sub-criterion layers, wherein each product comprises the target product, and each product is produced by a plurality of production lines;

constructing an evaluation index system scheme layer of the target product according to the target layer, the criterion layer and the sub-criterion layer;

Constructing a judgment matrix, and determining the production priority of each product according to the judgment matrix, the target layer, the criterion layer, the sub-criterion layer and the evaluation index system scheme layer;

and adjusting the production plans of the production lines according to the production priority of each product.

7. The production pre-warning method of claim 1, further comprising:

determining a plurality of groups of first images, second images and production manufacturing data corresponding to the unqualified products;

extracting fault features from the plurality of sets of first images, second images and production manufacturing data, wherein the fault features are features affecting the quality of products produced by a production line;

Training a preset fault discrimination model according to the fault characteristics to obtain a target fault discrimination model;

and determining whether the target product is qualified or not according to the target fault judging model.

8. The production early warning system is characterized by comprising a data management background, a first monitoring device and a second monitoring device, wherein,

The data management background is used for receiving production and manufacturing data sent by production equipment of a target production line and collecting a first image of the target production line through first monitoring equipment, wherein the target production line is any one production line of a plurality of production lines; acquiring a second image of the target product produced by the target production line through a second monitoring device, and determining brightness corresponding to the second image through an image sensor in the second monitoring device, wherein the relative position between the second monitoring device and the photographed target product is fixed; determining whether the production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, wherein determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image comprises: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product or not according to the adjusted second image; adjusting production plans of the plurality of production lines under the condition that abnormal production behaviors exist in the production process or unqualified products exist in the target products are determined;

9. A production pre-warning device, comprising:

The first processing module is used for receiving production manufacturing data sent by production equipment of a target production line and collecting a first image of the target production line through first monitoring equipment, wherein the target production line is any one of a plurality of production lines;

The second processing module is used for acquiring a second image of the target product produced by the target production line through a second monitoring device and determining brightness corresponding to the second image through an image sensor in the second monitoring device, wherein the relative position between the second monitoring device and the photographed target product is fixed;

The third processing module is configured to determine whether a production process of the target production line is abnormal according to the production and manufacturing data or the first image, and determine whether the target product is a defective product according to the second image and the image brightness corresponding to the second image, where determining whether the target product is a defective product according to the second image and the image brightness corresponding to the second image includes: adjusting the second image according to the image brightness corresponding to the second image, so that the image brightness of the second image is adjusted to be the preset image brightness, and the brightness corresponding to each pixel point in the second image is the same; determining whether the target product is a defective product or not according to the adjusted second image;

A fourth processing module, configured to adjust production plans of the multiple production lines when it is determined that abnormal production behaviors exist in the production process or that unqualified products exist in the target product;

The third processing module determining whether the target product is a defective product according to the second image and the brightness of the image corresponding to the second image includes: acquiring a reference image of the target product and the image brightness of the reference image; respectively carrying out image segmentation processing on the reference image and the second image to obtain a first segmented image corresponding to the reference image and a second segmented image corresponding to the second image, wherein the first segmented image comprises a product image area in the reference image, and the second segmented image comprises a product image area in the second image; determining the image brightness of the first divided image according to the image brightness of the reference image, and determining the image brightness of the second divided image according to the image brightness of the second image; adjusting the image brightness of the first divided image and the second divided image so that the image brightness of the first divided image and the image brightness of the second divided image are consistent; determining whether the target product is an unqualified product according to the adjusted first segmentation image and second segmentation image;

The third processing module determining whether the target product is a defective product according to the adjusted first segmentation image and second segmentation image includes: extracting first image features of the adjusted first segmented image through a residual error module, and extracting second image features of the adjusted first segmented image through an inverse residual error module with expansion coefficients; extracting third image features of the adjusted second divided image through the residual error module, and extracting fourth image features of the second divided image through the inverse residual error module; tiling the first image feature, the second image feature, the third image feature and the fourth image feature to one dimension respectively to obtain a first one-dimensional feature corresponding to the first image feature, a second one-dimensional feature corresponding to the second image feature, a third one-dimensional feature corresponding to the third image feature and a fourth one-dimensional feature corresponding to the fourth image feature; performing maximum pooling treatment and average pooling treatment on the first one-dimensional feature, the second one-dimensional feature, the third one-dimensional feature and the fourth one-dimensional feature to obtain a similarity evaluation score between the first segmentation image and the second segmentation image; and determining that the target product is qualified when the similarity evaluation score is larger than a preset score, and determining that the target product is unqualified when the similarity evaluation score is not larger than the preset score.

10. A nonvolatile storage medium, wherein a program is stored in the nonvolatile storage medium, and wherein the program, when executed, controls a device in which the nonvolatile storage medium is located to execute the production pre-warning method according to any one of claims 1 to 7.

11. An electronic device, comprising: a memory and a processor for executing a program stored in the memory, wherein the program is executed to perform the production pre-warning method according to any one of claims 1 to 7.

12. A computer program product comprising a computer program which, when executed by a processor, implements the production pre-warning method according to any one of claims 1 to 7.