CN113139957A - Inspection method, apparatus, system and medium in product assembly process - Google Patents
Inspection method, apparatus, system and medium in product assembly process Download PDFInfo
- Publication number
- CN113139957A CN113139957A CN202110521260.3A CN202110521260A CN113139957A CN 113139957 A CN113139957 A CN 113139957A CN 202110521260 A CN202110521260 A CN 202110521260A CN 113139957 A CN113139957 A CN 113139957A
- Authority
- CN
- China
- Prior art keywords
- accessory
- target product
- target
- equipment
- identification code
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 93
- 238000007689 inspection Methods 0.000 title claims abstract description 55
- 230000008569 process Effects 0.000 title abstract description 35
- 230000002159 abnormal effect Effects 0.000 claims description 31
- 238000013507 mapping Methods 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 18
- 238000004519 manufacturing process Methods 0.000 claims description 12
- 238000003860 storage Methods 0.000 claims description 12
- 238000004891 communication Methods 0.000 claims description 10
- 238000009434 installation Methods 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 description 10
- 238000010586 diagram Methods 0.000 description 7
- 238000013461 design Methods 0.000 description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000012552 review Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000005406 washing Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000005485 electric heating Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06K—GRAPHICAL DATA READING; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06K17/00—Methods or arrangements for effecting co-operative working between equipments covered by two or more of main groups G06K1/00 - G06K15/00, e.g. automatic card files incorporating conveying and reading operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method, equipment, a system and a medium for checking in a product assembly process, wherein the method comprises the following steps: receiving a target product identification code sent by scanning equipment; determining a target position of the mobile machine device corresponding to the target product identification code; controlling the mobile machine equipment to drive the shooting equipment to move to a target position; controlling a shooting device to shoot a target product to obtain a target product image, and identifying the accessory state in the target product image by adopting a preset accessory identification model; and storing the identification code of the target product and the corresponding accessory state in an associated manner. The product identification code is information representing the only product, so that the target product can be accurately identified, the shooting equipment is controlled to shoot the target product, after the target product image is obtained, the preset accessory identification model can be adopted to automatically and accurately identify the accessory state in the target product image, the inspection accuracy is improved, and the inspection efficiency is improved.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a method, equipment, a system and a medium for checking in a product assembling process.
Background
With the popularization of industrial production lines, the assembly quality of products is required to be higher and higher by consumers. In the face of products with more accessories, it is necessary to ensure that the products are not subjected to neglected assembly and wrong assembly of the accessories in the assembly process.
In the prior art, in order to ensure the quality requirement of product assembly, the product model is generally manually tested, and the state of the accessory is tested after the product of the model is assembled. This results in inefficient testing and less accurate testing due to the susceptibility of manual testing to test errors.
Disclosure of Invention
The invention provides a method, equipment, a system and a medium for checking in a product assembling process, which solve the technical problems that the product model is manually checked, and the checking efficiency and the accuracy are lower when the state of an accessory is checked after the product of the model is assembled.
A first aspect of the present invention provides a method for checking during product assembly, the method being applied to an electronic device, the method including:
receiving a target product identification code sent by scanning equipment;
determining a target position of the mobile machine device corresponding to the target product identification code;
controlling the mobile machine equipment to drive the shooting equipment to move to the target position;
controlling the shooting equipment to shoot the target product so as to obtain a target product image, and identifying the accessory state in the target product image by adopting a preset accessory identification model;
and storing the target product identification code and the corresponding accessory state in an associated manner.
In one possible design, the determining the target location of the mobile machine device corresponding to the target product identification code includes:
acquiring a mapping relation between a pre-established preset product identification code and a target position of mobile machine equipment;
and determining the target position of the mobile machine equipment corresponding to the target product identification code in the mapping relation.
In one possible design, the camera is located on top of the target product;
the method for recognizing the accessory state in the target product image by adopting the preset accessory recognition model comprises the following steps:
inputting the target product image into the accessory identification model;
identifying the appearance and the quantity of target accessories in the target product image through the accessory identification model, and determining the accessory state according to the appearance and the quantity of the target accessories;
outputting the accessory status through the accessory recognition model.
In one possible design, the determining the fitting status according to the target fitting appearance and quantity comprises:
determining the model of a target accessory according to the appearance of the target accessory, and determining whether the accessory state is a wrong-installation abnormal state or not according to the model of the target accessory;
and determining whether the accessory state is a neglected loading abnormal state or not according to the target accessory number.
In a possible design, after the recognizing the accessory status in the target product image by using the preset accessory recognition model, the method further includes:
generating a corresponding inspection result according to the state of the accessory;
acquiring relevant time information of inspection of a target product;
the associating and storing the target product identification code and the corresponding accessory state comprises the following steps:
and performing associated storage on the target product identification code, the corresponding accessory state, the inspection result and the inspection related time information.
In a possible design, after generating the corresponding inspection result according to the fitting status, the method further includes:
if the inspection result is determined to be an abnormal result, sending abnormal prompt information to carry out accessory state recheck according to the abnormal prompt information;
after the target product identification code, the corresponding accessory state, the inspection result and the inspection related time information are stored in an associated manner, the method further comprises the following steps:
and sending the data of the associated storage to a production management system (MES).
A second aspect of the present invention provides an electronic apparatus comprising: a memory, a processor, and a transceiver;
the processor, the memory, and the transceiver circuitry are interconnected;
the memory for storing a computer program; the transceiver is used for receiving and transmitting data with the scanning equipment, the mobile machine equipment, the shooting equipment and the production management system MES;
the processor is configured to be capable of performing the method of any one of the first aspect.
A third aspect of the present invention provides an inspection system in a product assembly process, comprising: a scanning device, a mobile machine device, a photographing device, a MES, and an electronic device as described in the second aspect;
the electronic equipment is in communication connection with the scanning equipment, the mobile machine equipment, the shooting equipment and the MES respectively;
the shooting equipment is fixed on the mobile machine equipment.
A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon a computer program;
the computer program, when executed by a processor, implements the method of any of the first aspects.
A fifth aspect of the invention provides a computer program product comprising a computer program which, when executed by a processor, performs the method according to any one of the first aspects.
The invention provides a method, a device, a system and a medium for checking in the product assembly process, which receive a target product identification code sent by a scanning device; determining a target position of the mobile machine device corresponding to the target product identification code; controlling the mobile machine equipment to drive the shooting equipment to move to the target position; controlling the shooting equipment to shoot the target product so as to obtain a target product image, and identifying the accessory state in the target product image by adopting a preset accessory identification model; and storing the target product identification code and the corresponding accessory state in an associated manner. The product identification code is information representing the only product, so that the target product can be accurately identified, the shooting equipment is controlled to shoot the target product, after the target product image is obtained, the preset accessory identification model can be adopted to automatically and accurately identify the accessory state in the target product image, the inspection accuracy is improved, and the inspection efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a block diagram of an inspection system during assembly of a product according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method of inspection during product assembly provided by an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a vision subsystem and a target product in a product assembly process according to an embodiment of the present invention;
FIG. 4 is a flow chart of a method of inspection during assembly of a product according to another embodiment of the present invention;
FIG. 5 is a diagram of a client-side operation interface of a checking method in a product assembly process according to another embodiment of the present invention;
FIG. 6 is a signaling interaction flow diagram of a checking method in a product assembling process according to still another embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
For a clear understanding of the technical solutions of the present application, a detailed description of the prior art solutions is first provided.
In the prior art, in order to ensure the quality requirement of product assembly, the product model is generally manually tested, and the state of the accessory is tested after the product of the model is assembled. Specifically, the product model is determined by manually and directly observing the appearance characteristics of the product. And then, after the assembly of the product of the model is finished, the appearance characteristics of the accessory are observed in a sensory mode, the state of the accessory is determined, and finally, the test result is recorded in a hand-copy mode. This results in inefficient testing and less accurate testing due to the susceptibility of manual testing to test errors.
Aiming at the technical problems, the inventor creatively discovers that the computer vision technology can be applied to the inspection link in the product assembly process. Specifically, because the models of the products are different and the sizes and the outlines of the products are also different, the mapping relation between the preset product identification code and the target position of the mobile machine equipment can be also pre-established, and when the target product flows to the inspection position through the conveying line, the target product identification code can be scanned through the scanning equipment and sent to the electronic equipment. The electronic equipment determines a target position of the mobile machine equipment corresponding to the target product identification code, controls the mobile machine equipment to drive the shooting equipment to move to the target position, controls the shooting equipment to shoot the target product to obtain a target product image, recognizes the accessory state in the target product image by adopting a preset accessory recognition model, and stores the recognized accessory state and the target product identification code in an associated manner. The product identification code is information representing the only product, so that the target product can be accurately identified, the shooting equipment is controlled to shoot the target product, after the target product image is obtained, the preset accessory identification model can be adopted to automatically and accurately identify the accessory state in the target product image, the inspection accuracy is improved, and the inspection efficiency is improved.
Therefore, the inventor proposes a technical scheme of the embodiment of the invention based on the above creative discovery. The system architecture and application scenario of the inspection method in the product assembly process provided by the embodiment of the present invention are described below.
Fig. 1 is an architecture diagram of an inspection system in a product assembly process according to an embodiment of the present invention, and as shown in fig. 1, a system architecture of an inspection method in a product assembly process according to the embodiment includes: the system comprises a scanning device 1, a mobile machine device 2, a shooting device 3, a production management system (MES) 4 and an electronic device 5. The electronic device 5 is in communication connection with the scanning device 1, the mobile device 2, the photographing device 3, and the production management system 4, respectively. The scanning device 2 scans the identification code of the target product and sends the identification code to the electronic device 5. The electronic device 5 determines a target position of the mobile machine device 2 corresponding to the target product identification code, and controls the mobile machine device 2 to drive the shooting device 3 to move to the target position. The shooting device 3 is controlled to shoot the target product to obtain a target product image, and a preset accessory identification model is stored in the electronic device 5 in advance. The electronic device 5 recognizes the accessory state in the target product image by using a preset accessory recognition model, and stores the identification code of the target product and the corresponding accessory state in an associated manner. The electronic device 5 transmits the associated stored data to the production management system 4.
The application scenario of the inspection method in the product assembly process provided by the embodiment of the invention is an application scenario of inspecting the assembly result of a target product after the target product with accessories is assembled. If the water heater assembling method can be applied to an application scene of testing the assembling result of the water heater after the water heater is assembled. It is understood that the target product may also be other products, such as various household appliances, such as a television, a refrigerator, a washing machine, an air conditioner, and the like, which is not limited in the embodiment of the present invention.
The technical solution of the present invention will be described in detail below with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.
Fig. 2 is a flowchart of an inspection method in a product assembly process according to an embodiment of the present invention. As shown in fig. 2, the main execution body of the inspection method in the product assembly process provided by the present embodiment is an inspection device in the product assembly process, and the inspection device in the product assembly process can be located in an electronic device. The method for checking in the product assembling process provided by the embodiment comprises the following steps:
s101, receiving the target product identification code sent by the scanning equipment.
In this embodiment, the target product is a product with a fitting that needs to be inspected after assembly is completed. For example, the electric heating device can be various household appliances, such as a water heater, a washing machine, a refrigerator, an air conditioner, a television and the like.
Specifically, in the present embodiment, an infrared sensor switch may be provided at the inspection position. After the infrared sensor switch senses that the target product reaches the inspection position, a trigger signal is sent to the scanning device, the scanning device scans and recognizes the target product identification code on the target product, the target product identification code obtained after scanning and recognition is sent to the electronic device, and the electronic device receives the target product identification code sent by the scanning device.
The identification code of the target product may be a one-dimensional code or a two-dimensional code, which is not limited in this embodiment.
The target product identification code can be arranged at a specific position of the target product packaging box, so that the target product identification code can be conveniently scanned and identified by the scanning equipment.
S102, determining the target position of the mobile machine equipment corresponding to the target product identification code.
In this embodiment, since the models of the products are different and the sizes and the outlines of the products are also different, in order to enable the photographing device to obtain an image of a target product that is accurately and easily recognized, a mapping relationship between a preset product identification code and a target position of the mobile machine device may be pre-constructed. And searching the target product identification code in the mapping relation, and determining the target position of the mobile machine equipment which has the mapping relation with the target product identification code.
Wherein the target position is a position to which the mobile machine device moves.
S103, controlling the mobile machine equipment to drive the shooting equipment to move to the target position.
In this embodiment, the mobile machine device may be a mobile robot, and may also be a mobile mechanical structure, for example, a mobile mechanical structure in which a stepping motor drives a screw rod and a slide rail to be combined. And a vision subsystem is fixedly connected to the mobile machine equipment. As shown in fig. 3, the vision subsystem includes at least one photographing apparatus 3. The photographing apparatus 3 may be fixed to the mobile machine apparatus 2 by a mechanical stand. In order to make the captured image of the target product more clear, the vision subsystem may further comprise at least one light source 6. The light source 6 may be fixed to the mobile machine device 2 by a mechanical support. The electronic equipment can send a movement control instruction to a control cabinet of the mobile machine equipment, and the movement control instruction comprises a target position to control the mobile machine equipment to drive the shooting equipment to move to the target position. After the mobile machine device moves to the target position, the shooting device and the light source also move to the corresponding positions.
Wherein, the control cabinet of the mobile machine equipment can be positioned at the bottom of the mobile machine equipment. The light source 6 can be positioned between the shooting device 3 and the target product 7, and the vertical distance between the light source 6 and the target product 7 can be in the interval of 1100mm + -100mm, so that the light source 6 can provide more proper shooting light for the shooting device 3.
And S104, controlling the shooting equipment to shoot the target product to obtain a target product image, and identifying the accessory state in the target product image by adopting a preset accessory identification model.
In this embodiment, the electronic device may send a shooting control instruction to the shooting device to control the shooting device to shoot the target product. The accessory is included in the captured image of the target product.
In this embodiment, the shot target product image is input into a preset accessory recognition model, and the accessory state in the target product image is recognized through the preset accessory recognition model.
It should be noted that, if a plurality of shooting devices are provided, the preset accessory identification model fuses target product images shot by each shooting device to obtain a target product image including a target product panorama, and then identifies the accessory state in the target product image.
The accessory state can be a normal state or an abnormal state. The abnormal state may be a misloading abnormal state or a missing loading abnormal state.
And S105, storing the identification code of the target product and the corresponding accessory state in an associated manner.
In this embodiment, after the identification code of the target product and the corresponding accessory state are stored in an associated manner, when the user queries the accessory state of the target product through the electronic device, the accessory state corresponding to the target product is accurately obtained.
According to the inspection method in the product assembly process, the target product identification code sent by the scanning equipment is received; determining a target position of the mobile machine device corresponding to the target product identification code; controlling the mobile machine equipment to drive the shooting equipment to move to a target position; controlling a shooting device to shoot a target product to obtain a target product image, and identifying the accessory state in the target product image by adopting a preset accessory identification model; and storing the identification code of the target product and the corresponding accessory state in an associated manner. The product identification code is information representing the only product, so that the target product can be accurately identified, the shooting equipment is controlled to shoot the target product, after the target product image is obtained, the preset accessory identification model can be adopted to automatically and accurately identify the accessory state in the target product image, the inspection accuracy is improved, and the inspection efficiency is improved.
Example two
Fig. 4 is a flowchart of a checking method in a product assembling process according to another embodiment of the present invention. As shown in fig. 4, the present embodiment provides a method for checking during product assembly, which further refines steps 102, 104 to 105 and further includes other steps based on the method for checking during product assembly provided in the first embodiment of the present invention. The method for checking in the product assembling process provided by the embodiment comprises the following steps:
s201, receiving the target product identification code sent by the scanning equipment.
In this embodiment, the implementation manner of S201 is similar to that of S101 in the first embodiment of the present invention, and is not described in detail here.
S202, determining the target position of the mobile machine equipment corresponding to the target product identification code.
As an alternative implementation manner, in this embodiment, S202 includes the following steps:
s2021, acquiring a mapping relation between a pre-constructed preset product identification code and a target position of the mobile machine device.
S2022, determining the target position of the mobile machine equipment corresponding to the target product identification code in the mapping relation.
Specifically, in the present embodiment, the vision subsystem includes: a camera device and a light source. The photographing device and the light source are fixed on the mobile machine device, respectively, so that the positional relationship of the photographing device and the light source with the mobile machine device is fixed. And because the models of the products are different and the sizes and the outlines of the products are also different, in order to ensure that the shooting equipment and the light source can keep the most suitable position with the products aiming at each product, the shooting equipment shoots a target product image of which the accessories are easy to identify, so that the mapping relation between the preset product identification code and the target position of the mobile machine equipment is constructed in advance, and after the target product identification code is obtained, the target position of the mobile machine equipment which has the mapping relation with the target product identification code is obtained from the mapping relation.
And S203, controlling the mobile machine equipment to drive the shooting equipment to move to the target position.
Wherein the photographing apparatus is located at the top of the target product.
In this embodiment, as shown in fig. 3, the shooting device is disposed on the top of the target product, and the light source may be located between the shooting device and the target product, so that the light source is closer to the target product, and the shot image of the target product is clearer.
Specifically, in the present embodiment, after the mobile machine device is controlled to move to the target position, the photographing device and the light source can also maintain the most suitable position with the target product.
S204, controlling the shooting equipment to shoot the target product to obtain a target product image, and recognizing the accessory state in the target product image by adopting a preset accessory recognition model.
As an optional implementation manner, in this embodiment, the identifying, in S204, the accessory state in the target product image by using the preset accessory identification model includes the following steps:
s2041, inputting the target product image into the accessory recognition model.
S2042, identifying the appearance and the quantity of the target accessories in the target product image through the accessory identification model, and determining the accessory state according to the appearance and the quantity of the target accessories.
Specifically, in this embodiment, the preset accessory recognition model is a model obtained after training the initial accessory recognition model to converge by using a training sample. The preset accessory identification model can be used for segmenting accessories in the target product image, and the segmented accessories are target accessories. And extracting the number and appearance of the target accessories, and further determining the accessory state according to the appearance and number of the target accessories.
Wherein the appearance of the target accessory may include: shape, size, etc.
Optionally, in S2042, determining the accessory status according to the appearance and the number of the target accessories includes the following steps:
s2042a, determining the model of the target accessory according to the appearance of the target accessory, and determining whether the accessory state is a misloading abnormal state or not according to the model of the target accessory.
In this embodiment, a mapping relationship between the shape, size, and model of the preset accessory is pre-constructed. After the appearance of the target accessory is identified, inquiring in the mapping relation, and acquiring the model which has the mapping relation with the appearance of the target accessory as the model of the target accessory.
In this embodiment, a mapping relationship between a preset product identification code and a corresponding preset accessory model is also pre-established, after the target product identification code is determined, a query is performed in the mapping relationship to obtain a model of a target accessory corresponding to the target product, the model of the target accessory queried in the mapping relationship is compared with the identified model of the target accessory, and if the model is determined to be inconsistent, whether the accessory state is an incorrect installation abnormal state is determined.
S2042b, determining whether the accessory state is a missing-loading abnormal state according to the target accessory number.
In this embodiment, a mapping relationship between a preset product identification code and the number of corresponding accessories is pre-established, after the identification code of the target product is determined, the mapping relationship is queried to obtain the number of target accessories corresponding to the target product, the number of target accessories queried in the mapping relationship is compared with the number of target accessories identified, and if the numbers are inconsistent, the accessory status is determined to be a missing-installation abnormal status.
It is understood that if it is determined that the accessory state is not the misload abnormal state and the neglected load abnormal state, the accessory state is determined to be the normal state.
And S2043, outputting the state of the accessory through the accessory recognition model.
And S205, generating a corresponding checking result according to the state of the accessory.
Specifically, in this embodiment, if the accessory state is the wrong-mounting abnormal state or the missing-mounting abnormal state, the generated inspection result is an abnormal result. And if the accessory state is a normal state, generating a normal result as a detection result.
As an optional implementation manner, in this embodiment, if it is determined that the inspection result is an abnormal result, an abnormal prompt message is sent out, so as to perform a component status review according to the abnormal prompt message.
Specifically, in this embodiment, if it is determined that the inspection result is an abnormal result, in order to ensure that the target product is assembled normally and then enters the next process, an abnormal prompt message is sent out to prompt an inspector to perform component status review according to the abnormal prompt message.
The abnormal prompt information can be in a voice form, an acousto-optic form or a voice integrated acousto-optic form. And the abnormal prompt message can include the specific accessory state, so that the inspector can quickly review the specific accessory state.
And S206, acquiring relevant time information of the target product for inspection.
Wherein checking the relevant time information may include: the code reading time of the scanning device and the verification time of the accessory identification model.
And S207, performing associated storage on the target product identification code, the corresponding accessory state, the inspection result and the inspection related time information.
In this embodiment, the table format may be adopted to perform the associated storage of the target product identification code, the corresponding accessory state, the inspection result, and the inspection-related time information.
It should be noted that the model of the target product, the inspection station code and other related information may also be stored in association.
S208, the data stored in the association mode are sent to a production management system MES.
Specifically, in this embodiment, the electronic device is in communication connection with the production management system MES, and specifically, a TCP/IP network communication protocol may be used for communication, and the data stored in association is sent to the production management system MES.
In this embodiment, as shown in fig. 5, the electronic device may further load a client of the verification method in the product assembly process, and the user may log in the client through a user name and a password. On the operation interface of the client, a user can trigger a query request, for example, other associated information during the checking can be queried through inputting a product identification code and/or checking time, so that the traceability of the checking process of product assembly can be enhanced.
As shown in fig. 5, information such as the status of each device or system communicatively connected to the electronic device may be displayed in a color-differentiated manner on the operation interface of the client. In FIG. 5, the scanning equipment, the electronic equipment and the MES system are all in a normal state.
And S209, controlling the mobile machine equipment to move to the original position.
In this embodiment, the mobile machine device is controlled to move to the home position to prepare for the next inspection of the target product.
According to the inspection method in the product assembly process, the shooting equipment is positioned at the top of the target product; when the accessory state in the target product image is recognized by adopting a preset accessory recognition model, inputting the target product image into the accessory recognition model; identifying the appearance and the quantity of target accessories in the target product image through an accessory identification model, and determining the accessory state according to the appearance and the quantity of the target accessories; and outputting the state of the accessory through the accessory recognition model. Because the shooting equipment is positioned at the top of the target product, a clearer and more complete target product image can be shot. And when the accessory state is identified, the appearance and the quantity of the target accessories are determined simultaneously, and whether the target accessories are mistakenly assembled or not and are neglected to be assembled can be effectively determined.
EXAMPLE III
Fig. 6 is a signaling interaction flowchart of a checking method in a product assembly process according to still another embodiment of the present invention, and as shown in fig. 6, an execution main body of the checking method in the product assembly process according to the present embodiment is a checking system in the product assembly process, and then the checking method in the product assembly process according to the present embodiment includes the following steps:
s301, scanning the identification code of the target product by the scanning device.
In the embodiment, after the infrared sensor switch senses that the target product reaches the inspection position, the trigger signal is sent to the scanning device, and the scanning device scans and identifies the target product identification code on the target product.
S302, the scanning device sends the target product identification code to the electronic device.
S303, the electronic equipment determines the target position of the mobile machine equipment corresponding to the target product identification code.
In this embodiment, the implementation manner of step 303 is similar to that of step 202 in the first embodiment of the present invention, and is not described in detail here.
S304, the electronic equipment sends a movement control instruction to the mobile machine equipment.
Wherein, the movement control command includes: a target location.
In this embodiment, the electronic device is in communication connection with the mobile machine device and the shooting device, respectively, and a specific communication connection mode is not limited.
S305, the mobile machine equipment drives the shooting equipment to move to the target position according to the movement control instruction.
In this embodiment, the mobile machine device analyzes the movement control instruction, obtains the target position, drives the shooting device to move together, and stops when the mobile machine device detects that the mobile machine device moves to the target position.
S306, the electronic equipment sends a shooting control instruction to the shooting equipment.
Wherein, the shooting control instruction comprises shooting parameters.
And S307, shooting the target product by the shooting equipment according to the shooting control instruction to obtain a target product image.
In this embodiment, the shooting device analyzes the shooting control instruction, obtains the shooting parameters, and shoots the target product according to the shooting parameters to obtain the target product image.
And S308, the shooting device sends the target product image to the electronic device.
S309, the electronic equipment identifies the accessory state in the target product image by adopting a preset accessory identification model, and stores the identification code of the target product and the corresponding accessory state in an associated manner.
At step 310, the electronic device sends the associated stored data to the manufacturing management system MES.
The implementation manners of S309 to S310 in this embodiment are similar to the implementation manners of the corresponding steps in the second embodiment of the present invention, and are not described in detail here.
Example four
Fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 7, an electronic device 40 according to this embodiment includes: a memory 41, a processor 42, and a transceiver 44.
a memory 41 for storing a computer program; and a transceiver 44 for transmitting and receiving data to and from the scanning device, the mobile machine device, the photographing device, and the manufacturing management system MES.
The processor 42 is configured to be capable of performing the method of embodiment one or embodiment two.
EXAMPLE five
A fifth embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the method provided in the first embodiment or the second embodiment.
EXAMPLE six
A fifth embodiment of the present invention provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the computer program implements the method provided in the first embodiment or the second embodiment.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional unit.
The integrated unit implemented in the form of a software functional unit may be stored in a computer readable storage medium. The software functional unit is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a server, or a network device) or a processor (processor) to execute some steps of the methods according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
It is obvious to those skilled in the art that, for convenience and simplicity of description, the foregoing division of the functional modules is merely used as an example, and in practical applications, the above function distribution may be performed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to perform all or part of the above described functions. For the specific working process of the device described above, reference may be made to the corresponding process in the foregoing method embodiment, which is not described herein again.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for checking during product assembly, the method being applied to an electronic device, the method comprising:
receiving a target product identification code sent by scanning equipment;
determining a target position of the mobile machine device corresponding to the target product identification code;
controlling the mobile machine equipment to drive the shooting equipment to move to the target position;
controlling the shooting equipment to shoot the target product so as to obtain a target product image, and identifying the accessory state in the target product image by adopting a preset accessory identification model;
and storing the target product identification code and the corresponding accessory state in an associated manner.
2. The method of claim 1, wherein determining the target location of the mobile machine device corresponding to the target product identification code comprises:
acquiring a mapping relation between a pre-established preset product identification code and a target position of mobile machine equipment;
and determining the target position of the mobile machine equipment corresponding to the target product identification code in the mapping relation.
3. The method of claim 1, wherein the camera is located on top of the target product;
the method for recognizing the accessory state in the target product image by adopting the preset accessory recognition model comprises the following steps:
inputting the target product image into the accessory identification model;
identifying the appearance and the quantity of target accessories in the target product image through the accessory identification model, and determining the accessory state according to the appearance and the quantity of the target accessories;
outputting the accessory status through the accessory recognition model.
4. The method of claim 3, wherein determining the fitting status based on the target fitting appearance and quantity comprises:
determining the model of a target accessory according to the appearance of the target accessory, and determining whether the accessory state is a wrong-installation abnormal state or not according to the model of the target accessory;
and determining whether the accessory state is a neglected loading abnormal state or not according to the target accessory number.
5. The method of claim 1, wherein after recognizing the accessory status in the target product image by using the preset accessory recognition model, the method further comprises:
generating a corresponding inspection result according to the state of the accessory;
acquiring relevant time information of inspection of a target product;
the associating and storing the target product identification code and the corresponding accessory state comprises the following steps:
and performing associated storage on the target product identification code, the corresponding accessory state, the inspection result and the inspection related time information.
6. The method of claim 5, after generating the corresponding verification result based on the fitting status, further comprising:
if the inspection result is determined to be an abnormal result, sending abnormal prompt information to carry out accessory state recheck according to the abnormal prompt information;
after the target product identification code, the corresponding accessory state, the inspection result and the inspection related time information are stored in an associated manner, the method further comprises the following steps:
and sending the data of the associated storage to a production management system (MES).
7. An electronic device, comprising: a memory, a processor, and a transceiver;
the processor, the memory, and the transceiver circuitry are interconnected;
the memory for storing a computer program; the transceiver is used for receiving and transmitting data with the scanning equipment, the mobile machine equipment, the shooting equipment and the production management system MES;
the processor is configured to be capable of performing the method of any one of claims 1-6.
8. An inspection system during product assembly, comprising: scanning equipment, mobile machine equipment, shooting equipment, MES and the electronic equipment of claim 7;
the electronic equipment is in communication connection with the scanning equipment, the mobile machine equipment, the shooting equipment and the MES respectively;
the shooting equipment is fixed on the mobile machine equipment.
9. A computer-readable storage medium, having stored thereon a computer program;
the computer program, when executed by a processor, implements the method of any one of claims 1-6.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the method of any of claims 1-6 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110521260.3A CN113139957A (en) | 2021-05-13 | 2021-05-13 | Inspection method, apparatus, system and medium in product assembly process |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110521260.3A CN113139957A (en) | 2021-05-13 | 2021-05-13 | Inspection method, apparatus, system and medium in product assembly process |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113139957A true CN113139957A (en) | 2021-07-20 |
Family
ID=76817896
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110521260.3A Pending CN113139957A (en) | 2021-05-13 | 2021-05-13 | Inspection method, apparatus, system and medium in product assembly process |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113139957A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114693313A (en) * | 2022-03-21 | 2022-07-01 | 北京环宇博亚科技有限公司 | Identification code-based warehousing equipment detection method and device, electronic equipment and medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107607043A (en) * | 2017-08-21 | 2018-01-19 | 奇瑞汽车股份有限公司 | A kind of blind nut automatic detection device and method based on MES system |
| CN107767069A (en) * | 2017-11-03 | 2018-03-06 | 河南科技大学 | A kind of assembly line assembling error-preventing method and system based on Bar Code |
| CN109544498A (en) * | 2018-11-29 | 2019-03-29 | 燕山大学 | A kind of image adaptive fusion method |
| WO2020114416A1 (en) * | 2018-12-07 | 2020-06-11 | 阿里巴巴集团控股有限公司 | Processing method for product restocking and product identification, device, and electronic apparatus |
| CN112113968A (en) * | 2020-09-14 | 2020-12-22 | 格力电器（武汉）有限公司 | Detection system, visual detection device and detection method |
-
2021
- 2021-05-13 CN CN202110521260.3A patent/CN113139957A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107607043A (en) * | 2017-08-21 | 2018-01-19 | 奇瑞汽车股份有限公司 | A kind of blind nut automatic detection device and method based on MES system |
| CN107767069A (en) * | 2017-11-03 | 2018-03-06 | 河南科技大学 | A kind of assembly line assembling error-preventing method and system based on Bar Code |
| CN109544498A (en) * | 2018-11-29 | 2019-03-29 | 燕山大学 | A kind of image adaptive fusion method |
| WO2020114416A1 (en) * | 2018-12-07 | 2020-06-11 | 阿里巴巴集团控股有限公司 | Processing method for product restocking and product identification, device, and electronic apparatus |
| CN112113968A (en) * | 2020-09-14 | 2020-12-22 | 格力电器（武汉）有限公司 | Detection system, visual detection device and detection method |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114693313A (en) * | 2022-03-21 | 2022-07-01 | 北京环宇博亚科技有限公司 | Identification code-based warehousing equipment detection method and device, electronic equipment and medium |
| CN114693313B (en) * | 2022-03-21 | 2022-09-20 | 北京环宇博亚科技有限公司 | Identification code-based warehousing equipment detection method and device, electronic equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7388977B2 (en) | Test system and method for portable electronic apparatus | |
| CN111104341B (en) | Automatic testing method, device, equipment and storage medium for intelligent household equipment | |
| CN110647894B (en) | Fault diagnosis method and system of electrical equipment, cloud server and storage medium | |
| CN110340636B (en) | Compressor assembling method and system | |
| US11499732B2 (en) | Method, device and system for wireless networking of air conditioning unit | |
| CN106403188B (en) | A kind of air-conditioning maintaining method and its maintenance device | |
| CN110275532B (en) | Robot control method and device and visual equipment control method and device | |
| CN110560376B (en) | Product surface defect detection method and device | |
| CN113139957A (en) | Inspection method, apparatus, system and medium in product assembly process | |
| CN115862177A (en) | Equipment inspection method and device | |
| CN115302411B (en) | Surface cleaning system based on image recognition and control method thereof | |
| CN109214251B (en) | Image recognition device and method for machine room monitoring | |
| CN112232454A (en) | Data writing method and system, terminal, household appliance and computer device | |
| CN110443099A (en) | The method of object identification system and its automatic identification object identity | |
| CN111598280A (en) | Equipment repair reporting method and device | |
| CN111626621A (en) | Intelligent equipment production test method and device and electronic device | |
| CN113326068A (en) | Cloud testing method, device, equipment and storage medium on intelligent household equipment | |
| CN110361978B (en) | Intelligent equipment control method, device and system based on Internet of things operating system | |
| CN216209508U (en) | Detection equipment and detection system | |
| CN111145444A (en) | Spacecraft switching test tool management method, system and medium | |
| CN103577964A (en) | Method and system for acquiring product circulation regional information and verifying authenticity of product | |
| CN109269034A (en) | A kind of Internet of Things air-conditioning system | |
| CN111652856B (en) | In-place detection method and system for lifting mechanism and related device | |
| CN114218430A (en) | Remote cooperative equipment operation and maintenance system, method and device | |
| CN109272504B (en) | Method, device and system for detecting and tracing concave-convex defect of vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210720 |