WO2021164448A1

WO2021164448A1 - Quality abnormity recording method and apparatus, and augmented reality device, system and medium

Info

Publication number: WO2021164448A1
Application number: PCT/CN2021/070296
Authority: WO
Inventors: 崔维存; 陈录城; 孙明; 王勇
Original assignee: 青岛海尔工业智能研究院有限公司; 海尔卡奥斯物联生态科技有限公司
Priority date: 2020-02-20
Filing date: 2021-01-05
Publication date: 2021-08-26
Also published as: CN113282778A

Abstract

A quality abnormity recording method and apparatus, and an augmented reality device, a system and a medium. The quality abnormity recording method comprises: obtaining an evidence-obtaining picture of a rejected product by means of a camera on the augmented reality device (110); and uploading the evidence-obtaining picture to a background system, and performing classified processing on the evidence-obtaining picture by means of the background system (120).

Description

Quality abnormal recording method, device, augmented reality equipment, system and medium

The present disclosure claims the priority of a Chinese patent application filed with the Chinese Patent Office with an application number of 202010103677.3 on February 20, 2020, and the entire content of the above application is incorporated into the present disclosure by reference.

Technical field

The embodiments of the present application relate to quality detection technology, for example, to a quality abnormal recording method, device, augmented reality equipment, system, and medium.

Background technique

In the process of product manufacturing, abnormal product quality often occurs. For example, due to poor feeding or poor pre-process, the defect rate is high or a large number of defects are found. Therefore, in every manufacturing It is very important to carry out quality inspections on products in the link, and forensic records of abnormal quality scenes.

In related technologies, generally, the operator holds the inspection equipment, inspects the products in production according to the set inspection items and inspection procedures, and records when abnormal quality products are detected. On the one hand, the operator needs to The inspection information is recorded, and the exception handling form is filled in and sent to the quality management department. The inspection information cannot be notified to the quality inspector in time. The quality inspection efficiency is low. On the other hand, because most of the information is displayed on the handheld device, the operator cannot be relieved. With both hands, the long-term inspection process is likely to cause fatigue of the staff, resulting in a decrease in the accuracy of the quality inspection.

Summary of the invention

The embodiments of the present application provide a method, device, augmented reality equipment, system and medium for recording abnormal quality. The augmented reality device is used to collect evidence and record abnormal quality products, freeing the hands of the operator and improving the efficiency and quality of recording abnormal quality. Check accuracy.

In the first aspect, an embodiment of the present application provides a quality abnormality recording method, the method is applied to an augmented reality device, and the quality abnormality recording method includes:

Obtain forensic pictures of unqualified products through the camera on the augmented reality device;

Upload the forensic picture to a back-end system to classify the forensic picture through the back-end system.

In a second aspect, an embodiment of the present application provides a quality abnormality recording method, the method is applied to a background system, and the quality abnormality recording method includes:

Receive forensic pictures of substandard products uploaded by augmented reality devices;

Perform cluster analysis on the forensic pictures, and construct a quality abnormal database based on the analysis results;

Wherein, the abnormal quality database includes forensic pictures of all abnormal quality types.

In a third aspect, an embodiment of the present application provides a quality abnormality recording method, and the quality abnormality recording method includes:

Augmented reality equipment obtains forensic pictures of unqualified products through the camera;

The augmented reality device uploads the forensic picture to the background system;

The background system classifies the forensic pictures.

In a fourth aspect, an embodiment of the present application also provides a quality abnormality recording device, and the quality abnormality recording device includes:

The image acquisition module is configured to acquire forensic pictures of substandard products through the camera on the augmented reality device;

The audio and video transmission module is configured to upload the forensic picture to a background system to classify and process the forensic picture through the background system.

In a fifth aspect, an embodiment of the present application also provides an augmented reality device, the augmented reality device includes:

processor;

The memory is configured to store programs;

When the program is executed by the processor, the processor realizes the quality abnormality recording method provided by any embodiment of the present application.

In the sixth aspect, an embodiment of the present application also provides a quality abnormality recording system, including: an augmented reality device and a background system;

The augmented reality device is configured to obtain forensic pictures of unqualified products through a camera, and send the forensic pictures to a background system;

The background system is configured to classify the forensic pictures.

In a seventh aspect, an embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the quality provided by any embodiment of the present application is realized. Abnormal record quality Abnormal record method.

Description of the drawings

FIG. 1 is a flowchart of a method for recording abnormal quality in Embodiment 1 of the present application;

2 is a flowchart of a method for recording abnormal quality in the second embodiment of the present application;

FIG. 3 is a flowchart of a method for recording abnormal quality in the third embodiment of the present application;

4 is a flowchart of a method for recording abnormal quality in the fourth embodiment of the present application;

5 is a schematic structural diagram of a quality abnormality recording device in Embodiment 5 of the present application;

FIG. 6 is a schematic structural diagram of an AR device provided by Embodiment 6 of the present application;

FIG. 7 is a schematic structural diagram of a quality abnormality recording system provided by Embodiment 7 of the present application.

Detailed ways

The application will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described here are only used to explain the present application. In addition, it should be noted that, for ease of description, the drawings only show a part of the structure related to the present application instead of all of the structure.

Example one

Figure 1 is a flow chart of a method for recording abnormal quality in the first embodiment of the application. The technical solution of this embodiment is applied to an AR device and is suitable for the case of obtaining a forensic picture of a substandard product through the AR device. The method can be The quality abnormality recording device is executed, which can be implemented by software and/or hardware, and can be integrated in various general-purpose computer equipment, and can include the following steps:

Step 110: Obtain a forensic picture of a substandard product through a camera on the AR device.

Among them, AR (Augmented Reality) technology is a technology that ingeniously integrates virtual information with the real world. It uses a variety of technical methods such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing. After computer-generated text, images, three-dimensional models, music, video and other virtual information are simulated and simulated, they are applied to the real world.

The AR device (ie, augmented reality device) used in this embodiment is dual-display glasses, equipped with a camera, lens, microphone, speaker, and processor, and is suitable for scenes that require a wide field of view and high-definition display of the scene. , It can be used alone or in conjunction with a mobile phone or a PDA (Personal Digital Assistant, handheld computer).

In this embodiment, the camera configured on the AR device is used to obtain forensic pictures of unqualified products. In one embodiment, the operator wears the AR device to perform quality inspection on the product. When it is detected that the product does not meet the testing standards, the AR device The on-board camera obtains forensic pictures of unqualified products for later management. Exemplarily, the operator wears an AR device to detect whether the product has a scratch. When a product with a scratch is detected, it can initiate a voice command, for example, input "photograph for evidence" through a microphone to indicate that the AR device is unqualified The product takes photos for evidence and obtains forensic pictures.

Optionally, in order to ensure the quality of forensic pictures, in this embodiment, the method further includes:

Display the forensic picture through the lens on the AR device;

Judge whether the forensic picture meets the standard;

If not, re-obtain forensic pictures through the camera.

In this optional embodiment, in order to ensure that the captured forensic pictures are clear and usable, after shooting the target product, the forensic pictures are displayed through the lens on the AR device, and the operator can determine whether the captured pictures are clear, and Whether to take a complete forensic picture, when the taken forensic picture does not meet the standard, you can re-shoot to ensure the usability of the forensic picture in the later management process.

Optionally, after the AR device obtains the forensic picture, the forensic picture can also be viewed through the PDA or mobile phone connected to the AR device through the network, and the forensic picture can be simply marked. Illustratively, when the forensic picture contains the product Overall, in order to highlight the location where the product does not meet the testing standards, you can mark the location to facilitate subsequent processing and evidence collection.

Step 120: Upload the forensic pictures to the back-end system to classify the forensic pictures through the back-end system.

Among them, the back-end system and the AR device are connected through the network and can be divided into a back-end data management subsystem, a device management subsystem, and an intelligent analysis subsystem, which are used to store forensic pictures, manage AR devices, and perform forensic pictures. Classification processing.

In this embodiment, after the AR device sends the obtained forensic pictures to the background system, on the one hand, the forensic pictures can be stored through the data management subsystem, and on the other hand, the forensic pictures can be gathered through the intelligent analysis subsystem. Class analysis, to achieve the effect of categorizing the forensic pictures according to the set rules, which is convenient for subsequent forensics and management. Illustratively, it can be classified according to the unqualified types of products corresponding to the forensic pictures, and the forensic pictures corresponding to the products with scratches in appearance As the first type, the forensic pictures corresponding to products with unclear trademarks on the outer surface of the product are regarded as the second type, and the corresponding forensic pictures of products whose size and standard size error exceeds the allowable error range are regarded as the third type.

The technical solution of the embodiment of the present application obtains forensic pictures of unqualified products through the camera on the AR device, and uploads the forensic pictures to the back-end system to classify the forensic pictures through the back-end system. On the one hand, it frees the operator’s attention. Both hands improve the efficiency of the operator’s quality inspection. On the other hand, the forensic pictures can be uploaded directly to the background system, which is intuitive and clear, and there is no need to fill in the quality inspection form to feed back the quality abnormalities to the quality inspection department, which further improves the quality Abnormal recording efficiency and quality inspection accuracy.

Example two

FIG. 2 is a flowchart of a method for recording abnormal quality in the second embodiment of the application. This embodiment is modified on the basis of the above-mentioned embodiment, and provides the steps that can be included in obtaining the forensic picture of the unqualified product. The following describes a quality abnormal recording method provided in the second embodiment of the present application with reference to FIG. 2, which includes the following steps:

Step 210: Display the information of the quality inspection process and quality inspection standards pre-stored in the AR device to prompt the operator to perform product quality inspection based on the information of the quality inspection process and quality inspection standards;

Among them, the quality inspection process includes all the quality inspection items of the product and the execution sequence of all the quality inspection items, and the quality inspection standards include the qualification standards of the quality inspection items.

In this embodiment, the information of the quality inspection process and quality inspection standards pre-stored in the AR device is displayed through the lens of the AR device, which serves as a reminder to the operator of the quality inspection process and the inspection standards of each process, and improves the accuracy of the quality inspection. Spend.

Exemplarily, the pre-stored quality inspection process and quality inspection standards are displayed in front of the operator through the lens of the AR device. For example, the lens displays "Step 1, measure the outer dimensions of the product, the standard size is 15 cm, and the allowable error is 3mm", or "Step 2. Observe whether there are color spots on the surface of the product, if there are more than 3 color spots, it will be unqualified." At this time, the operator can perform product quality inspection according to the above prompt information and follow the set steps.

Step 220: When at least one quality inspection item of the target product does not meet the quality inspection standard, photograph the target product to obtain a forensic picture of the target product.

In this embodiment, when it is detected that at least one quality inspection item of the target product does not meet the quality inspection standard, the product is photographed through the camera of the AR device to obtain the target forensic picture. Exemplarily, when it is detected that at least one quality inspection item of the target product does not meet the quality inspection standards, for example, if it is detected that the product size differs from the standard size by more than 3 mm, the target product can be photographed by initiating a voice command to obtain the target Forensic images of the product.

Optionally, when at least one quality inspection item of the target product does not meet the quality inspection standards, the target product is photographed to obtain forensic pictures of the target product, including:

When at least one quality inspection item of the target product does not meet the quality inspection standards, in response to the recording start instruction initiated by the operator, the target product is photographed and the forensic picture of the target product is obtained;

Among them, the recording start instruction includes a button instruction or a voice instruction.

In this optional embodiment, when it is detected that at least one quality inspection item of the target product does not meet the quality inspection standards, the operator initiates a recording start instruction to the AR device, and the AR device responds to the recording start instruction and uses the camera to shoot the target product , Obtain forensic pictures of the target product, where the recording start instruction includes a key instruction or a voice instruction. Illustratively, the operator can shoot the target product by pressing the shooting button set on the AR device, or input it through a microphone. "Take a picture for evidence" instructs the AR device to take a picture.

Optionally, in order to ensure the standardization of operator quality inspection operations, in this embodiment, the method further includes:

Record the quality inspection operation of the operator through the camera of the AR device, obtain the video information of the quality inspection operation, and send the video information to the background system;

When receiving a message that the operator's quality inspection operation is not standardized based on the video information sent by the background system, a voice prompting the quality inspection operation error is played.

In this optional embodiment, an optional way to monitor the operator’s quality inspection operation is provided. First, when the operator wears the AR device to perform the quality inspection operation, the operator will be monitored through the AR device’s camera. The quality inspection operation of the company is recorded, the video information of the quality inspection operation is obtained, and the video information is sent to the back-end system in real time. The back-end system judges whether the operator’s quality inspection operation meets the specifications. If the operation does not meet the specifications, it will report to the AR device Send a message that the quality inspection operation is not standardized. At this time, the AR device will play a voice prompting the quality inspection operation error through the speaker.

Step 230: Upload the forensic pictures to the back-end system to classify the forensic pictures through the back-end system.

In the technical solution of the embodiment of this application, the operator performs product quality inspection when the AR device prompts the quality inspection process and quality inspection standards, and when the product does not meet the quality inspection standards, obtains the unqualified product through the camera on the AR device Forensic pictures and upload the forensic pictures to the back-end system to classify the forensic pictures through the back-end system. During the forensic process, the quality inspector's operations can also be monitored and prompts for operational errors in time. On the one hand, liberation The operator’s hands are improved, and the operator’s quality inspection efficiency and the standardization of quality inspection operations are improved. On the other hand, the forensic pictures can be directly uploaded to the back-end system, which is intuitive and clear, and there is no need to fill in the quality inspection form to make the quality abnormal. The situation is fed back to the quality inspection department, which further improves the efficiency of recording abnormal quality.

Example three

Figure 3 is a flowchart of a quality abnormal recording method in the third embodiment of the application. The technical solution of this embodiment is applied to a background system, and is suitable for receiving and processing the forensic pictures of unqualified products sent by the AR device through the background system Case. The following describes a quality abnormal recording method provided in Embodiment 3 of the present application with reference to FIG. 3, which includes the following steps:

Step 310: Receive a forensic picture of a substandard product uploaded by the AR device.

In this embodiment, the back-end system first receives forensic pictures of unqualified products uploaded by the AR device. In one embodiment, the AR device and the back-end system are connected via a network, that is, the back-end system receives the forensic pictures sent by the AR device through the network to verify It is saved and processed later.

Optionally, before receiving the forensic image of the substandard product uploaded by the AR device, the method further includes:

Initialize the AR device, and bind the AR device and the operator's ID.

In this optional embodiment, the operation of the background system before receiving the forensic picture of the substandard product uploaded by the AR device is provided, which may include initializing the AR device and binding the AR device and the operator ID On the one hand, binding the AR device and the operator ID is convenient for setting the usage rights of the AR device. For example, the AR device can only perform related forensics operations by logging in with the user ID bound to it, which improves information security. On the other hand, , The forensic pictures obtained by the AR device can be correspondingly stored with the operator number, so that the responsibility is assigned to the person.

Receive the video information of the quality inspection operation sent by the AR device;

According to the video information, judge whether the quality inspection operation of the operator matches the quality inspection operation specification;

If not, send a message to the AR device that the operator's quality inspection operation is not standardized.

In this optional embodiment, a way to detect whether the operator’s quality inspection operation is standardized is provided. In one embodiment, the background system receives the video information of the quality inspection operation sent by the AR device, and extracts the quality inspection operation from the video information. Inspect operation action, compare the quality inspection operation action with the quality inspection operation standard, determine whether the operator's quality inspection operation matches the quality inspection operation specification, if not, send the operator's quality inspection operation to the AR device that is not standardized information.

Exemplarily, after the background system receives the video information of the quality inspection operation, it needs to determine whether the quality inspection operation of the inspection label meets the specifications. The operation specification of the quality inspection operation is that the operator needs to touch the label and confirm it for 1-2 seconds. At this time, the background system extracts the quality inspection operation of the operator to detect whether the label is correctly pasted from the video information, and detects whether the label is blocked for 1-2 seconds, if it is, it is considered that the detection operation is in compliance with the specification; otherwise, it is considered the operation If the operator misses the inspection operation or the operation is not standardized, it is necessary to send a message to the AR device that the operator’s quality inspection operation is not standardized.

Step 320: Perform cluster analysis on the forensic pictures, and construct a quality abnormal database according to the analysis results;

Among them, the abnormal quality database contains all the forensic pictures of abnormal quality types.

In this embodiment, after receiving the forensic pictures sent by the AR device, the background system stores the forensic pictures, and at the same time, performs cluster analysis on the forensic pictures, and constructs a quality abnormality database based on the analysis results. Exemplarily, it can be classified according to the unqualified type of the products corresponding to the forensic pictures, the forensic pictures corresponding to the products with scratches on the appearance are regarded as the first type, and the forensic pictures corresponding to the products with unclear trademarks on the outer surface of the product are regarded as the second type. , The product size and standard size error exceeds the allowable range of product corresponding forensic pictures as the third category.

The technical solution of the embodiment of this application receives the forensic pictures of unqualified products uploaded by the AR device through the background system, and performs cluster analysis on the forensic pictures, and finally builds a quality abnormal database based on the analysis results. On the one hand, it liberates the operator’s Both hands improve the operator's quality inspection efficiency and the accuracy of quality inspection operations. On the other hand, cluster analysis of the forensic pictures through the back-end system generates a quality abnormal database, which is convenient for management personnel to follow-up inquiries, forensics, and supervision and management.

Example four

FIG. 4 is a flowchart of a quality abnormality recording method in the fourth embodiment of the application. The technical solution of this embodiment is applied to a quality abnormality recording system, and is suitable for obtaining forensic pictures of substandard products through AR equipment. The following describes a quality abnormal recording method provided in the fourth embodiment of the present application with reference to FIG. 4, which includes the following steps:

Step 410: The AR device obtains a forensic picture of the unqualified product through the camera.

In this embodiment, the camera configured on the AR device is used to obtain forensic pictures of unqualified products. In one embodiment, the operator wears the AR device to perform quality inspection on the product. When it is detected that the product does not meet the testing standards, the AR device The on-board camera obtains forensic pictures of unqualified products for later management. Exemplarily, the operator wears an AR device to detect whether there are color points in the appearance of the product. When it detects that there are more than 3 color points in the appearance, the product is deemed to not meet the detection standards. At this time, a voice command can be initiated, for example, through a microphone. Enter "photograph for evidence" to instruct the AR device to take photographs of unqualified products and obtain forensic pictures.

Step 420: The AR device uploads the forensic picture to the background system.

In this embodiment, the AR device uploads the forensic pictures to the background system through the audio and video transmission module, so as to store and process the forensic pictures through the background system.

Step 430: The background system classifies the forensic pictures.

In this embodiment, the background system stores the forensic pictures after receiving the forensic pictures sent by the AR device, and at the same time, implements the classification processing of the forensic pictures by performing cluster analysis on the forensic pictures. Exemplarily, it can be classified according to the unqualified type of the product corresponding to the forensic picture, which is convenient for later management and query.

The technical solution of the embodiment of this application obtains forensic pictures of unqualified products through the camera on the AR device, uploads the forensic pictures to the back-end system, and then classifies and processes the forensic pictures through the back-end system. On the one hand, it liberates the operator’s Both hands, and you can directly upload the forensic pictures to the back-end system, which is intuitive and clear, which improves the operator's quality inspection efficiency and accuracy of quality inspection operations. On the other hand, the back-end system performs cluster analysis on the forensic images to generate a quality abnormal database , It is convenient for management staff to inquire, obtain evidence and supervise management work.

Example five

FIG. 5 is a schematic structural diagram of a quality abnormality recording device provided in Embodiment 5 of this application. The quality abnormality recording device includes: an image acquisition module 510 and an audio and video transmission module 520.

The image acquisition module 510 is configured to acquire forensic pictures of unqualified products through a camera on the AR device;

The audio and video transmission module 520 is configured to upload the forensic picture to a background system to classify the forensic picture through the background system.

The technical solution of the embodiment of the present application obtains forensic pictures of unqualified products through the camera on the AR device, and uploads the forensic pictures to the back-end system to classify the forensic pictures through the back-end system. On the one hand, it frees the operator’s attention. Both hands improve the efficiency of the operator. On the other hand, the forensic pictures can be directly uploaded to the back-end system, which is intuitive and clear, and there is no need to fill in the quality inspection form to feed back the quality abnormalities to the quality inspection department, which further improves the quality abnormal records Efficiency and accuracy of quality inspection.

Optionally, the image acquisition module 510 includes:

The information display unit is configured to display the information of the quality inspection process and quality inspection standards pre-stored in the AR device to prompt the operator to perform product inspection based on the information of the quality inspection process and the quality inspection standards. Quality inspection

Wherein, the quality inspection process includes all the quality inspection items of the product and the execution sequence of all the quality inspection items, and the quality inspection standards include the qualification standards of the quality inspection items;

The image acquisition unit is configured to take a picture of the target product when at least one quality inspection item of the target product does not meet the quality inspection standard, and obtain a forensic picture of the target product.

Optionally, the image acquisition unit may be configured as:

When at least one quality inspection item of the target product does not meet the quality inspection standard, in response to the recording start instruction initiated by the operator, photograph the target product to obtain a forensic picture of the target product;

Wherein, the recording start instruction includes a key instruction or a voice instruction.

Optionally, the abnormal quality recording device further includes:

An information display module configured to display the forensic picture through a lens on the AR device;

The standard judgment module is configured to judge whether the forensic picture meets the standard;

The image acquisition module 510, when the forensic picture does not meet the standard, re-acquires the forensic picture through the camera.

Optionally, the abnormal quality recording device further includes:

The audio and video transmission module is configured to record the quality inspection operation of the operator through the camera of the AR device, obtain the video information of the quality inspection operation, and send the video information to the background system;

The voice interaction module is configured to, when receiving a message that the operator's quality inspection operation is not standardized based on the video information sent by the background system, play a voice prompting the quality inspection operation error.

The quality anomaly recording device provided by the embodiment of the present application can execute the quality anomaly recording method provided by any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

Example Six

FIG. 6 is a schematic structural diagram of an AR device provided by Embodiment 6 of the application. As shown in FIG. 6, the electronic device includes a processor 60 and a memory 61; the number of processors 60 in the device may be one or more, as shown in FIG. In FIG. 6, a processor 60 is taken as an example; the processor 60 and the memory 61 in the device may be connected by a bus or other methods, and the connection by a bus is taken as an example in FIG. 6.

As a computer-readable storage medium, the memory 61 can be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to a quality abnormality recording method in the embodiment of the present application (for example, a quality abnormality recording device). The image acquisition module 510 and the audio and video transmission module 520 in the ). The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, that is, realizes the above-mentioned abnormal quality recording method.

The method includes:

Obtain forensic pictures of unqualified products through the camera on the AR device;

The memory 61 may mainly include a program storage area and a data storage area. The program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the terminal, and the like. In addition, the memory 61 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some examples, the memory 61 may include a memory remotely provided with respect to the processor 60, and these remote memories may be connected to the device through a network. Examples of the aforementioned network may include the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

Optionally, the AR device further includes a camera 62, a lens 63, a microphone 64, and a speaker 65;

The camera 62 is configured to record the quality inspection operation of the operator, obtain video information of the quality inspection operation, and obtain forensic pictures.

Among them, the camera 62 is installed directly in front of the AR device, and is used to record the quality inspection operation of the operator when the operator is performing the quality inspection operation, and obtain the video information of the quality inspection operation, so as to facilitate the verification of whether the quality inspection operation of the operator conforms to the specification In one embodiment, when at least one quality inspection item of the target product does not meet the quality inspection standard, the camera 62 responds to the operator's recording start instruction to shoot the target product to obtain a forensic picture of the target product.

The lens 63 is configured to display the information of the quality inspection process and quality inspection standards pre-stored in the AR device, and to display the forensic picture.

Among them, the lens 63 of the AR device is used to display the information of the quality inspection process and quality inspection standards pre-stored in the AR device on the one hand, prompting the operator to perform quality inspection according to the quality inspection process and quality inspection standards, and on the other hand In order to display the forensic pictures, it is convenient for the operator to judge whether the currently taken forensic pictures meet the shooting standards, for example, whether it is clear, whether the shooting is comprehensive, etc.

The microphone 64 is configured to collect voice instructions input by the operator.

Among them, the microphone 64 is mainly used to collect voice instructions input by the operator, for example, to collect the “photograph for evidence” instruction initiated by the operator when a certain product does not meet the detection standard.

The speaker 65 is configured to play a voice prompting the quality inspection operation error when the AR device receives a message that the operator's quality inspection operation is not standardized based on the video information sent by the background system.

The speaker 65 is configured to play quality inspection related information to the operator. For example, when the AR device receives a message that the operator’s quality inspection operation is not standardized based on the video information sent by the background system, it will play a voice prompting the quality inspection operation error. .

Example Seven

FIG. 7 is a schematic structural diagram of a quality abnormality recording system provided by Embodiment 7 of the application. The quality abnormality recording system includes: an AR device 1 and a background system 2;

The AR device 1 is configured to obtain forensic pictures of unqualified products through a camera, and send the forensic pictures to the background system.

Among them, the AR device is dual-display glasses, equipped with a camera 62, a lens 63, a microphone 64, a speaker 65, a memory 61 and a processor 60, which are used to obtain forensic pictures of unqualified products through the camera and send the forensic pictures to the background system On the other hand, the AR device is also used to record the quality inspection operation of the operator during the quality inspection operation of the operator, obtain the video information of the quality inspection operation, and send the video information to the background system.

The background system 2 is configured to classify the forensic pictures.

Among them, the background system 2 is used to classify the forensic pictures, and includes a background data management subsystem 21, a device management subsystem 22, and an intelligent analysis subsystem 23. Among them, the background data management subsystem 21 is configured to store forensic data uploaded by the AR device; the device management subsystem 22 is configured to initialize the AR device and bind the AR device and the operator's ID , To facilitate subsequent management; the intelligent analysis subsystem 23 is configured to process the forensic data, generate a quality abnormal database, and process video information to determine whether the operator’s quality inspection operation matches the quality inspection operation specification and improve quality inspection Operation accuracy.

In an embodiment, the background data management subsystem 21 includes an account management module 211, a log management module 212, a data management module 213, and a data storage module 214. Among them, the account management module 211 is configured to store and manage the login account of the AR device 1 connected to the back-end system 2 through the network; the log management module 212 is configured to record the operation log of the back-end system 2; the data management module 213 is configured to Perform preliminary processing on the received forensic pictures or video information; the data storage module 214 is configured to store the received forensic pictures or video information.

The equipment management subsystem 22 includes an equipment configuration module 221, an equipment management module 222, and an industrial information data interface module 223. Among them, the device configuration module 221 is configured to configure the basic parameters of the AR device 1; the device management module 222 is configured to manage the authority and login account of the AR device 1; the industrial information data interface module 223 is configured to configure the AR device Data transmission authority and data transmission parameters of 1 and system background 2.

The intelligent analysis subsystem 23 includes a video cluster analysis module 231, a video retrieval module 232, an image cluster analysis module 233, and an image retrieval module 234. Among them, the video clustering analysis module 231 is configured to perform cluster analysis on the video information uploaded by the AR device 1; the video retrieval module 232 is configured to respond to the user's video retrieval operation and retrieve the video information according to the set retrieval conditions; The class analysis module 233 is configured to perform cluster analysis on the forensic pictures uploaded by the AR device; the image retrieval module 234 is configured to retrieve the forensic pictures according to the set retrieval conditions in response to the user's image retrieval operation.

Example eight

The eighth embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. The computer program is used to perform a quality abnormality recording method when the computer program is executed by a computer processor. The method includes:

Through the above description of the implementation manners, those skilled in the art can clearly understand that this application can be implemented by software and necessary general-purpose hardware, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of this application essentially or the part that contributes to the related technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, Read-Only Memory (ROM), Random Access Memory (RAM), Flash memory (FLASH), hard disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, A server, or a network device, etc.) execute the method described in each embodiment of the present application.

It is worth noting that in the embodiment of the above-mentioned abnormal quality recording device, the units and modules included are only divided according to functional logic, as long as the corresponding functions can be realized; in addition, the names of the functional units are only In order to distinguish from each other.

Claims

A quality abnormality recording method, wherein the quality abnormality recording method is applied to an augmented reality device, and the method includes:

Obtain forensic pictures of unqualified products through the camera on the augmented reality device;

Upload the forensic picture to a back-end system to classify the forensic picture through the back-end system.
The method for recording abnormal quality according to claim 1, wherein said obtaining a forensic picture of a substandard product comprises:

Displaying the information of the quality inspection process and quality inspection standards pre-stored in the augmented reality device to prompt the operator to perform product quality inspection according to the information of the quality inspection process and the quality inspection standards;

Wherein, the quality inspection process includes all the quality inspection items of the product and the execution sequence of all the quality inspection items, and the quality inspection standards include the qualification standards of the quality inspection items;

When at least one of the quality inspection items of the target product does not meet the quality inspection standard, the target product is photographed to obtain a forensic picture of the target product.
The quality abnormality recording method according to claim 2, wherein when at least one of the quality inspection items of the target product does not meet the quality inspection standard, the target product is photographed to obtain evidence of the target product Pictures, including:

When at least one of the quality inspection items of the target product does not meet the quality inspection standards, in response to a recording start instruction initiated by an operator, photograph the target product to obtain a forensic picture of the target product;

Wherein, the recording start instruction includes a key instruction or a voice instruction.
The method for recording abnormal quality according to claim 1 or 3, further comprising:

Displaying the forensic picture through the lens on the augmented reality device;

Determine whether the forensic picture meets the standard;

Based on the judgment result that the forensic picture does not meet the standard, the forensic picture is acquired through the camera again.
The method for recording abnormal quality according to claim 2, further comprising:

Record the quality inspection operation of the operator through the camera of the augmented reality device, obtain the video information of the quality inspection operation, and send the video information to the background system;

When receiving a message that the operator's quality inspection operation is not standardized based on the video information sent by the background system, a voice prompting the quality inspection operation error is played.
A method for recording abnormal quality, the method for recording abnormal quality is applied to a background system, and the method for recording abnormal quality includes:

Receive forensic pictures of substandard products uploaded by augmented reality devices;

Perform cluster analysis on the forensic pictures, and construct a quality abnormal database based on the analysis results;

Wherein, the abnormal quality database includes forensic pictures of all abnormal quality types.
The method for recording abnormal quality according to claim 6, before receiving the forensic picture of the substandard product uploaded by the augmented reality device, the method for recording abnormal quality further comprises:

The augmented reality device is initialized, and the augmented reality device and the operator ID are bound.
7. The quality abnormality recording method according to claim 7, wherein the quality abnormality recording method further comprises:

Receiving the video information of the quality inspection operation sent by the augmented reality device;

According to the video information, determine whether the quality inspection operation of the operator matches the quality inspection operation specification;

Based on the judgment result that the operator's quality inspection operation does not match the quality inspection operation specification, a message that the operator's quality inspection operation is not standardized is sent to the augmented reality device.
A method for recording abnormal quality, including:

Augmented reality equipment obtains forensic pictures of unqualified products through the camera;

The augmented reality device uploads the forensic picture to the background system;

The background system classifies the forensic pictures.
A quality abnormal recording device, including:

The image acquisition module is configured to acquire forensic pictures of substandard products through the camera on the augmented reality device;

The audio and video transmission module is configured to upload the forensic picture to a background system to classify and process the forensic picture through the background system.
An augmented reality device including:

processor;

The storage device is configured to store a program,

When the program is executed by the processor, the processor realizes the quality abnormality recording method according to any one of claims 1-9.
The device according to claim 11, further comprising: a camera, a lens, a microphone, and a speaker;

The camera is configured to record the quality inspection operation of the operator, obtain video information of the quality inspection operation, and obtain forensic pictures;

The lens is configured to display the information of the quality inspection process and quality inspection standards pre-stored in the augmented reality device, and to display the forensic picture;

The microphone is configured to collect voice instructions input by an operator;

The speaker is configured to play a voice prompting the quality inspection operation error when the augmented reality device receives a message that the operator's quality inspection operation is not standardized based on the video information sent by the background system.
A quality anomaly recording system, including: augmented reality equipment and background system;

The augmented reality device is configured to obtain forensic pictures of unqualified products through a camera, and send the forensic pictures to a background system;

The background system is configured to classify the forensic pictures.
A computer-readable storage medium with a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, the method for recording abnormal quality according to any one of claims 1-9 is realized.