CN117788763A - Method and system for enhancing AR prompt function based on big data analysis - Google Patents
Method and system for enhancing AR prompt function based on big data analysis Download PDFInfo
- Publication number
- CN117788763A CN117788763A CN202311743483.XA CN202311743483A CN117788763A CN 117788763 A CN117788763 A CN 117788763A CN 202311743483 A CN202311743483 A CN 202311743483A CN 117788763 A CN117788763 A CN 117788763A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- gesture
- prompt
- worker
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000007405 data analysis Methods 0.000 title claims abstract description 30
- 230000002708 enhancing effect Effects 0.000 title claims abstract description 25
- 238000012549 training Methods 0.000 claims abstract description 64
- 230000009471 action Effects 0.000 claims abstract description 52
- 238000004519 manufacturing process Methods 0.000 claims abstract description 39
- 238000012545 processing Methods 0.000 claims description 58
- 238000004364 calculation method Methods 0.000 claims description 18
- 238000007781 pre-processing Methods 0.000 claims description 18
- 230000033001 locomotion Effects 0.000 claims description 17
- 230000002093 peripheral effect Effects 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 10
- 230000009467 reduction Effects 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000004140 cleaning Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 description 15
- 238000004458 analytical method Methods 0.000 description 11
- 238000012544 monitoring process Methods 0.000 description 9
- 230000006870 function Effects 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 6
- 238000001514 detection method Methods 0.000 description 3
- 238000001035 drying Methods 0.000 description 3
- 230000007774 longterm Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- General Factory Administration (AREA)
Abstract
The invention provides a method and a system for enhancing an AR prompt function based on big data analysis. The method for enhancing the AR prompt function based on big data analysis comprises the following steps: collecting sample data, and training an AR prompt content model by using the sample data to obtain a trained AR prompt content model; collecting operation data of each worker on a production line in real time; the operation actions of workers are identified in real time through the gesture identification model, and operation error information of the workers is obtained; and sending the operation error information to AR equipment worn by a worker for prompt content display. The system comprises modules corresponding to the steps of the method.
Description
Technical Field
The invention provides a method and a system for enhancing an AR prompt function based on big data analysis, and belongs to the technical field of AR data processing.
Background
The AR may provide real-time operational guidance for the plant operator, displaying relevant information, such as operating steps of the device, parameter settings, and safety prompts, through a head-mounted display device or smart glasses. This may help new employees adapt to work faster and reduce human error. Partial AR guidance prompt is not accurate enough, and workers with different business cognition degrees and business familiarity degrees still have a certain degree of operation errors under the AR guidance prompt. If the effect of the AR prompt is not fed back according to the actual operation error condition of the current user, the effect of the AR prompt is greatly discounted.
Disclosure of Invention
The invention provides a method and a system for enhancing an AR prompt function based on big data analysis, which are used for solving the problem that if the effect of AR prompt is not fed back according to the actual operation error condition of the current user, the effect of AR prompt is greatly discounted:
a method for enhancing an AR prompt facility based on big data analysis, the method for enhancing an AR prompt facility based on big data analysis comprising:
collecting sample data, and training an AR prompt content model by using the sample data to obtain a trained AR prompt content model;
collecting operation data of each worker on a production line in real time;
the operation actions of workers are identified in real time through the gesture identification model, and operation error information of the workers is obtained;
and sending the operation error information to AR equipment worn by a worker for prompt content display.
Further, collecting sample data, and training the AR prompt content model by using the sample data to obtain a trained AR prompt content model, including:
information data of operation guidance of workers in production by AR equipment is collected in real time through a camera; the information data comprises text data information, image data information, feedback information of workers and record information of worker operation of AR prompt;
Carrying out data preprocessing on the information data to obtain preprocessed information data, wherein the data preprocessing comprises data cleaning processing, data denoising processing, format conversion processing and standardization processing;
because the collected operation data of workers on the production line have great uncorrelated noise interference, for example, other uncorrelated objects enter a mirror during operation, other objects are blocked during operation, and the like, the noise data have great influence on the subsequent model training and the model accuracy, and in order to eliminate the drying of the data under the above conditions, the data is denoised by adopting the following algorithm:
step one: the value at the set point (x, y) is f (x, y), and the values of the adjacent 8 points around the set point are marked as f (x) i ,y i ) I is the number of its peripheral adjacent points, then the average value of the values of its peripheral adjacent points is:
wherein the method comprises the steps ofIs the average of the values of the neighboring points around it.
Step two: calculating an adjustment coefficient according to the values at the points (x, y) and the values of the adjacent points around the points, wherein the calculation formula is as follows:
where k (x, y) is the adjustment coefficient.
Step three: according to the calculation results of the first step and the second step, calculating the numerical value at the point (x, y) after denoising, wherein the calculation formula is as follows:
Where F (x, y) is a value at the point (x, y) after the denoising process.
The method carries out denoising treatment on the data by adopting the mean value and the adjustment coefficient according to the conditions of other points around the image point, effectively reduces noise interference in the collected operation data of workers on the production line, particularly noise interference caused by the conditions of mirror entering of irrelevant objects, shielding of other objects and the like, provides accurate data for subsequent model training, and improves the accuracy of the model.
Generating a sample data set by using the preprocessed information data;
calling an AR prompt content model from a database;
and training the AR prompt content model by using the sample data set to obtain the trained AR prompt content model.
Further, the operation data of each worker on the production line is collected in real time, including:
a camera is arranged on each worker station;
the camera is controlled in real time to adjust the focal length according to the distance between the moving position of the worker and the camera, so as to obtain the operation action video data of the worker with definition meeting the image recognition requirement;
performing frame processing on the operation motion video data to obtain a plurality of frame image blocks corresponding to the operation motion video data;
And carrying out noise reduction processing on the plurality of frame image blocks and then sending the frame image blocks to the gesture recognition model.
Further, the operation actions of the workers are identified in real time through the gesture identification model, and operation error information of the workers is obtained, and the method comprises the following steps:
after receiving a plurality of frame image blocks, the gesture recognition model recognizes gesture actions of workers in each frame image block to obtain operation gesture information of the workers;
comparing the operation gesture information of the worker with an operation standard gesture to obtain gesture similarity;
and when the gesture similarity is lower than a preset gesture similarity threshold, marking and extracting the operation gesture information corresponding to the gesture similarity lower than the preset gesture similarity threshold.
Further, the operation error information is sent to AR equipment worn by a worker for prompt content display, and the method comprises the following steps:
inputting operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold value into an AR prompt content model;
generating prompt information corresponding to operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold through an AR prompt content model;
And transmitting the prompt information to the AR equipment for prompt content display.
A system for enhancing AR prompt facility based on big data analysis, the system for enhancing AR prompt facility based on big data analysis comprising:
the model training module is used for collecting sample data, training the AR prompt content model by using the sample data, and obtaining the AR prompt content model after training;
the real-time acquisition module is used for acquiring operation data of each worker on the production line in real time;
the error information acquisition module is used for identifying the operation actions of the workers in real time through the gesture identification model to acquire operation error information of the workers;
and the content display module is used for sending the operation error information to AR equipment worn by a worker for displaying prompt content.
Further, the model training module includes:
the information data acquisition module is used for acquiring information data of operation guidance of a worker in production by using AR equipment in real time through a camera; the information data comprises text data information, image data information, feedback information of workers and record information of worker operation of AR prompt;
the information data preprocessing module is used for carrying out data preprocessing on the information data to obtain preprocessed information data, wherein the data preprocessing comprises data cleaning processing, data denoising processing, format conversion processing and standardization processing;
A sample data set generating module, configured to generate a sample data set using the preprocessed information data;
the content model calling module is used for calling the AR prompt content model from the database;
and the training execution module is used for training the AR prompt content model by utilizing the sample data set to obtain the AR prompt content model after training.
The data is denoised by adopting the following algorithm:
step one: the value at the set point (x, y) is f (x, y), and the values of the adjacent 8 points around the set point are marked as f (x) i ,y i ) I is the number of its peripheral adjacent points, then the average value of the values of its peripheral adjacent points is:
wherein the method comprises the steps ofIs the average of the values of the neighboring points around it.
Step two: calculating an adjustment coefficient according to the values at the points (x, y) and the values of the adjacent points around the points, wherein the calculation formula is as follows:
where k (x, y) is the adjustment coefficient.
Step three: according to the calculation results of the first step and the second step, calculating the numerical value at the point (x, y) after denoising, wherein the calculation formula is as follows:
where F (x, t) is a value at the point (x, y) after the denoising process.
Further, the real-time acquisition module includes:
the camera configuration module is used for configuring cameras on the stations of each worker;
The focal length adjusting module is used for controlling the camera to adjust focal length in real time according to the distance between the moving position of the worker and the camera, and acquiring operation action video data of the worker with definition meeting the image recognition requirement;
the frame image acquisition module is used for carrying out frame processing on the operation action video data to obtain a plurality of frame image blocks corresponding to the operation action video data;
the noise reduction processing module is used for carrying out noise reduction processing on the plurality of frame image blocks and then sending the frame image blocks to the gesture recognition model.
Further, the error information acquisition module includes:
the gesture information acquisition module is used for identifying the gesture actions of the workers in each frame image block after the gesture identification model receives the plurality of frame image blocks, so as to obtain the operation gesture information of the workers;
the gesture similarity acquisition module is used for comparing the operation gesture information of the worker with an operation standard gesture to obtain gesture similarity;
and the marking and information extracting module is used for marking and extracting the operation gesture information corresponding to the gesture similarity lower than the preset gesture similarity threshold when the gesture similarity is lower than the preset gesture similarity threshold.
Further, the content presentation module includes:
the information input module is used for inputting the operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold value into the AR prompt content model;
the prompt information acquisition module is used for generating prompt information corresponding to the operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold through the AR prompt content model;
and the content display execution module is used for transmitting the prompt information to the AR equipment to display the prompt content.
The invention has the beneficial effects that:
according to the method and the system for enhancing the AR prompt function based on big data analysis, when a worker starts AR operation guidance on a specific station on a production line, the prompt content of the current AR and the actual operation result of the worker are tracked and recorded. According to the data of human errors still generated by an operator when the AR prompt is started, the reasons for the errors of the operator are analyzed, and the AR prompt content is improved. The AR prompt is more detailed and accurate, and loss caused by human errors is reduced.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
fig. 2 is a system block diagram of the system of the present invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
The embodiment of the invention provides a method for enhancing an AR prompt function based on big data analysis, which comprises the following steps of:
s1, collecting sample data, and training an AR prompt content model by using the sample data to obtain a trained AR prompt content model;
s2, collecting operation data of each worker on a production line in real time;
s3, identifying the operation actions of the workers in real time through the gesture identification model, and obtaining operation error information of the workers;
and S4, sending the operation error information to AR equipment worn by a worker for displaying prompt contents.
The working principle of the technical scheme is as follows: data collection and training (S1): first, the system collects sample data, which may include worker's operational behavior and errors, as well as information related to the production line. These data are used to train an AR hint content model that learns features and patterns about operational errors.
Real-time operation data acquisition (S2): on the production line, the system collects operation data of each worker in real time, which may include hand motions of the worker, tool use, operation steps, and the like. These data are used to analyze the operation of the worker and detect potential errors.
Gesture recognition (S3): the system uses the gesture recognition model to recognize the worker's operational actions, particularly erroneous operational actions, in real time. This may involve using cameras or other sensor technology to monitor the posture and movements of the worker.
AR hint content presentation (S4): when the system detects an erroneous operation of the worker, it transmits related error information to the AR device worn by the worker. The AR device displays a corresponding prompt according to the error information to remind a worker how to correct the error or improve the operation.
The technical scheme has the effects that: operational error prevention: by monitoring and identifying worker operation errors in real time, the system can provide a prompt before an error occurs, helping the worker avoid the error.
Real-time feedback: the worker may obtain real-time AR cues, which may help to improve the accuracy and efficiency of the operation.
And (3) personalized guidance: the trained AR prompt content based model may provide personalized guidance to each worker based on their specific mistakes and needs.
Data analysis: the collected operational data may be used for further analysis to improve the production flow and training methods.
In general, the technical solution of the present embodiment combines big data analysis and AR technology, providing a method to help workers avoid errors and to improve operational efficiency. It can provide useful support and guidance in the production environment through real-time monitoring, recognition and feedback.
In one embodiment of the present invention, collecting sample data, and training an AR hint content model using the sample data to obtain a trained AR hint content model, comprising:
s101, acquiring information data of operation guidance of a worker in production by using AR equipment in real time through a camera; the information data comprises text data information, image data information, feedback information of workers and record information of worker operation of AR prompt;
s102, carrying out data preprocessing on the information data to obtain preprocessed information data, wherein the data preprocessing comprises data cleaning processing, data denoising processing, format conversion processing and standardization processing;
s103, generating a sample data set by using the preprocessed information data;
S104, calling an AR prompt content model from a database;
s105, training the AR prompt content model by using the sample data set to obtain a trained AR prompt content model.
Because the collected operation data of workers on the production line have great uncorrelated noise interference, for example, other uncorrelated objects enter a mirror during operation, other objects are blocked during operation, and the like, the noise data have great influence on the subsequent model training and the model accuracy, and in order to eliminate the drying of the data under the above conditions, the data is denoised by adopting the following algorithm:
step one: the value at the set point (x, y) is f (x, y), and the values of the adjacent 8 points around the set point are marked as f (x) i ,y i ) I is the number of its peripheral adjacent points, then the average value of the values of its peripheral adjacent points is:
wherein the method comprises the steps ofIs the average of the values of the neighboring points around it.
Step two: calculating an adjustment coefficient according to the values at the points (x, y) and the values of the adjacent points around the points, wherein the calculation formula is as follows:
where k (x, y) is the adjustment coefficient.
Step three: according to the calculation results of the first step and the second step, calculating the numerical value at the point (x, y) after denoising, wherein the calculation formula is as follows:
Where F (x, y) is a value at the point (x, y) after the denoising process.
The method carries out denoising treatment on the data by adopting the mean value and the adjustment coefficient according to the conditions of other points around the image point, effectively reduces noise interference in the collected operation data of workers on the production line, particularly noise interference caused by the conditions of mirror entering of irrelevant objects, shielding of other objects and the like, provides accurate data for subsequent model training, and improves the accuracy of the model.
The working principle of the technical scheme is as follows: data acquisition (S101): information data of operation guidance of workers on a production line by using AR equipment is collected in real time through a camera. The information data includes text data information of the AR prompt, image data information, feedback information of the worker, and record information of the worker's operation. These data provide sample information required for AR hint content model training.
Data preprocessing (S102): the collected information data is required to be subjected to data preprocessing, including data cleaning processing, data denoising processing, format conversion processing and standardization processing. These steps are intended to ensure data quality and consistency for use in training the model.
Sample data set generation (S103): a sample data set is generated using the preprocessed information data. This sample dataset contains data samples for training the AR hint content model, including the input data and corresponding output tags.
Model invocation and training (S104 and S105): the AR hint content model is invoked from the database and then trained using the generated sample dataset. During the training process, the model will learn how to generate relevant AR cues from the input data (information data collected by the camera) for subsequent real-time cues.
The technical scheme has the effects that: personalized prompting content: by training the AR prompt content model, the system can provide personalized prompt content according to the requirements of workers and the operation situation, so that the working efficiency and accuracy are improved.
Real-time feedback: the model can process the acquired data in real time and generate corresponding AR prompt content, so that workers can obtain instant feedback and guidance in operation.
Data driven model: through collecting and utilizing sample data to train, the AR prompt content model can be continuously improved and optimized according to actual operation data so as to adapt to different working scenes.
The training effect is improved: through the AR prompt content model, more effective training and operation guidance can be provided, helping workers to master new skills faster and reduce errors.
In general, the technical scheme of the embodiment combines big data analysis and AR technology, aims at providing personalized and real-time operation prompts, and is expected to improve production efficiency and quality.
In one embodiment of the present invention, collecting operation data of each worker in real time on a production line includes:
s201, configuring cameras on stations of each worker;
s202, controlling the camera to adjust the focal length in real time according to the distance between the moving position of the worker and the camera, so as to obtain operation motion video data of the worker with definition meeting the image recognition requirement;
s203, performing frame processing on the operation motion video data to obtain a plurality of frame image blocks corresponding to the operation motion video data;
s204, carrying out noise reduction processing on the plurality of frame image blocks and then sending the frame image blocks to the gesture recognition model.
The working principle of the technical scheme is as follows: configuration camera (S201): a camera is arranged at each worker's station. These cameras are used to capture the movements and gestures of a worker during operation.
Focal length adjustment (S202): and controlling the camera in real time to adjust the focal length according to the moving position of the worker and the distance between the camera and the camera. This ensures that the video data obtained has sufficient sharpness to meet the subsequent image recognition requirements. The focus adjustment helps to ensure image quality for subsequent image processing and analysis.
Frame processing (S203): and carrying out frame processing on the operation motion video data, and dividing the video stream into a plurality of frame image blocks. Each frame image block represents a snapshot of the video stream for subsequent gesture recognition and analysis.
Noise reduction processing and gesture recognition (S204): the processed frame image blocks are sent to a gesture recognition model for analysis. This model is used to detect and identify the operational actions and gestures of the worker. The method can analyze the hand actions of workers, detect errors or anomalies and generate corresponding AR prompt contents.
The technical scheme has the effects that: monitoring operation actions in real time: by configuring the camera and the focal length adjustment, the system can monitor the operation actions of workers on the production line in real time and capture the detailed information of the operation.
Clear image data: the focal length adjustment ensures that the camera is able to capture image data with sharpness that meets the image recognition requirements for subsequent processing and analysis.
Automatic gesture recognition: the gesture recognition model may automatically detect the gesture and motion of the worker, helping to identify potential errors and provide relevant AR cues.
Improved operation guidance: based on real-time monitoring and gesture recognition, the system can provide more accurate operation guidance and error detection to improve work efficiency and reduce errors.
In general, the technical solution of the present embodiment combines camera, focal length adjustment and gesture recognition techniques for real-time monitoring and analyzing of the operation actions of workers to improve operation guidance and training.
According to one embodiment of the invention, the operation actions of the workers are identified in real time through the gesture identification model, and the operation error information of the workers is obtained, and the method comprises the following steps:
s301, after receiving a plurality of frame image blocks, the gesture recognition model recognizes gesture actions of workers in each frame image block, and operation gesture information of the workers is obtained;
s302, comparing the operation gesture information of the worker with an operation standard gesture to obtain gesture similarity;
and S303, when the gesture similarity is lower than a preset gesture similarity threshold, marking and extracting the operation gesture information corresponding to the gesture similarity lower than the preset gesture similarity threshold.
The working principle of the technical scheme is as follows: gesture recognition model analysis (S301): the gesture recognition model receives a plurality of frame image tiles that contain the gestures and operational actions of the worker. The model recognizes the posture actions of the worker in each frame image block to obtain the operation posture information of the worker.
Comparing the operation gesture (S302): the recognized worker operation posture information is compared with the operation standard posture. This step involves comparing the worker's pose with a predefined standard operating pose.
Error flag and information extraction (S303): when the posture similarity is lower than a preset posture similarity threshold, the posture is considered to be insufficiently similar to the standard operation posture. In this case, the operation posture information of the worker is marked as possibly erroneous, and the related erroneous information is extracted. This information may be used to generate AR cues to alert workers to correct the operation.
The technical scheme has the effects that: real-time error detection: through the gesture recognition model, the system can detect the operation actions of workers in real time, particularly those actions which are not similar to the standard operation gestures, so that potential errors can be found in advance.
Automatic error marking: based on the similarity comparison, the system may automatically mark that there may be an erroneous operation action when the operation pose of the worker does not match the standard pose.
Improving the operation quality: the technical scheme of the embodiment can help to improve the operation quality and reduce the occurrence of potential errors, thereby improving the working efficiency and the product quality.
In summary, the technical solution of the present embodiment allows for real-time recognition of the operation actions of a worker through gesture recognition to discover and mark potential errors in advance, providing opportunities for improved operation guidance and training.
According to one embodiment of the invention, the operation error information is sent to AR equipment worn by a worker for prompt content display, and the method comprises the following steps:
s401, inputting operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold value into an AR prompt content model;
s402, generating prompt information corresponding to operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold through an AR prompt content model;
s403, transmitting the prompt information to the AR equipment for prompt content display.
The working principle of the technical scheme is as follows: operation error information input (S401): first, operation error information marked by the previous step, in particular, operation posture information having a posture similarity lower than a preset posture similarity threshold value is input to the AR hint content model.
Prompt information generation (S402): the AR presentation content model analyzes the inputted operation error information and generates presentation information corresponding to the error information. These hints typically include specific guidance on how to correct the operational error.
Information is transmitted to the AR device (S403): finally, the generated hint information is transmitted to the AR device worn by the worker for display to the worker on the AR device.
The technical scheme has the effects that: real-time error correction: by sending the prompt message to the AR equipment, a worker can receive instant feedback and correct errors in actual operation, and operation accuracy is improved.
The training requirements are reduced: by providing real-time guidance, the technical solution of the present embodiment may reduce the need for worker training, as workers may obtain guidance while operating without having to rely on long-term training.
Work efficiency is improved: the worker can adjust the operation thereof more rapidly, reducing errors and operation dead time, thereby improving production efficiency.
In general, the technical solution of the present embodiment helps workers improve operations, reduce errors, improve efficiency, and reduce training dependency by providing immediate operation error prompt information on the AR device.
The embodiment of the invention provides a system for enhancing an AR prompt function based on big data analysis, as shown in fig. 2, the system for enhancing the AR prompt function based on big data analysis comprises:
the model training module is used for collecting sample data, training the AR prompt content model by using the sample data, and obtaining the AR prompt content model after training;
The real-time acquisition module is used for acquiring operation data of each worker on the production line in real time;
the error information acquisition module is used for identifying the operation actions of the workers in real time through the gesture identification model to acquire operation error information of the workers;
and the content display module is used for sending the operation error information to AR equipment worn by a worker for displaying prompt content.
The working principle of the technical scheme is as follows: first, the system collects sample data, which may include worker's operational behavior and errors, as well as information related to the production line. These data are used to train an AR hint content model that learns features and patterns about operational errors.
On the production line, the system collects operation data of each worker in real time, which may include hand motions of the worker, tool use, operation steps, and the like. These data are used to analyze the operation of the worker and detect potential errors.
The system uses the gesture recognition model to recognize the worker's operational actions, particularly erroneous operational actions, in real time. This may involve using cameras or other sensor technology to monitor the posture and movements of the worker.
When the system detects an erroneous operation of the worker, it transmits related error information to the AR device worn by the worker. The AR device displays a corresponding prompt according to the error information to remind a worker how to correct the error or improve the operation.
The technical scheme has the effects that: operational error prevention: by monitoring and identifying worker operation errors in real time, the system can provide a prompt before an error occurs, helping the worker avoid the error. The worker may obtain real-time AR cues, which may help to improve the accuracy and efficiency of the operation. The trained AR prompt content based model may provide personalized guidance to each worker based on their specific mistakes and needs. The collected operational data may be used for further analysis to improve the production flow and training methods.
In general, the technical solution of the present embodiment combines big data analysis and AR technology, providing a method to help workers avoid errors and to improve operational efficiency. It can provide useful support and guidance in the production environment through real-time monitoring, recognition and feedback.
In one embodiment of the present invention, the model training module includes:
the information data acquisition module is used for acquiring information data of operation guidance of a worker in production by using AR equipment in real time through a camera; the information data comprises text data information, image data information, feedback information of workers and record information of worker operation of AR prompt;
The information data preprocessing module is used for carrying out data preprocessing on the information data to obtain preprocessed information data, wherein the data preprocessing comprises data cleaning processing, data denoising processing, format conversion processing and standardization processing;
a sample data set generating module, configured to generate a sample data set using the preprocessed information data;
the content model calling module is used for calling the AR prompt content model from the database;
and the training execution module is used for training the AR prompt content model by utilizing the sample data set to obtain the AR prompt content model after training.
Because the collected operation data of workers on the production line have great uncorrelated noise interference, for example, other uncorrelated objects enter a mirror during operation, other objects are blocked during operation, and the like, the noise data have great influence on the subsequent model training and the model accuracy, and in order to eliminate the drying of the data under the above conditions, the data is denoised by adopting the following algorithm:
step one: the value at the set point (x, y) is f (x, y), and the values of the adjacent 8 points around the set point are marked as f (x) i ,y i ) I is the number of its peripheral adjacent points, then the average value of the values of its peripheral adjacent points is:
Wherein the method comprises the steps ofIs the average of the values of the neighboring points around it.
Step two: calculating an adjustment coefficient according to the values at the points (x, y) and the values of the adjacent points around the points, wherein the calculation formula is as follows:
where k (x, y) is the adjustment coefficient.
Step three: according to the calculation results of the first step and the second step, calculating the numerical value at the point (x, y) after denoising, wherein the calculation formula is as follows:
where F (x, y) is a value at the point (x, y) after the denoising process.
The method carries out denoising treatment on the data by adopting the mean value and the adjustment coefficient according to the conditions of other points around the image point, effectively reduces noise interference in the collected operation data of workers on the production line, particularly noise interference caused by the conditions of mirror entering of irrelevant objects, shielding of other objects and the like, provides accurate data for subsequent model training, and improves the accuracy of the model.
The working principle of the technical scheme is as follows: information data of operation guidance of workers on a production line by using AR equipment is collected in real time through a camera. The information data includes text data information of the AR prompt, image data information, feedback information of the worker, and record information of the worker's operation. These data provide sample information required for AR hint content model training.
The collected information data is required to be subjected to data preprocessing, including data cleaning processing, data denoising processing, format conversion processing and standardization processing. These steps are intended to ensure data quality and consistency for use in training the model.
A sample data set is generated using the preprocessed information data. This sample dataset contains data samples for training the AR hint content model, including the input data and corresponding output tags.
The AR hint content model is invoked from the database and then trained using the generated sample dataset. During the training process, the model will learn how to generate relevant AR cues from the input data (information data collected by the camera) for subsequent real-time cues.
The technical scheme has the effects that: by training the AR prompt content model, the system can provide personalized prompt content according to the requirements of workers and the operation situation, so that the working efficiency and accuracy are improved. The model can process the acquired data in real time and generate corresponding AR prompt content, so that workers can obtain instant feedback and guidance in operation. Through collecting and utilizing sample data to train, the AR prompt content model can be continuously improved and optimized according to actual operation data so as to adapt to different working scenes. Through the AR prompt content model, more effective training and operation guidance can be provided, helping workers to master new skills faster and reduce errors.
In general, the technical scheme of the embodiment combines big data analysis and AR technology, aims at providing personalized and real-time operation prompts, and is expected to improve production efficiency and quality.
In one embodiment of the present invention, the real-time acquisition module includes:
the camera configuration module is used for configuring cameras on the stations of each worker;
the focal length adjusting module is used for controlling the camera to adjust focal length in real time according to the distance between the moving position of the worker and the camera, and acquiring operation action video data of the worker with definition meeting the image recognition requirement;
the frame image acquisition module is used for carrying out frame processing on the operation action video data to obtain a plurality of frame image blocks corresponding to the operation action video data;
the noise reduction processing module is used for carrying out noise reduction processing on the plurality of frame image blocks and then sending the frame image blocks to the gesture recognition model.
The working principle of the technical scheme is as follows: a camera is arranged at each worker's station. These cameras are used to capture the movements and gestures of a worker during operation. And controlling the camera in real time to adjust the focal length according to the moving position of the worker and the distance between the camera and the camera. This ensures that the video data obtained has sufficient sharpness to meet the subsequent image recognition requirements. The focus adjustment helps to ensure image quality for subsequent image processing and analysis. And carrying out frame processing on the operation motion video data, and dividing the video stream into a plurality of frame image blocks. Each frame image block represents a snapshot of the video stream for subsequent gesture recognition and analysis. The processed frame image blocks are sent to a gesture recognition model for analysis. This model is used to detect and identify the operational actions and gestures of the worker. The method can analyze the hand actions of workers, detect errors or anomalies and generate corresponding AR prompt contents.
The technical scheme has the effects that: by configuring the camera and the focal length adjustment, the system can monitor the operation actions of workers on the production line in real time and capture the detailed information of the operation. The focal length adjustment ensures that the camera is able to capture image data with sharpness that meets the image recognition requirements for subsequent processing and analysis. The gesture recognition model may automatically detect the gesture and motion of the worker, helping to identify potential errors and provide relevant AR cues. Based on real-time monitoring and gesture recognition, the system can provide more accurate operation guidance and error detection to improve work efficiency and reduce errors.
In general, the technical solution of the present embodiment combines camera, focal length adjustment and gesture recognition techniques for real-time monitoring and analyzing of the operation actions of workers to improve operation guidance and training.
In one embodiment of the present invention, the error information acquisition module includes:
the gesture information acquisition module is used for identifying the gesture actions of the workers in each frame image block after the gesture identification model receives the plurality of frame image blocks, so as to obtain the operation gesture information of the workers;
the gesture similarity acquisition module is used for comparing the operation gesture information of the worker with an operation standard gesture to obtain gesture similarity;
And the marking and information extracting module is used for marking and extracting the operation gesture information corresponding to the gesture similarity lower than the preset gesture similarity threshold when the gesture similarity is lower than the preset gesture similarity threshold.
The working principle of the technical scheme is as follows: the gesture recognition model receives a plurality of frame image tiles that contain the gestures and operational actions of the worker. The model recognizes the posture actions of the worker in each frame image block to obtain the operation posture information of the worker. The recognized worker operation posture information is compared with the operation standard posture. This step involves comparing the worker's pose with a predefined standard operating pose. When the posture similarity is lower than a preset posture similarity threshold, the posture is considered to be insufficiently similar to the standard operation posture. In this case, the operation posture information of the worker is marked as possibly erroneous, and the related erroneous information is extracted. This information may be used to generate AR cues to alert workers to correct the operation.
The technical scheme has the effects that: through the gesture recognition model, the system can detect the operation actions of workers in real time, particularly those actions which are not similar to the standard operation gestures, so that potential errors can be found in advance. Based on the similarity comparison, the system may automatically mark that there may be an erroneous operation action when the operation pose of the worker does not match the standard pose. The technical scheme of the embodiment can help to improve the operation quality and reduce the occurrence of potential errors, thereby improving the working efficiency and the product quality.
In summary, the technical solution of the present embodiment allows for real-time recognition of the operation actions of a worker through gesture recognition to discover and mark potential errors in advance, providing opportunities for improved operation guidance and training.
In one embodiment of the present invention, the content presentation module includes:
the information input module is used for inputting the operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold value into the AR prompt content model;
the prompt information acquisition module is used for generating prompt information corresponding to the operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold through the AR prompt content model;
and the content display execution module is used for transmitting the prompt information to the AR equipment to display the prompt content.
The working principle of the technical scheme is as follows: first, operation error information marked by the previous step, in particular, operation posture information having a posture similarity lower than a preset posture similarity threshold value is input to the AR hint content model. The AR presentation content model analyzes the inputted operation error information and generates presentation information corresponding to the error information. These hints typically include specific guidance on how to correct the operational error. Finally, the generated hint information is transmitted to the AR device worn by the worker for display to the worker on the AR device.
The technical scheme has the effects that: by sending the prompt message to the AR equipment, a worker can receive instant feedback and correct errors in actual operation, and operation accuracy is improved. By providing real-time guidance, the technical solution of the present embodiment may reduce the need for worker training, as workers may obtain guidance while operating without having to rely on long-term training. The worker can adjust the operation thereof more rapidly, reducing errors and operation dead time, thereby improving production efficiency.
In general, the technical solution of the present embodiment helps workers improve operations, reduce errors, improve efficiency, and reduce training dependency by providing immediate operation error prompt information on the AR device.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. The method for enhancing the AR prompt function based on the big data analysis is characterized by comprising the following steps of:
Collecting sample data, and training an AR prompt content model by using the sample data to obtain a trained AR prompt content model;
collecting operation data of each worker on a production line in real time;
the operation actions of workers are identified in real time through the gesture identification model, and operation error information of the workers is obtained;
and sending the operation error information to AR equipment worn by a worker for prompt content display.
2. The method for enhancing an AR prompt facility based on big data analysis according to claim 1, wherein collecting sample data and training an AR prompt content model using the sample data to obtain a trained AR prompt content model comprises:
information data of operation guidance of workers in production by AR equipment is collected in real time through a camera; the information data comprises text data information, image data information, feedback information of workers and record information of worker operation of AR prompt;
carrying out data preprocessing on the information data to obtain preprocessed information data, wherein the data preprocessing comprises data cleaning processing, data denoising processing, format conversion processing and standardization processing;
Generating a sample data set by using the preprocessed information data;
calling an AR prompt content model from a database;
training the AR prompt content model by utilizing the sample data set to obtain a trained AR prompt content model;
the denoising processing for the data by adopting the following algorithm comprises the following steps:
step one: the value at the set point (x, y) is f (x, y), and the values of the adjacent 8 points around the set point are marked as f (x) i ,y i ) I is the number of its peripheral adjacent points, then the average value of the values of its peripheral adjacent points is:
wherein the method comprises the steps ofAn average value of the values of the neighboring points around the same;
step two: calculating an adjustment coefficient according to the values at the points (x, y) and the values of the adjacent points around the points, wherein the calculation formula is as follows:
wherein k (x, y) is an adjustment coefficient;
step three: according to the calculation results of the first step and the second step, calculating the numerical value at the point (x, y) after denoising, wherein the calculation formula is as follows:
where F (x, y) is a value at the point (x, y) after the denoising process.
3. The method for enhancing an AR prompt facility based on big data analysis according to claim 1, wherein collecting operation data of each worker on a production line in real time comprises:
A camera is arranged on each worker station;
the camera is controlled in real time to adjust the focal length according to the distance between the moving position of the worker and the camera, so as to obtain the operation action video data of the worker with definition meeting the image recognition requirement;
performing frame processing on the operation motion video data to obtain a plurality of frame image blocks corresponding to the operation motion video data;
and carrying out noise reduction processing on the plurality of frame image blocks and then sending the frame image blocks to the gesture recognition model.
4. The method for enhancing an AR prompt function based on big data analysis according to claim 1, wherein the identifying of the operation actions of the worker by the gesture recognition model in real time, obtaining the operation error information of the worker, comprises:
after receiving a plurality of frame image blocks, the gesture recognition model recognizes gesture actions of workers in each frame image block to obtain operation gesture information of the workers;
comparing the operation gesture information of the worker with an operation standard gesture to obtain gesture similarity;
and when the gesture similarity is lower than a preset gesture similarity threshold, marking and extracting the operation gesture information corresponding to the gesture similarity lower than the preset gesture similarity threshold.
5. The method for enhancing an AR prompt function based on big data analysis according to claim 1, wherein the step of sending the operation error information to an AR device worn by a worker for prompt content presentation comprises:
inputting operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold value into an AR prompt content model;
generating prompt information corresponding to operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold through an AR prompt content model;
and transmitting the prompt information to the AR equipment for prompt content display.
6. A system for enhancing an AR prompt function based on big data analysis, the system for enhancing an AR prompt function based on big data analysis comprising:
the model training module is used for collecting sample data, training the AR prompt content model by using the sample data, and obtaining the AR prompt content model after training;
the real-time acquisition module is used for acquiring operation data of each worker on the production line in real time;
the error information acquisition module is used for identifying the operation actions of the workers in real time through the gesture identification model to acquire operation error information of the workers;
And the content display module is used for sending the operation error information to AR equipment worn by a worker for displaying prompt content.
7. The system for enhancing AR prompt facility based on big data analysis of claim 6, wherein the model training module comprises:
the information data acquisition module is used for acquiring information data of operation guidance of a worker in production by using AR equipment in real time through a camera; the information data comprises text data information, image data information, feedback information of workers and record information of worker operation of AR prompt;
the information data preprocessing module is used for carrying out data preprocessing on the information data to obtain preprocessed information data, wherein the data preprocessing comprises data cleaning processing, data denoising processing, format conversion processing and standardization processing;
a sample data set generating module, configured to generate a sample data set using the preprocessed information data;
the content model calling module is used for calling the AR prompt content model from the database;
the training execution module is used for training the AR prompt content model by utilizing the sample data set to obtain a trained AR prompt content model;
The denoising processing for the data by adopting the following algorithm comprises the following steps:
step one: the value at the set point (x, y) is f (x, y), and the values of the adjacent 8 points around the set point are marked as f (x) i ,y i ) I is the number of its peripheral adjacent points, then the average value of the values of its peripheral adjacent points is:
wherein the method comprises the steps ofAn average value of the values of the neighboring points around the same;
step two: calculating an adjustment coefficient according to the values at the points (x, y) and the values of the adjacent points around the points, wherein the calculation formula is as follows:
wherein k (x, y) is an adjustment coefficient;
step three: according to the calculation results of the first step and the second step, calculating the numerical value at the point (x, y) after denoising, wherein the calculation formula is as follows:
where F (x, y) is a value at the point (x, y) after the denoising process.
8. The system for enhancing AR prompt facility based on big data analysis according to claim 6, wherein the real-time acquisition module comprises:
the camera configuration module is used for configuring cameras on the stations of each worker;
the focal length adjusting module is used for controlling the camera to adjust focal length in real time according to the distance between the moving position of the worker and the camera, and acquiring operation action video data of the worker with definition meeting the image recognition requirement;
The frame image acquisition module is used for carrying out frame processing on the operation action video data to obtain a plurality of frame image blocks corresponding to the operation action video data;
the noise reduction processing module is used for carrying out noise reduction processing on the plurality of frame image blocks and then sending the frame image blocks to the gesture recognition model.
9. The system for enhancing AR prompt facility based on big data analysis according to claim 6, wherein the error information acquisition module comprises:
the gesture information acquisition module is used for identifying the gesture actions of the workers in each frame image block after the gesture identification model receives the plurality of frame image blocks, so as to obtain the operation gesture information of the workers;
the gesture similarity acquisition module is used for comparing the operation gesture information of the worker with an operation standard gesture to obtain gesture similarity;
and the marking and information extracting module is used for marking and extracting the operation gesture information corresponding to the gesture similarity lower than the preset gesture similarity threshold when the gesture similarity is lower than the preset gesture similarity threshold.
10. The system for enhancing AR prompt facility based on big data analysis according to claim 6, wherein the content presentation module comprises:
The information input module is used for inputting the operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold value into the AR prompt content model;
the prompt information acquisition module is used for generating prompt information corresponding to the operation gesture information corresponding to the gesture similarity lower than a preset gesture similarity threshold through the AR prompt content model;
and the content display execution module is used for transmitting the prompt information to the AR equipment to display the prompt content.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311743483.XA CN117788763A (en) | 2023-12-18 | 2023-12-18 | Method and system for enhancing AR prompt function based on big data analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311743483.XA CN117788763A (en) | 2023-12-18 | 2023-12-18 | Method and system for enhancing AR prompt function based on big data analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117788763A true CN117788763A (en) | 2024-03-29 |
Family
ID=90382755
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311743483.XA Pending CN117788763A (en) | 2023-12-18 | 2023-12-18 | Method and system for enhancing AR prompt function based on big data analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117788763A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106257356A (en) * | 2016-03-25 | 2016-12-28 | 深圳增强现实技术有限公司 | Intelligence helmet and intelligence donning system |
| CN111325694A (en) * | 2020-02-25 | 2020-06-23 | 深圳市景阳科技股份有限公司 | Image noise removing method and device |
| CN211293956U (en) * | 2019-11-14 | 2020-08-18 | 北京国电通网络技术有限公司 | AI-based identification and alarm system for abnormal agent on construction site |
| CN112241712A (en) * | 2020-10-22 | 2021-01-19 | 山东省地质矿产勘查开发局第一地质大队 | Mineral resource acquisition and monitoring system |
| KR20220028654A (en) * | 2020-08-31 | 2022-03-08 | (주)프로맥스 | Apparatus and method for providing taekwondo movement coaching service using mirror dispaly |
| CN116778579A (en) * | 2023-06-15 | 2023-09-19 | 中国工商银行股份有限公司 | Multi-person gesture recognition method and device, storage medium and electronic equipment |
-
2023
- 2023-12-18 CN CN202311743483.XA patent/CN117788763A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106257356A (en) * | 2016-03-25 | 2016-12-28 | 深圳增强现实技术有限公司 | Intelligence helmet and intelligence donning system |
| CN211293956U (en) * | 2019-11-14 | 2020-08-18 | 北京国电通网络技术有限公司 | AI-based identification and alarm system for abnormal agent on construction site |
| CN111325694A (en) * | 2020-02-25 | 2020-06-23 | 深圳市景阳科技股份有限公司 | Image noise removing method and device |
| KR20220028654A (en) * | 2020-08-31 | 2022-03-08 | (주)프로맥스 | Apparatus and method for providing taekwondo movement coaching service using mirror dispaly |
| CN112241712A (en) * | 2020-10-22 | 2021-01-19 | 山东省地质矿产勘查开发局第一地质大队 | Mineral resource acquisition and monitoring system |
| CN116778579A (en) * | 2023-06-15 | 2023-09-19 | 中国工商银行股份有限公司 | Multi-person gesture recognition method and device, storage medium and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108960067B (en) | Real-time train driver action recognition system and method based on deep learning | |
| CN108596148B (en) | System and method for analyzing labor state of construction worker based on computer vision | |
| CN112396658B (en) | Indoor personnel positioning method and system based on video | |
| CN110610127B (en) | Face recognition method and device, storage medium and electronic equipment | |
| CN110738135A (en) | worker work step specification visual identification judgment and guidance method and system | |
| CN112016409A (en) | Deep learning-based process step specification visual identification determination method and system | |
| CN111289538A (en) | PCB element detection system and detection method based on machine vision | |
| CN111507261B (en) | Visual target positioning-based process operation quality monitoring method | |
| CN108986137B (en) | Human body tracking method, device and equipment | |
| JP2021114700A (en) | Work support system and work support method | |
| CN113936340B (en) | AI model training method and device based on training data acquisition | |
| CN113807240A (en) | Intelligent transformer substation personnel dressing monitoring method based on uncooperative face recognition | |
| US20210295053A1 (en) | Observed-object recognition system and method | |
| TW202201275A (en) | Device and method for scoring hand work motion and storage medium | |
| CN116259002A (en) | Human body dangerous behavior analysis method based on video | |
| CN115331002A (en) | Method for realizing remote processing of heating power station fault based on AR glasses | |
| CN114187561A (en) | Abnormal behavior identification method and device, terminal equipment and storage medium | |
| CN113752266B (en) | Human-computer cooperation method, system and medium based on cooperative driving and controlling integrated robot | |
| CN114584836B (en) | Method, device, system and medium for detecting using behavior of electronic product | |
| CN111199172A (en) | Terminal screen recording-based processing method and device and storage medium | |
| CN117788763A (en) | Method and system for enhancing AR prompt function based on big data analysis | |
| CN117252353A (en) | Shield construction management platform and management method | |
| CN116229236A (en) | Bacillus tuberculosis detection method based on improved YOLO v5 model | |
| CN111831135A (en) | Mouse cursor image detection comparison and feedback state judgment method | |
| CN115804579A (en) | Process control system for standardized blood pressure measurement and implementation method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |