CN116233354A - Production equipment monitoring processing method, device and system and AR glasses - Google Patents
Production equipment monitoring processing method, device and system and AR glasses Download PDFInfo
- Publication number
- CN116233354A CN116233354A CN202111467296.4A CN202111467296A CN116233354A CN 116233354 A CN116233354 A CN 116233354A CN 202111467296 A CN202111467296 A CN 202111467296A CN 116233354 A CN116233354 A CN 116233354A
- Authority
- CN
- China
- Prior art keywords
- production equipment
- production
- monitoring
- monitoring result
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000004519 manufacturing process Methods 0.000 title claims abstract description 385
- 238000012544 monitoring process Methods 0.000 title claims abstract description 228
- 239000011521 glass Substances 0.000 title claims abstract description 92
- 238000003672 processing method Methods 0.000 title claims abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 46
- 238000009877 rendering Methods 0.000 claims abstract description 20
- 238000012545 processing Methods 0.000 claims description 50
- 238000007689 inspection Methods 0.000 claims description 45
- 238000004891 communication Methods 0.000 claims description 22
- 230000008569 process Effects 0.000 claims description 20
- 238000003860 storage Methods 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 21
- 230000006870 function Effects 0.000 description 21
- 238000005516 engineering process Methods 0.000 description 8
- 238000012423 maintenance Methods 0.000 description 8
- 239000004576 sand Substances 0.000 description 6
- 238000004590 computer program Methods 0.000 description 5
- 238000003032 molecular docking Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 3
- 238000009776 industrial production Methods 0.000 description 3
- 230000003993 interaction Effects 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 239000002994 raw material Substances 0.000 description 2
- 229910000831 Steel Inorganic materials 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000004043 dyeing Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000802 evaporation-induced self-assembly Methods 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- -1 mobile phones Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000010959 steel Substances 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44012—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- General Factory Administration (AREA)
Abstract
The application provides a production equipment monitoring processing method, device and system and AR glasses. The method comprises the following steps: according to a monitoring request of production equipment input by a user, acquiring a monitoring result of the production equipment, wherein the monitoring result of the production equipment is used for indicating whether the production equipment fails or not; rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result; and displaying a virtual picture of the monitoring result.
Description
Technical Field
The application relates to industrial internet technology, in particular to a production equipment monitoring processing method, device and system and AR glasses.
Background
The operation state of the production equipment is monitored, so that the problems encountered in the production process can be found and solved in time, and the production efficiency is ensured.
At present, a large number of sensors are arranged on production equipment, operation data of the production equipment are collected through the sensors, and then a production monitoring system analyzes according to the operation data of the production equipment collected by the sensors to determine whether the production equipment is in normal operation. And under the condition that the production equipment is determined to be faulty, alarm information is sent to terminal equipment of operation and maintenance personnel, the operation and maintenance personnel check operation data of the production equipment through the terminal equipment, and the fault cause is determined by combining the detection of the production equipment on site.
However, when the operation and maintenance personnel in the prior art check the production equipment on site, if the data are frequently checked on the terminal equipment, the operation is inconvenient, and the fault processing efficiency is low.
Disclosure of Invention
The application provides a production equipment monitoring processing method, device and system and AR glasses, which are used for solving the problems that in the prior art, when operation and maintenance personnel check production equipment on site, if data are frequently checked on terminal equipment, operation is inconvenient and failure processing efficiency is low.
In a first aspect, the present application provides a production facility monitoring processing method, including: according to a monitoring request of production equipment input by a user, obtaining a monitoring result of the production equipment, wherein the monitoring result of the production equipment is used for indicating whether the production equipment has faults or not; rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result; and displaying the virtual picture of the monitoring result.
In a second aspect, the present application provides a production monitoring and processing device, comprising: the system comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring a monitoring result of production equipment according to a monitoring request of the production equipment input by a user, and the monitoring result of the production equipment is used for indicating whether the production equipment has faults or not; the rendering module is used for rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result; and the display module is used for displaying the virtual picture of the monitoring result.
In a third aspect, the present application provides a production facility monitoring processing system, comprising: AR glasses and a production monitoring system; the AR glasses for performing the method of the first aspect; the production monitoring system is used for receiving a monitoring request of production equipment and sending a monitoring result of the production equipment to the AR glasses.
In a fourth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively coupled to the processor;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement the method as described in the first aspect.
In a fifth aspect, the present application provides a computer-readable storage medium having stored therein computer-executable instructions for performing the method according to the first aspect when executed by a processor.
According to the production equipment monitoring processing method, the device and the system and the AR glasses, the monitoring result of the production equipment is obtained according to the monitoring request of the production equipment input by a user, and the monitoring result of the production equipment is used for indicating whether the production equipment has faults or not; rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result; and displaying a virtual picture of the monitoring result. Because the monitoring result of the production equipment is presented in a virtual picture mode, a user can free both hands, and the problem of low equipment monitoring efficiency caused by frequent switching operation between the terminal equipment and the production equipment is avoided.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
Fig. 1 is an application scenario diagram provided in an embodiment of the present application;
FIG. 2 is a flowchart of a method for monitoring and processing production equipment according to an embodiment of the present application;
FIG. 3 is a logic block diagram of AR glasses provided in an embodiment of the present application;
fig. 4 is another application scenario diagram provided in an embodiment of the present application;
fig. 5 is another application scenario diagram provided in an embodiment of the present application;
fig. 6 is a further application scenario diagram provided in an embodiment of the present application;
FIG. 7 is a schematic diagram of a network architecture of a production facility monitoring and processing scheme according to an embodiment of the present application;
fig. 8 is an application scenario diagram of a production facility monitoring processing scheme provided in an embodiment of the present application;
FIG. 9 is a functional block diagram of a production facility monitoring process scheme provided in an embodiment of the present application;
fig. 10 is a schematic structural diagram of a production facility monitoring processing apparatus according to an embodiment of the present application;
fig. 11 is a schematic structural diagram of AR glasses according to an embodiment of the present application.
Specific embodiments thereof have been shown by way of example in the drawings and will herein be described in more detail. These drawings and the written description are not intended to limit the scope of the inventive concepts in any way, but to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments.
Detailed Description
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples are not representative of all implementations consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present application as detailed in the accompanying claims.
Industrial production is mainly a process of processing or assembling natural resources and raw materials. Industrial production includes energy industry, steel industry, mechanical industry, etc. Industrial production is not separated from production equipment, and normal operation of the production equipment is critical to production output. In the related art, when judging whether the production equipment is operating normally, equipment maintenance personnel need to check the production equipment on site, and in the process of checking on site, some information may need to be checked through terminal equipment to determine whether the fault occurs, the reason of the fault, and the like. In this process, the equipment maintainer may need to frequently view the information, which requires the equipment maintainer to constantly switch between the production equipment and the terminal equipment. The equipment maintenance personnel can not liberate both hands, and the time spent is long, influences the monitoring efficiency of production equipment.
Aiming at the technical problems, the application provides the following technical conception: augmented reality (Augmented Reality, AR) refers to real-time sensing and calculation of a real environment by a computing device, and three-dimensional contents and information such as characters, pictures and videos are superimposed in the real world, so that the augmented reality has the characteristics of virtual reality fusion, real-time interaction, three-dimensional registration and the like. Therefore, the information required to be checked by the equipment maintenance personnel can be displayed in an AR picture mode based on the AR technology, so that the equipment maintenance personnel can be liberated from both hands, and the monitoring efficiency of the production equipment is improved.
The following describes the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
Fig. 1 is an application scenario diagram provided in an embodiment of the present application. As shown in fig. 1, the application scenario includes: a production facility 11, AR glasses 12, and a production monitoring system 13; the production facility 11 is in communication with a production monitoring system 13, and the AR glasses 12 are in communication with the production monitoring system 13.
The production facility 11 refers to labor data or things necessary for a production worker to directly change the properties, performance, morphology, or enhanced appearance value of raw materials during the production process. Such as blast furnaces, machine tools, reactors, dyeing machines, paper machines, etc. The production facility 11 is provided with sensors, such as temperature sensors, pressure sensors, etc.
The AR glasses 12 may be head-mounted AR glasses. The AR glasses are further provided with sensors, which can collect operation data of the production facility 11. The sensor may be an image sensor or other sensor.
The production monitoring system 13 is used for monitoring the production facility 11. The production monitoring system 13 may employ an existing production monitoring system.
The user wears AR glasses and can monitor and maintain equipment for production equipment in a production workshop. Based on the application scenario shown in fig. 1, how the user wears AR glasses to monitor and maintain the production equipment in the production shop is described in detail as follows:
fig. 2 is a flowchart of a production facility monitoring processing method according to an embodiment of the present application. As shown in fig. 2, the production equipment monitoring processing method comprises the following steps:
s201, according to a monitoring request of production equipment input by a user, a monitoring result of the production equipment is obtained, and the monitoring result of the production equipment is used for indicating whether the production equipment has faults or not.
The main implementation body of the method of the embodiment may be AR glasses, and fig. 3 is a logic block diagram of AR glasses provided in the embodiment of the present application. As shown in fig. 3, the logical block diagram of the AR glasses includes: a voice acquisition unit 31, an image acquisition unit 32, a processing unit 33, and a display unit 34; the voice capturing unit 31 may be a microphone and the image capturing unit may be a camera.
In some embodiments, the user may input a voice command to the AR glasses in a voice manner to request acquisition of the monitoring result of the production equipment. The voice acquisition unit 31 acquires a voice command input by a user and sends the voice command to the processing unit 33, and the processing unit 33 sends an acquisition request of the monitoring result of the production equipment to the production monitoring system according to the voice command and receives the monitoring result of the production equipment returned by the production monitoring system so as to acquire the monitoring result of the production equipment from the production monitoring system.
In other embodiments, the user may also input a gesture command to the AR glasses by means of a gesture, so as to request to obtain a monitoring result of the production equipment. The image acquisition unit 32 acquires the gesture instruction input by the user and sends the gesture instruction to the processing unit 33, and the processing unit 33 sends an acquisition request of the monitoring result of the production equipment to the production monitoring system according to the gesture instruction and receives the monitoring result of the production equipment returned by the production monitoring system, so as to acquire the monitoring result of the production equipment from the production monitoring system.
In still other embodiments, the processing unit 33 may also be communicatively coupled to a terminal device. The terminal device may provide a graphical user interface, on which a control may be displayed, and by triggering the control, the user may send an acquisition request of the monitoring result of the production device to the processing unit 33, and the processing unit further forwards the acquisition request of the monitoring result of the production device to the production monitoring system, and receives the monitoring result of the production device returned by the production monitoring system, so as to obtain the monitoring result of the production device from the production monitoring system. As an optional embodiment, the user may further send a request for obtaining the monitoring result of the production device to the production monitoring system by triggering the control, and the production monitoring system further sends the monitoring result of the production device to the AR glasses, so as to obtain the monitoring result of the production device from the production monitoring system.
In this embodiment, reference may be made to the description of the prior art as to how the production monitoring system obtains the monitoring results of the production equipment.
Specifically, step S201 obtains a monitoring result of the production device according to a monitoring request of the production device input by a user, including:
And a1, acquiring an image of a target position of the production equipment through a sensor arranged on the AR glasses according to a monitoring request of the production equipment input by a user.
As described above, after the user triggers the monitoring request of the production equipment, the processing unit controls the sensor on the AR glasses to collect the image of the target position of the production equipment, and sends the image to the production monitoring system.
And a2, acquiring a monitoring result of the production equipment according to the image of the target position of the production equipment.
Wherein step a2 may comprise at least two alternative embodiments as follows:
in a first alternative embodiment, step a2 comprises: a21, sending an image of the target position of the production equipment to the production monitoring system; a22, receiving a monitoring result returned by the production monitoring system based on the image of the target position of the production equipment. In this embodiment, the AR glasses are mainly used to collect images of the target position of the production equipment, and the production monitoring system determines the monitoring result of the production equipment according to the images of the target position collected by the AR glasses.
In a second alternative embodiment, step a2 comprises: a23, acquiring a historical image of a target position of the production equipment from the production monitoring system; a24, determining a monitoring result of the production equipment according to the image of the target position of the production equipment and the historical image of the target position of the production equipment. In this embodiment, the AR glasses are mainly used to collect images of target positions of the production equipment, and to determine monitoring results of the production equipment based on the images of the target positions collected by the AR glasses. The difference from the first embodiment is that in the first embodiment, the AR glasses are not provided with a monitoring function, while in the second embodiment, the AR glasses are provided with a monitoring function, that is, a monitoring policy for monitoring the production equipment by the production monitoring system is preset in the AR glasses, and after a historical image of the target position of the production equipment is acquired from the production monitoring system, the monitoring result of the production equipment can be determined by self according to the image of the target position of the production equipment, the historical image of the target position of the production equipment and the monitoring policy.
In some optional embodiments, a plurality of sensors, such as a pressure sensor, a temperature sensor, and the like, may be further disposed on the production device, and step S201 may further include:
and a3, acquiring an image of a target position of the production equipment through a sensor arranged on the AR glasses according to a monitoring request of the production equipment input by a user, and acquiring sensor data acquired by a sensor arranged on the production equipment.
And a4, acquiring a monitoring result of the production equipment according to the image of the target position of the production equipment and sensor data acquired by a sensor arranged on the production equipment.
Specifically, step a4 includes:
a41, acquiring historical images and historical sensor data of the target position of the production equipment from the production monitoring system.
and a42, determining a monitoring result of the production equipment according to the image of the target position of the production equipment, the historical image of the target position of the production equipment, and the sensor data and the historical sensor data acquired by the sensor installed on the production equipment.
In this embodiment, the AR glasses are provided with a monitoring function, that is, a monitoring policy for monitoring the production equipment by the production monitoring system is preset in the AR glasses, and after a history image of a target position of the production equipment and history sensor data of the production equipment are obtained from the production monitoring system, a monitoring result of the production equipment can be determined by itself according to the image of the target position of the production equipment, the history image of the target position of the production equipment, the sensor data collected by a sensor mounted on the production equipment, the history sensor data and the monitoring policy.
The monitoring policy may refer to the description of the existing monitoring policy, which is not described in this embodiment.
And S202, rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result.
Specifically, the processing unit 33 may further perform rendering according to the monitoring result of the production device, obtain a virtual image of the monitoring result, and send the virtual image to the display unit 34.
Wherein, rendering is performed according to the monitoring result of the production equipment, and a virtual picture of the monitoring result is obtained, which can be referred to the related description about rendering the virtual picture in the AR technology.
S203, displaying a virtual picture of the monitoring result.
In the present embodiment, the display unit 34 displays a virtual screen of the monitoring result in front of the user wearing the AR glasses. Optionally, the virtual image of the monitoring result may be displayed around the production device, so that the user can check the production device on site according to the virtual image of the monitoring result.
According to the embodiment, the monitoring result of the production equipment is obtained according to the monitoring request of the production equipment input by the user, and the monitoring result of the production equipment is used for indicating whether the production equipment has faults or not; rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result; and displaying a virtual picture of the monitoring result. Because the monitoring result of the production equipment is presented in a virtual picture mode, a user can free both hands, and the problem of low equipment monitoring efficiency caused by frequent switching operation between the terminal equipment and the production equipment is avoided.
On the basis of the embodiment, if the monitoring result of the production equipment indicates that the production equipment fails, failure processing instruction information of the production equipment can be obtained; rendering according to the fault processing guide information of the production equipment to obtain a virtual picture of the fault processing guide information of the production equipment; and displaying a virtual picture of the fault handling guide information.
Based on the application scenario shown in fig. 1, fig. 4 is another application scenario diagram provided in the embodiment of the present application. As shown in fig. 4, the application scenario further includes: a database server 41; the database server 41 is provided with a database in which fault handling instruction information of different fault types of different production apparatuses is stored. Based on the application scenario of fig. 4, if the monitoring result of the production device indicates that the production device fails, the AR glasses may send a request for failure processing guiding information of the production device to the database server, where the request includes an identifier of the production device and a failure type, and the database server returns the failure processing guiding information according to the request. The AR glasses render according to the fault processing guide information to obtain a virtual picture of the fault processing guide information, and display the virtual picture of the fault processing guide information. The user can input a request of the fault handling guide information of the production equipment to the AR glasses in a voice instruction or gesture instruction mode, and the AR glasses can send the request of the fault handling guide information of the production equipment to the database server.
On the basis of the above embodiment, fig. 5 is a further application scenario diagram provided in the embodiment of the present application. As shown in fig. 5, the application scenario includes: a terminal device 51 and AR glasses 52; the AR glasses 52 establish transmission of video data with the terminal device 51 through the 5G gateway, the base station, the cloud system, the 5G core network, the internet. The AR glasses communicate with the terminal device 51 through the 5G communication method.
If the monitoring result of the production equipment indicates that the production equipment fails, the user can also initiate a video request through the AR glasses, for example, the video request is input to the AR glasses in a voice command or gesture command mode, the AR glasses can respond to the video request input by the user, establish video communication with the remote guidance platform, display a virtual video communication picture, and the video communication is used for the remote guidance platform to conduct remote guidance on the failure of the production equipment. The user can watch the virtual video picture through the AR glasses and perform fault processing according to fault guiding information in the virtual video picture.
Optionally, the video communication screen may include an image of the production device, and fault handling guide information of the production device, where the fault handling guide information includes comment information and/or marking information added by a user; rendering the video communication picture to obtain a virtual video picture, including: and superposing the fault processing guide information of the production equipment in an image of the production equipment to obtain a virtual video picture. Through the annotation information and/or the marking information, the user can understand the fault guiding information, key contents in the fault guiding information can be found quickly, and the time for reading the fault guiding information is saved.
On the basis of the embodiment, the user can also wear the AR glasses to carry out inspection in the production workshop, and in the inspection process, the user can input an inspection request of the production equipment through the AR glasses, so that the AR glasses can acquire inspection operation guide information of the production equipment according to the inspection request of the production equipment, render the inspection operation guide information of the production equipment, obtain a virtual picture of the inspection operation guide information and display the virtual picture.
With continued reference to fig. 4, the database may also store inspection operation guidance information for each production facility in the production facility. The AR glasses may send a request for the inspection operation guiding information of the production equipment to the database server, where the request includes the identifier of the production equipment and the request content, and the request content is the inspection operation guiding information. The database server returns inspection operation instruction information according to the request. Fig. 6 is a further application scenario diagram provided in an embodiment of the present application. As shown in fig. 6, the AR glasses render according to the patrol operation guide information received from the database server 41, obtain a virtual screen of the patrol operation guide information, and display the virtual screen of the patrol operation guide information. After the virtual picture of the inspection guiding information is displayed, a user can check the virtual picture of the inspection guiding information through the AR glasses, so that the inspection guiding information is obtained, and the problem that the inspection efficiency is low due to the fact that the virtual picture needs to be checked through the terminal equipment and is switched back and forth between the production equipment and the terminal equipment is avoided.
In the process of inspecting the production equipment according to the inspection guiding information, the user can acquire pictures or videos in the inspection process through the cameras on the AR glasses and upload the pictures or videos to the server for storage. Specifically, a user can trigger the shooting function of the AR glasses in a voice instruction or gesture instruction mode, and then the camera can acquire the inspection video of the production equipment in the process of inspecting the production equipment by the user; and sending the inspection video of the production equipment to a server. The server stores the inspection video of the production equipment.
On the basis of the embodiment, scene data of a production workshop where the production equipment is located can be collected in the process of inspecting the production equipment by a user; and sending the scene data of the production plant to a server, wherein the scene data is used for constructing a three-dimensional model of the production plant.
In connection with the production facility monitoring process scheme in the various embodiments described above, this scheme is illustrated below by way of a specific example.
Fig. 7 is a schematic diagram of a network architecture of a production facility monitoring processing scheme according to an embodiment of the present application.
As shown in fig. 7, the network architecture includes: AR glasses, 5G portable WIFI (wireless network communication technology), 5G base station, user plane function (User Port Function, UPF), edge computing technology (Multi-Acess Edge Computing, MEC) platform, firewall, customer premise equipment (Customer Premise Equipment, CPE) for fixed network, intranet for remote guiding user side and terminal equipment for remote guiding user connected in sequence.
Based on the network architecture shown in fig. 7, if the user on the AR glasses side inserts the internet of things card with the preconfigured data network name (Data Network Name, DNN) into the 5G portable WIFI, the data of the AR glasses will be transmitted based on the network architecture shown in fig. 7. Taking remote video guidance as an example, video pictures on the user side of the AR glasses collected by the AR glasses are sequentially transmitted to terminal equipment of the remote guidance user through 5G carry-on WIFI, 5G base stations, user plane functions (User Port Function, UPF), edge computing technology (Multi-Acess Edge Computing, MEC) platforms, firewalls, customer premise equipment (Customer Premise Equipment, CPE) of fixed networks and an intranet of the remote guidance user side. Similarly, the terminal device of the remote guiding user may collect video pictures of the remote guiding user side, and sequentially transmit the video pictures to the AR glasses through the customer premise equipment (Customer Premise Equipment, CPE), firewall, edge computing technology (Multi-Acess Edge Computing, MEC) platform, user plane function (User Port Function, UPF), 5G base station and 5G portable WIFI of the customer intranet and the fixed network.
Fig. 8 is an application scenario diagram of a production facility monitoring processing scheme provided in an embodiment of the present application.
As shown in fig. 8, the application scenario includes AR glasses, 5G portable WIFI, 5G base station, user plane function (User Port Function, UPF), edge computing technology (Multi-Acess Edge Computing, MEC) platform, firewall, customer premise equipment (Customer Premise Equipment, CPE) of fixed network, customer intranet and application server connected in sequence, the application server is connected with database server; the application server provides an application program for the AR glasses side user, and the AR glasses side user can access resources in the database server through the application program. For example, the video, the picture, and the like in the inspection process collected by the AR glasses side user may be uploaded to the database server for storage, and the AR glasses side user may log in the production equipment monitoring processing system to view the resources such as the video, the picture, and the like in the inspection process.
Specifically, the application program provided by the application server has the following functions:
fig. 9 is a functional framework diagram of a production facility monitoring processing scheme according to an embodiment of the present application. As shown in fig. 9, the production equipment monitoring processing system comprises an AR intelligent patrol module, an AR remote guidance module, a digital sand table module and a background management module (not shown in the figure);
The AR intelligent inspection module comprises an intelligent inspection unit, an inspection information display unit, an equipment identification unit, an equipment state monitoring unit, a fault alarm unit, a system docking function and the like. Wherein, the definite inspection refers to the inspection of production equipment at a designated period.
1) The account login unit, the AR glasses side user can log in the production equipment monitoring processing system by inputting an account and a password on the terminal equipment, and after successful login, the functions of the following units can be triggered:
2) The intelligent checking unit is used for performing intelligent checking, namely providing operation standards of checking contents for users, wherein the operation standards of each checking link can be displayed in a mode of graphics, characters, videos and the like, and various checking steps or processes can be recorded in a plurality of modes of photographing, videos and the like. With continued reference to fig. 8, for example, if the user triggers the intelligent checking function by means of a voice command or a gesture control command, the AR glasses send a checking request of the target production device to the database server, where the checking request includes an identifier of the target production device, and the database server obtains, according to the identifier of the target production device, a checking operation standard of the target production device from the pre-stored checking operation standards of the plurality of production devices, and returns the checking operation standard to the AR glasses, where the AR glasses render the checking operation standard of the target production device, so as to obtain a virtual picture of the checking operation standard of the target production device, and display the checking operation standard of the target production device for the user in a virtual reality manner.
3) And the inspection information display unit is used for displaying inspection information, namely, informationizing equipment and returning the equipment information in real time. The equipment information is visualized, and a user can conduct data comparison in real time to quickly remove faults. For example, please continue to refer to fig. 8, the user at the AR glasses side scans the two-dimensional code on the target production device through the camera on the AR glasses, and may trigger the AR glasses to send an acquisition request of the basic information of the target production device to the database server, where the acquisition request includes the identifier of the target production device, and the database server acquires the basic information of the target production device from the device basic information of the multiple production devices stored in advance according to the identifier of the target production device, and returns the basic information to the AR glasses, where the AR glasses render the basic information of the target production device to obtain a virtual picture of the basic information of the target production device, and displays the basic information of the target production device for the user in a virtual reality manner. If the database server returns a query result that the basic information of the target production equipment does not exist, or the user considers that the basic information of the target production equipment returned by the database server is incomplete and needs to be supplemented and increased, the user can acquire the basic information of the equipment through a sensor on the AR glasses and send the basic information to the database server for storage.
4) And the equipment identification unit is used for carrying out equipment identification, namely automatically identifying the production equipment and acquiring the operation state data of the identified production equipment from the database according to the identified production equipment. For example, if the AR glasses automatically identify that the production device is the B-platform database server, the AR glasses may obtain the CPU usage, the memory usage, the storage status, the network traffic, and other operation status data of the B-platform database server from the database, and display the operation status data by means of an AR screen.
5) The equipment state monitoring unit is used for performing equipment state monitoring, namely, docking with the production monitoring system to obtain production monitoring data, and performing real-time monitoring on the running state of the equipment to comprehensively monitor the overall running condition of the production control line. A specific implementation may be seen in the description of the embodiment shown in fig. 2.
6) And the fault alarm unit is used for carrying out fault alarm, namely controlling the sign of the indicator lamp to be highlighted when the fault of the production equipment is determined, and simultaneously displaying the fault reason and predicting the influence range of the fault. The user may obtain an electronic manual or video type fault handling guideline from the database server by means of voice instructions or gesture instructions. The specific implementation process of the fault handling guide of electronic manual or video type obtained by the user from the database server through voice instruction or gesture instruction can be referred to the description of the embodiment shown in fig. 4.
7) The system docking unit is used for performing system docking, namely accessing an existing production monitoring system, acquiring monitoring data and an alarm threshold value from the production monitoring system, and providing the monitoring data and the alarm threshold value for the AR glasses to perform fault judgment.
The AR remote guidance module comprises an intelligent remote guidance unit, an annotating unit and a system support unit;
the intelligent remote guiding unit is used for providing a remote guiding function, namely providing a user with a remote guiding mode for guiding the on-site user to operate and process, so that the problem of faults is rapidly and accurately processed. Thereby reducing downtime and repeated door-on operations and improving safety.
The intelligent remote guiding unit is used for providing a remote guiding function for a user, the user can conduct remote guiding video through an active calling mode or a passive calling receiving mode, video communication is established with a remote user side when faults cannot be solved, after connection is established with the remote user side, a first-person video picture (a first visual angle picture) of the user can be synchronized to the remote user side through AR glasses, the remote user can conduct annotation and/or marking on the first-person video picture through terminal equipment, accurate instructions can be conducted on a field user, expertise is shared remotely, and intelligent remote guiding is achieved. In addition, the voice and video call in the remote guidance process can be automatically recorded and automatically uploaded to the server for storage. On the basis that the remote user can perform voice and video interaction with the user through the terminal equipment, video pictures can be edited, marked, intercepted, sent, marked and the like.
Wherein, reference may be made to the description of the remote video guidance in the above embodiments.
And the annotating unit is used for carrying out screenshot preservation on the current picture or the video picture, amplifying the screenshot content and annotating and describing the image. The on-site user can screen shot and annotate the on-site image and push the on-site image to the terminal equipment of the remote user, and the remote user can annotate the static image fed back by the on-site user. The remote user can connect the AR glasses in real time on a terminal device such as a mobile phone or a personal computer (Personal Computer, PC) to view the live view. And the remote user marks the circles on the video picture through the terminal equipment, so that the on-site user can see that virtual circles information is superimposed on the real production equipment.
The system support unit is used for supporting access of various remote devices and operating systems, such as AR glasses, mobile phones, tablets and computers, and supporting an Android operating system and an IOS operating system. And supports multiple clients to establish connections. The AR glasses device supports 5G connection, WIFI connection. A maximum of 100 user connections are supported, and 10 concurrent fault scenarios are supported.
The digital sand table module is used for carrying out simulation data acquisition on a production workshop in the inspection process, carrying out three-dimensional visual modeling, and carrying out digitization and three-dimensional visualization on the production line workshop by importing a 3D (three-dimensional) digital model manufactured by acquisition to comprehensively display the condition of the production line. Meanwhile, the existing production monitoring system is docked, production data are obtained and combined with a papermaking production line, so that the production line data are displayed in a visual mode. I.e. the production data is presented by means of labels on a digital sand table.
In addition, the user can carry out interactive control on the digital sand table in a mode of identifying a touch interaction module (Maker) or finger touch, so that three-dimensional visual roaming in a virtual scene of a production line workshop is realized, and the scene structure of the production control line is checked in all directions.
In addition, the working principle of the production line and the structure principle of key equipment can be displayed in a plane reel mode, the structure and the principle of the production line are abstracted, and the viewing is convenient. The digital sand table can be matched with a Light-Emitting Diode (LED) large screen to carry out digital display, and the production line production data is displayed in all directions.
The digital sand table module can also display the production video of the corresponding production line according to the production links of the corresponding production line, and fully display the working flow of the production line.
The background management module comprises functions of account management, a knowledge base, data statistics and the like, and is mainly used for overall background management, inspection information record checking and related event record in the inspection process.
On the basis of the above method embodiment, fig. 10 is a schematic structural diagram of a production equipment monitoring processing device according to an embodiment of the present application. As shown in fig. 10, the apparatus includes: an acquisition module 101, a rendering module 102, and a display module 103;
the obtaining module 101 is configured to obtain a monitoring result of the production equipment according to a monitoring request of the production equipment input by a user, where the monitoring result of the production equipment is used to indicate whether the production equipment has a fault;
the rendering module 102 is used for rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result;
and the display module 103 is used for displaying the virtual picture of the monitoring result.
In some embodiments, the obtaining module 101 obtains a monitoring result of the production equipment according to a monitoring request of the production equipment input by a user, and specifically includes:
acquiring an image of a target position of production equipment through a sensor arranged on the AR glasses according to a monitoring request of the production equipment input by a user;
And acquiring a monitoring result of the production equipment according to the image of the target position of the production equipment.
In some embodiments, the obtaining module 101 obtains a monitoring result of the production apparatus according to an image of a target location of the production apparatus, including:
transmitting an image of a target location of the production facility to a production monitoring system;
and receiving a monitoring result returned by the production monitoring system based on the image of the target position of the production equipment.
In some embodiments, the obtaining module 101 obtains a monitoring result of the production apparatus according to an image of a target location of the production apparatus, including:
acquiring a historical image of a target position of the production equipment from the production monitoring system;
and determining a monitoring result of the production equipment according to the image of the target position of the production equipment and the historical image of the target position of the production equipment.
In some embodiments, the obtaining module 101 is further configured to obtain fault handling instruction information of the production equipment if the monitoring result of the production equipment indicates that the production equipment is faulty; and the display module 103 is used for displaying a virtual picture of the fault processing guide information.
In some embodiments, the apparatus further comprises: the video communication module 104 is configured to establish video communication with a remote guidance platform in response to a video request input by a user, and display a virtual video communication screen, where the video communication is used for the remote guidance platform to remotely guide a fault of the production equipment.
In some embodiments, the video communication screen comprises an image of the production equipment, fault handling guide information of the production equipment, the fault handling guide information comprising annotation information and/or marking information added by a user; the rendering module 102 renders the video communication picture to obtain a virtual video picture, which specifically includes: and superposing fault processing guide information of the production equipment in an image of the production equipment to obtain the virtual video picture.
In some embodiments, the apparatus further comprises: a receiving module 105, configured to receive a patrol request of the production device, where the patrol request is input by a user; the obtaining module 101 is further configured to obtain inspection operation instruction information of the production equipment according to an inspection request of the production equipment; the display module 103 is further configured to display a virtual screen of the inspection guidance information.
In some embodiments, the apparatus further comprises: an acquisition module 106 and a transmission module 107; the receiving module 105 is further configured to receive a shooting request input by a user; the acquisition module 106 is configured to acquire an inspection video of the production equipment during the inspection process of the user on the production equipment; and the sending module 107 is used for sending the inspection video of the production equipment to a server.
In some embodiments, the collection module 106 is further configured to collect, during a user's inspection of the production facility, scene data of a production plant where the production facility is located; the sending module 107 is further configured to send, to a server, scene data of the production plant, where the scene data is used to construct a three-dimensional model of the production plant.
The production equipment monitoring processing device provided by the embodiment of the application can be used for executing the technical scheme of the production equipment monitoring processing method in the embodiment, and the implementation principle and the technical effect are similar, and are not repeated here.
It should be noted that, it should be understood that the division of the modules of the above apparatus is merely a division of a logic function, and may be fully or partially integrated into a physical entity or may be physically separated. And these modules may all be implemented in software in the form of calls by the processing element; or can be realized in hardware; the method can also be realized in a form of calling software by a processing element, and the method can be realized in a form of hardware by a part of modules. For example, the acquisition module 106 may be a processing element that is set up separately, may be implemented in a chip of the above-described apparatus, or may be stored in a memory of the above-described apparatus in the form of program codes, and the functions of the above-described acquisition module 106 may be called and executed by a processing element of the above-described apparatus. The implementation of the other modules is similar. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element here may be an integrated circuit with signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.
Fig. 11 is a schematic structural diagram of AR glasses according to an embodiment of the present application. As shown in fig. 11, the AR glasses may include: a transceiver 111, a processor 112, a memory 113 and a display 114.
The processor 112 executes the computer-executable instructions stored in the memory, causing the processor 112 to perform the aspects of the embodiments described above. The processor 112 may be a general-purpose processor including a central processing unit CPU, a network processor (network processor, NP), etc.; but may also be a digital signal processor DSP, an application specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.
The memory 113 is connected to the processor 112 via a system bus and communicates with each other, and the memory 113 is adapted to store computer program instructions.
Transceiver 111 may be used to receive user-entered monitoring requests and to transmit monitoring requests to the production monitoring system.
The display 114 may display a virtual screen of the monitoring result and other virtual screens.
The system bus may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The system bus may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus. The transceiver is used to enable communication between the database access device and other computers (e.g., clients, read-write libraries, and read-only libraries). The memory may include random access memory (random access memory, RAM) and may also include non-volatile memory (non-volatile memory).
The electronic device provided in the embodiment of the present application may be a terminal device in the above embodiment.
The embodiment of the application also provides a chip for running the instruction, and the chip is used for executing the technical scheme of the production equipment monitoring processing method in the embodiment.
The embodiment of the application also provides a computer readable storage medium, wherein the computer readable storage medium stores computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the technical scheme of the production equipment monitoring processing method in the embodiment.
The embodiment of the application also provides a computer program product, which comprises a computer program stored in a computer readable storage medium, wherein at least one processor can read the computer program from the computer readable storage medium, and the technical scheme of the production equipment monitoring processing method in the embodiment can be realized when the at least one processor executes the computer program.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It is to be understood that the present application is not limited to the precise arrangements and instrumentalities shown in the drawings, which have been described above, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the application is limited only by the appended claims.
Claims (14)
1. A production facility monitoring and processing method, comprising:
according to a monitoring request of production equipment input by a user, obtaining a monitoring result of the production equipment, wherein the monitoring result of the production equipment is used for indicating whether the production equipment has faults or not;
rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result;
and displaying the virtual picture of the monitoring result.
2. The method according to claim 1, wherein the obtaining the monitoring result of the production equipment according to the monitoring request of the production equipment input by the user comprises:
acquiring an image of a target position of production equipment through a sensor arranged on the AR glasses according to a monitoring request of the production equipment input by a user;
and acquiring a monitoring result of the production equipment according to the image of the target position of the production equipment.
3. The method according to claim 2, wherein the acquiring the monitoring result of the production equipment according to the image of the target position of the production equipment includes:
Transmitting an image of a target location of the production facility to a production monitoring system;
and receiving a monitoring result returned by the production monitoring system based on the image of the target position of the production equipment.
4. The method according to claim 2, wherein the acquiring the monitoring result of the production equipment according to the image of the target position of the production equipment includes:
acquiring a historical image of a target position of the production equipment from the production monitoring system;
and determining a monitoring result of the production equipment according to the image of the target position of the production equipment and the historical image of the target position of the production equipment.
5. The method according to any one of claims 1 to 4, further comprising, after the obtaining the monitoring result of the production apparatus:
if the monitoring result of the production equipment indicates that the production equipment fails, acquiring failure processing guide information of the production equipment;
and displaying a virtual picture of the fault handling guide information.
6. The method according to any one of claims 1 to 4, further comprising, after the obtaining the monitoring result of the production apparatus:
and responding to a video request input by a user, establishing video communication with a remote guidance platform, and displaying a virtual video communication picture, wherein the video communication is used for the remote guidance platform to remotely guide the fault of the production equipment.
7. The method according to claim 6, wherein the video communication screen comprises an image of a production apparatus, fault handling guide information of the production apparatus, the fault handling guide information comprising annotation information and/or marking information added by a user;
rendering the video communication picture to obtain a virtual video picture, including:
and superposing fault processing guide information of the production equipment in an image of the production equipment to obtain the virtual video picture.
8. A method according to any one of claims 1 to 3, wherein before the monitoring result of the production equipment is obtained according to the monitoring request of the production equipment input by the user, the method further comprises:
receiving a patrol request of the production equipment, which is input by a user;
acquiring inspection operation guiding information of the production equipment according to the inspection request of the production equipment;
and displaying a virtual picture of the inspection guiding information.
9. The method of claim 8, wherein after displaying the virtual view of the inspection guidance information, further comprising:
receiving a shooting request input by a user;
collecting inspection videos of the production equipment in the process of inspecting the production equipment by a user;
And sending the inspection video of the production equipment to a server.
10. The method according to claim 9, wherein the method further comprises:
during the process of inspecting the production equipment by a user, acquiring scene data of a production workshop where the production equipment is located;
and sending the scene data of the production plant to a server, wherein the scene data is used for constructing a three-dimensional model of the production plant.
11. A production facility monitoring processing apparatus, characterized by comprising:
the system comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring a monitoring result of production equipment according to a monitoring request of the production equipment input by a user, and the monitoring result of the production equipment is used for indicating whether the production equipment has faults or not;
the rendering module is used for rendering according to the monitoring result of the production equipment to obtain a virtual picture of the monitoring result;
and the display module is used for displaying the virtual picture of the monitoring result.
12. A production facility monitoring processing system, comprising: AR glasses and a production monitoring system;
AR glasses for performing the method according to any one of claims 1-10;
the production monitoring system is used for receiving a monitoring request of production equipment and sending a monitoring result of the production equipment to the AR glasses.
13. An AR glasses, comprising: a processor, and a transceiver, a memory and a display communicatively coupled to the processor, respectively;
the transceiver is used for acquiring a monitoring result of the production equipment according to a monitoring request of the production equipment input by a user;
the display is used for displaying the virtual picture of the monitoring result;
the memory stores computer-executable instructions;
the processor executes computer-executable instructions stored in the memory to implement the method of any one of claims 1-10.
14. A computer readable storage medium having stored therein computer executable instructions which when executed by a processor are adapted to carry out the method of any one of claims 1-10.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111467296.4A CN116233354A (en) | 2021-12-03 | 2021-12-03 | Production equipment monitoring processing method, device and system and AR glasses |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111467296.4A CN116233354A (en) | 2021-12-03 | 2021-12-03 | Production equipment monitoring processing method, device and system and AR glasses |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116233354A true CN116233354A (en) | 2023-06-06 |
Family
ID=86570171
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111467296.4A Pending CN116233354A (en) | 2021-12-03 | 2021-12-03 | Production equipment monitoring processing method, device and system and AR glasses |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116233354A (en) |
-
2021
- 2021-12-03 CN CN202111467296.4A patent/CN116233354A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101199290B1 (en) | Method and System for Plant Management by Augmentation Reality | |
| CN109690570B (en) | Computer room management method and dynamic loop system | |
| JP5961905B2 (en) | Inspection system and inspection method | |
| CN107203196B (en) | Industrial equipment detection system and method | |
| KR20160021419A (en) | Mobile terminal test system and mobile terminal test method using the system | |
| CN102272724A (en) | System and method for remote monitoring and control of field device | |
| KR20150083480A (en) | System for managing facilities by using 3d stereoscopic images | |
| KR102350702B1 (en) | System and Method for Augmented Reality-Based Predictive Maintenance in Facilities | |
| CN107979649B (en) | AOI multi-line multi-station system and method based on Inline server | |
| CN109921960B (en) | IDC machine room network anomaly testing method and device | |
| US11785184B2 (en) | Maintenance support system, maintenance support method, and program | |
| TW201508297A (en) | Test monitoring system and method for circuit board | |
| CN108805512B (en) | Attendance information recording method, equipment and system thereof | |
| JP2016173622A (en) | Information processing system | |
| CN106302011B (en) | Multi-terminal-based test method and terminal | |
| CN116233354A (en) | Production equipment monitoring processing method, device and system and AR glasses | |
| CN110543149A (en) | Aviation seat bullet loading and unloading auxiliary system based on intelligent glasses and use method | |
| CN115190151A (en) | Power inspection method, system and device and electronic equipment | |
| CN112671814B (en) | Cross-platform equipment development method, device and system | |
| CN106547485A (en) | Data migration method and device | |
| CN114066799A (en) | Detection system and detection method | |
| KR102626346B1 (en) | Method and apparatus for detecting whether a machine tool is in a state of wear and tear | |
| KR102656142B1 (en) | Device for managing field monitoring video and method of operating the device | |
| CN111708684B (en) | Recording method and device for page access and computer system | |
| KR102556986B1 (en) | Augmented Reality-Based Non-face-to-face Technical Support Service System and Method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |