TWI738149B - Smart inspection method and smart inspection system - Google Patents

Abstract

A smart inspection method includes steps of: providing a gateway, an instant message system and a wearable electronic device having an augmented reality (hereinafter “AR”) function, wearing the wearable electronic device having an AR function and inspecting in a field, allowing the wearable electronic device having an AR function to receive operation information of a plurality of apparatuses through the instant message system, and instantly displaying the operation information on corresponded positions corresponding to the apparatuses in a visual field of the wearable electronic device having an AR function. Therefore, statuses of the apparatuses can be instantly inspected. Since no meter has to be found or read, it may significantly reduce the inspection time and enhance the inspection efficiency, and further avoid dangers and instantly control the existing problems in the field.

Description

Smart inspection method and intelligent inspection system

This case is about an inspection method, especially a smart inspection method and a smart inspection system.

The traditional factory inspection method is that factory inspectors hold inspection manuals or tablet computers to the factory premises, and communicate with the back office through walkie-talkies. When arriving at the machine to be inspected or repaired, the inspector reads the inspection manual or browses the tablet computer, and finds out the inspection items and precautions corresponding to the machine, and repeats the machine inspection work of the entire factory. In addition, inspectors may also need to check the working environment and personnel status.

However, it involves the placement of the equipment and the instrument panel design of the machine itself, which makes it difficult to inspect some machines or equipment, and even requires climbing or squatting, resulting in poor inspection efficiency and even possible There are industrial safety risks. In addition, in traditional inspection operations, back-end personnel need to rely on a variety of external devices (such as monitors) in a control center to confirm on-site conditions in real time, resulting in high costs and time-consuming inspections or training. To put it simply, the traditional inspection method requires the purchase of a variety of specific software and hardware to build an inspection environment, which is a closed system and has a very high construction cost.

On the other hand, when inspectors need to carry out specific inspections, they must not only carry work manuals or tablet computers, but also specific hand tools, which often results in low inspection efficiency and may even cause safety problems due to the inability to free their hands. Concerns and so on.

Therefore, how to develop a smart inspection method and a smart inspection system that can effectively solve the aforementioned problems and shortcomings of the prior art is actually a problem that remains to be resolved.

The main purpose of this case is to provide a smart inspection method and a smart inspection system to solve and improve the aforementioned problems and shortcomings of the prior art.

Another purpose of this case is to provide a smart inspection method and a smart inspection system, through which the wearable electronic device with augmented reality function can display the operation information of the device in real time in the augmented reality, and the device status can be checked in real time. , Since there is no need to search for equipment meters or readings, inspection time can be greatly reduced and inspection efficiency can be effectively improved. In addition, danger can be avoided and problems existing in the field can be grasped in real time.

Another purpose of this case is to provide a smart inspection method and a smart inspection system, which can free both hands for inspection personnel to increase convenience and avoid safety problems, and also allow inspectors to receive equipment in the first time Maintenance notifications, and back-end management personnel can control the field conditions at any time and remotely, and can use big data calculations to remind potential equipment to be maintained to prevent systemic risks. In addition, it also has the advantages of identifying sites to facilitate inspection operations and being diversified for navigation and training, so as to greatly reduce labor burden and cost. Furthermore, it can also have industrial safety prevention and auditing functions to effectively ensure that workers comply with safety regulations, thereby avoiding industrial safety incidents and confirming the safety of the work area.

To achieve the above objective, a preferred implementation aspect of this case is to provide a smart inspection method, including the steps: (a) Provide a gateway, an instant messaging system and a wearable electronic device with augmented reality functions ; (B) Wear the wearable electronic device with augmented reality function to conduct inspections in a field; (c) The wearable electronic device with augmented reality function receives a plurality of operation information of a plurality of devices in the field transmitted by the gateway through the instant communication system; and (d) displays the plurality of operation information in real time Corresponding positions of the plurality of devices within a field of view of the wearable electronic device with augmented reality function.

In order to achieve the above purpose, a preferred implementation aspect of this case is to provide a smart inspection system, including: a gateway; an instant messaging system connected to the gateway to receive multiple data sent by the gateway A plurality of operation information of a device, and the plurality of operation information is converted into a plurality of instant communication messages; and a wearable electronic device with augmented reality function, wirelessly connects to the instant communication system, and receives the A plurality of instant communication messages, and the corresponding positions of the plurality of devices within a field of view of the plurality of operation information corresponding to the plurality of instant communication messages are displayed in real time.

1: Smart inspection system

11: Gateway

12: Instant messaging system

121: main server

122: instant messaging server

123: client device

13: Wearable electronic devices with augmented reality functions

14: Sensor

15: Device host

16: Remote monitoring device

17: Site identification

S100, S200, S300, S400: steps

S110, S120, S130, S140, S150: steps

S210, S220, S230: steps

S260, S270: steps

S310, S320: steps

S321, S322: steps

S410, S420: steps

S500, S600: steps

Figure 1 is a schematic diagram showing the architecture of the smart inspection system applicable to the smart inspection method in this case.

Figure 2 is a flowchart showing the smart inspection method of an embodiment of this case.

Figure 3 is a block diagram showing the architecture of the smart inspection system in one embodiment of the present case.

Figure 4 shows a partial flow chart of the smart inspection method of an embodiment of this case.

Fig. 5 shows a detailed flowchart of step S300 of the smart inspection method shown in Fig. 2.

Fig. 6 shows a detailed flow chart of step S320 of the smart inspection method shown in Fig. 5.

Figure 7 shows a partial flow chart of the smart inspection method of an embodiment of this case.

Figure 8 shows a partial flow chart of the smart inspection method of an embodiment of this case.

Figure 9 shows a partial flow chart of the smart inspection method in an embodiment of this case.

Figure 10 shows a partial flow chart of the smart inspection method of an embodiment of this case.

Some typical embodiments embodying the features and advantages of this case will be described in detail in the following description. It should be understood that this case can have various changes in different aspects, which do not depart from the scope of this case, and the descriptions and illustrations therein are essentially for illustrative purposes, rather than being constructed to limit the case.

Please refer to Figures 1 and 2. Figure 1 is a schematic diagram showing the structure of the smart inspection system applicable to the smart inspection method in this case, and Figure 2 is a flowchart showing the smart inspection method of an embodiment of this case . As shown in Figures 1 and 2, according to an embodiment of the present case, the smart inspection method includes the following steps: First, as shown in step S100, a gateway 11, an instant messaging system 12, and an augmented reality are provided Functional wearable electronic device 13. The gateway 11 can be, for example, a server, but it is not limited to this. The instant messaging system 12 can be a system based on an instant messaging software (or application). The instant messaging software can be, for example, Line, Whatsapp, WeChat (ie WeChat), Skype, or iMessage, but not limited to this. In addition, the wearable electronic device 13 with augmented reality function may preferably be, for example, an augmented reality glasses (AR Glass), but it is not limited thereto.

Then, as shown in step S200, the wearable electronic device 13 with augmented reality function is worn in the field for inspection. The field may be, for example, a factory or a clean room. Then, as shown in step S300, the wearable electronic device 13 with augmented reality function receives multiple operation information of multiple devices in the field transmitted by the gateway 11 through the instant communication system 12. Then, as shown in step S400, the corresponding positions of a plurality of operation information in the field of view of the wearable electronic device 13 with augmented reality function are displayed in real time. In other words, the wearable electronic device 13 with augmented reality function in this case can display the operating information related to the device in the field of view, such as pointer readings, operating parameters, and operating status, on the device through the augmented reality function. The position corresponding to the device on the display screen of the wearable electronic device 13 with augmented reality function. The inspectors wearing the wearable electronic device 13 with augmented reality function can see clearly and directly Receive information about the operation of each device. In this way, the status of the equipment can be checked in real time, and at the same time, since there is no need to search for equipment meters or readings, the inspection time can be greatly reduced and the inspection efficiency can be effectively improved. In addition, dangers can be avoided and the problems in the field can be grasped in real time.

Please refer to Figures 2, 3 and 4. Figure 3 is a block diagram showing the architecture of the smart inspection system of an embodiment of this case, and Figure 4 is a block diagram of the smart inspection method of an embodiment of this case. Part of the flowchart. As shown in Figures 2 to 4, in the smart inspection method of this case, the wearable electronic device 13 with augmented reality function is worn by the inspector, and at least the gateway 11 and the plurality of devices A device host 15 or at least one sensor 14 is connected to construct an Internet of Things. Since wearing the wearable electronic device 13 with augmented reality function does not need to hold anything, it can effectively free up hands to increase convenience and avoid safety problems. Further, after the aforementioned step S100, that is, between step S100 and step S200, between step S200 and step S300, between step S300 and step S400, or after step S400, the smart inspection method of this case may include step S110, As shown in step S110, the wearable electronic device 13 with augmented reality function is connected to the remote monitoring device 16 through the instant messaging system 12 to facilitate monitoring by a background manager. The remote monitoring device 16 can be a remote mobile device or a remote computer (as shown in FIG. 1), but it is not limited to this.

The following will explain several ways that the back-end management personnel may implement based on the smart inspection method of this case, but this case is not limited to this. Please refer to FIGS. 2 to 4 again. After the aforementioned step S110, the first method is as shown in step S120, the wearable electronic device 13 with augmented reality function communicates with the remote monitoring device through the instant messaging system 12 16 Perform one-way video communication or two-way video communication to share the field of vision to the remote monitoring device. 16. The one-way video communication or two-way video communication can be determined according to requirements, or according to bandwidth or field characteristics, but neither Limited by this.

The second method is as shown in step S130 and step S140, the remote monitoring device 16 transmits a communication message to the wearable electronic device 13 with augmented reality function through the instant messaging system 12, for example, transmission Send a real-time text message of Line software, and display the communication message on the wearable electronic device 13 with augmented reality function in real time, so the inspector can immediately and directly watch the communication message.

In addition, the third method is as shown in step S150, the wearable electronic device 13 with augmented reality function performs voice communication with the remote monitoring device 16 through the instant messaging system 12, so that the inspection personnel and the background management personnel can directly communicate with each other. call. Through the above methods, the back-office management personnel can directly watch the inspectors' field of view and confirm the on-site conditions in real time, and can direct the inspectors to perform operations through microphones or typing to ensure the correct operation steps and reduce industrial safety accidents.

In some embodiments, the smart inspection method of this case can implement more than one of the above three methods at the same time, or only one of them can be selected. In other words, after step S110, the above steps S120, S130, S140, and S150 can be executed simultaneously, or the above steps S120, S130, S140, and S150 can be executed alternately, or only steps S120, S130, and S150 can be executed. Step S140, or step S150. At the same time, because the remote monitoring device 16 used by the background manager is not limited to a specific device, it only needs to have instant messaging function or install instant messaging software to be used as a remote mobile device or remote computer, so the background manager can Control field conditions at any time and from different places.

Please refer to FIG. 2, FIG. 3, and FIG. 5. FIG. 5 shows a detailed flowchart of step S300 of the smart inspection method shown in FIG. 2. As shown in Figures 2, 3, and 5, according to an embodiment of the present case, step S300 of the smart inspection method may include detailed flow steps as follows: First, as shown in step S310, the gateway 11 transmits Multiple operation information of multiple devices in the field are sent to the instant messaging system 12. Then, as shown in step S320, the instant messaging system 12 transmits a plurality of operational information in the form of instant messaging messages to the wearable electronic device 13 with augmented reality function. In some embodiments, the instant communication system 12 includes a main server 121 and an instant communication server 122. The gateway 11 synchronously transmits a plurality of operation information to the main server 121 and the instant communication server 122, and the main server 121 calculates with big data based on a plurality of operation information, predicts the equipment to be maintained among the plurality of equipment. In this way, it can alert potential equipment to be maintained and prevent systemic risks.

In this case, the wearable electronic device 13 with augmented reality function used in the smart inspection method mainly has two types. One is that the wearable electronic device 13 with augmented reality function only has a head-mounted display. , Cameras, communication components, etc., the main data transmission is based on integrated computing. The client device, such as a smart phone, is used as a carrier; the second is that the wearable electronic device 13 with augmented reality function has sufficient computing functions. , Can directly carry out data transmission and calculation. However, no matter which one of these two categories is, it is applicable to the smart inspection method of this case. In addition, the client device described here can exist independently or be regarded as a part of the instant messaging system 12, which will be described below.

Please refer to FIG. 2, FIG. 3, FIG. 5, and FIG. 6, where FIG. 6 is a detailed flowchart of step S320 of the smart inspection method shown in FIG. 5. As shown in Figures 2, 3, 5, and 6, the instant messaging system 12 in this case may further include a client device 123. The client device 123 may be, for example, but not limited to, a smart phone and an instant messaging system. The communication server 122 and the wearable electronic device 13 with augmented reality function are connected wirelessly. The client device 123 and the instant communication server 122 are preferably connected to the client device 123 or with the augmented reality function. The real-world wearable electronic device 13 is connected in different ways, such as the Internet and Bluetooth communication, but not limited to this. In addition, the aforementioned step S320 may further include detailed flow steps as follows: First, as shown in step S321, the instant messaging server 122 transmits a plurality of operational information to the client device 123 in the form of instant messaging messages. Then, as shown in step S322, the client device 123 transmits a plurality of operation information to the wearable electronic device 13 with augmented reality function in the form of instant messaging messages.

Please refer to Fig. 2, Fig. 3 and Fig. 7. Fig. 7 shows a partial flowchart of the smart inspection method according to an embodiment of the present case. As shown in Figures 2, 3, and 7, according to an embodiment of the present case, the smart inspection method between step S300 and step S400, or after step S400, may further include the following steps: First, as As shown in step S500, the main server 121 sends a maintenance notification in the form of an instant communication message to the wearable electronic device 13 with augmented reality function through the instant communication server 122. Then, as shown in step S600, the maintenance notification is displayed on the wearable electronic device with augmented reality function in real time. Device. The implementation of step S600 described here can be to directly display the maintenance notification in a warning manner on the wearable electronic device 13 with augmented reality function, so that the inspector can receive the notification in the first time; or It is displayed on the corresponding position on the wearable electronic device 13 with augmented reality function when there is a device that needs to be repaired corresponding to the maintenance notice in the inspector's field of view.

Please refer to Fig. 2, Fig. 3 and Fig. 8. Fig. 8 shows a partial flowchart of the smart inspection method according to an embodiment of the present case. As shown in Fig. 2, Fig. 3 and Fig. 8, according to an embodiment of the present case, after step S200, after including any steps after step S200, the smart inspection method may further include the following steps: First, as As shown in step S210, the wearable electronic device 13 with augmented reality function scans one site identification 17 of the plurality of site identifications 17. Then, as shown in step S220, the site corresponding to the site identification 17 is confirmed, and the name of the site is displayed on the wearable electronic device 13 with augmented reality function in real time. In some embodiments, the site identification 17 may be a barcode, such as a Quick Response CODE (QR CODE), but it is not limited to this. The wearable electronic device 13 with augmented reality function can be operated by patrol personnel to scan the site identification 17 specially, or it can scan automatically when the site identification 17 exists in the field of view and identify the site at which it is located. It is conducive to the monitoring of the inspection process or background management personnel.

Furthermore, the smart inspection method of this case can be applied to multiple applications, such as navigation and training. Please refer to Figure 8 again. According to the concept of this case, after the site is confirmed in step S220, the smart inspection method of this case may further include step S230. As shown in step S230, the inspection standard operating procedure (SOP) of the corresponding site, the maintenance confirmation item of the corresponding site, the inspection teaching of the corresponding site or the site navigation of the corresponding site are displayed in real time. Wearable electronic device 13 with real-world functions. In other words, since the inspection standard operating procedures, maintenance confirmation items, inspection teaching and site navigation corresponding to each site can be built and displayed according to requirements, the smart inspection method of this case is needed Training new personnel or guiding visiting visitors can greatly reduce the labor burden and cost.

Please refer to Fig. 2, Fig. 3 and Fig. 9, where Fig. 9 shows a partial flowchart of the smart inspection method according to an embodiment of the present case. As shown in Figures 2, 3 and 9, the smart inspection method of this case may further include the following steps after step S400: First, as shown in step S410, input a control command to Wearable electronic device 13 with augmented reality function. Then, as shown in step S420, the wearable electronic device 13 with augmented reality function transmits control commands through the instant messaging system 12. In this embodiment, the inspector wearing the wearable electronic device 13 with augmented reality function can use gestures, such as OK or questionable gestures (pre-trained), or voice, such as saying " Sentences such as “site inspection completed”, “machine abnormality”, “temperature adjustment increased by 2 degrees” or “enable site navigation” (pre-training is possible), and the wearable electronic device 13 with augmented reality The gesture or voice is converted into a control instruction, and then the control instruction is transmitted. The transmission object can be a background manager, equipment or a host of inspection records, but it is not limited to this.

Please refer to Fig. 2, Fig. 3 and Fig. 10. Fig. 10 shows a partial flowchart of the smart inspection method according to an embodiment of this case. As shown in Figures 2, 3, and 10, the smart inspection method of this case may include an audit function. Specifically, after step S200 of the smart inspection method of this case, any steps after step S200 are included. After that, the system may further include the following steps: First, as shown in step S260, the wearable electronic device 13 with augmented reality function recognizes the working area in the field. Then, as shown in step S270, confirm whether the plurality of audit items in the work area meet the field regulations, and display the confirmation result on the wearable electronic device 13 with augmented reality function in real time. Among them, the audit items can include whether debris is piled up in the field, or whether there is clear space within a specific area of the field, or whether the equipment operation meets specific conditions, which can be determined and realized by image recognition technology through pre-training. Achieve safety in the work area.

Please refer to Figure 2 again. In combination with the various embodiments described above, this case proposes a smart inspection system 1 including a gateway 11, an instant messaging system 12, and a wearable electronic device 13 with augmented reality functions. Among them, the instant messaging system 12 is connected to the gateway 11 to receive multiple devices sent by the gateway 11. Prepare multiple operation information, and convert multiple operation information into multiple instant messaging messages. On the other hand, the wearable electronic device 13 with augmented reality function wirelessly connects to the instant messaging system 12, receives a plurality of instant communication messages, and displays the plurality of operation information corresponding to the plurality of instant communication messages in real time. Corresponding positions of multiple devices in the field of view. It should be noted that the smart inspection system 1 proposed in this case is an open system. Any gateway 11, any instant messaging system 12, and any wearable electronic device 13 with augmented reality functions are only The structure described here can be applied to the foregoing embodiments, so that no need to purchase specific hardware can be achieved, thereby effectively reducing the cost.

To sum up, this project provides a smart inspection method and a smart inspection system. A wearable electronic device with augmented reality function can display the operation information of the device in real time. In the augmented reality, the status of the device can be checked in real time. , Since there is no need to search for equipment meters or readings, inspection time can be greatly reduced and inspection efficiency can be effectively improved. In addition, danger can be avoided and problems existing in the field can be grasped in real time. Furthermore, it can free up both hands for inspectors to increase convenience and avoid safety issues, and it can also allow inspectors to receive equipment maintenance notifications at the first time, and back-end managers can control the field conditions at any time and remotely. , And can use big data calculations to remind potential equipment to be maintained to prevent systemic risks. In addition, it also has the advantages of identifying sites to facilitate inspection operations and being diversified for navigation and training, so as to greatly reduce labor burden and cost. Furthermore, it can also have industrial safety prevention and auditing functions to effectively ensure that workers comply with safety regulations, thereby avoiding industrial safety incidents and confirming the safety of the work area.

Even though the present invention has been described in detail by the above-mentioned embodiments, it can be modified in many ways by those skilled in the art, but it does not deviate from the scope of the attached patent application.

S100, S200, S300, S400: steps

Claims (12)

A smart inspection method, including the steps: (a) providing a gateway, an instant messaging system and a wearable electronic device with augmented reality functions, wherein the instant messaging system includes a main server and an instant messaging A server and a client device, the client device is wirelessly connected with the instant communication server and the wearable electronic device with augmented reality function, and the instant communication system is a system based on an instant communication software; (b) Wear the wearable electronic device with augmented reality function to conduct inspections in a field; (c) The wearable electronic device with augmented reality function receives the data transmitted by the gateway through the instant messaging system A plurality of operation information of a plurality of devices in the field; and (d) real-time display of the plurality of operation information corresponding to the plurality of devices in the field of view of the wearable electronic device with augmented reality function Location; wherein, the step (c) further includes the step: (c1) the gateway transmits the plurality of operation information of the plurality of devices in the field to the instant communication system, wherein the instant communication system includes a master Server and an instant communication server, the gateway synchronously sends the plurality of operation information to the main server and the instant communication server, and the main server uses big data calculations based on the plurality of operation information Predict the device that needs to be maintained among the plurality of devices; and (c2) the instant messaging system transmits the plurality of operational information in the form of instant messaging messages to the wearable electronic device with augmented reality function; wherein, the Step (c2) further includes the steps: (c21) the instant messaging server sends the plurality of operational information to the client device in the form of instant messaging messages; and (c22) The client device sends the plurality of operation information to the wearable electronic device with augmented reality function in the form of instant messaging messages; wherein the client device and the instant messaging server are connected , The connection method is different from the client device and the wearable electronic device with augmented reality function. For example, the smart inspection method described in item 1 of the scope of patent application, wherein the wearable electronic device with augmented reality function is worn by an inspector, and at least one of the gateway and the plurality of devices The host or at least one sensor is connected to construct an Internet of Things. For example, the smart inspection method described in item 2 of the scope of patent application further includes the following steps after step (a): (a1) the wearable electronic device with augmented reality function communicates with a remote through the instant messaging system The monitoring device is connected to facilitate monitoring by a back-end manager. For example, the smart inspection method described in item 3 of the scope of patent application further includes the following steps after step (a1): (a2) the wearable electronic device with augmented reality function communicates with the remote through the instant messaging system The monitoring device performs one-way video communication or two-way video communication to share the field of view to the remote monitoring device. For example, the smart inspection method described in item 3 of the scope of patent application includes the following step after step (a1): (a3) the remote monitoring device sends a communication message to the augmented reality through the instant messaging system Functional wearable electronic device; and (a4) display the communication message on the wearable electronic device with augmented reality function in real time. For example, the smart inspection method described in item 3 of the scope of patent application further includes the steps after step (a1): (a5) the wearable electronic device with augmented reality function communicates with the remote through the instant messaging system The monitoring device conducts voice communication, so that the inspector can talk with the background manager. For example, the smart inspection method described in item 1 of the scope of patent application, between the step (c) and the step (d), or after the step (d), further includes the step: (e) the main servo The device sends a maintenance notification in the form of an instant messaging message to the wearable electronic device with augmented reality function through the instant communication server; and (f) displays the maintenance notification in real time on the augmented reality function Wearable electronic devices. The smart inspection method described in item 1 of the scope of patent application, wherein the field has a plurality of site identifications, and after the step (b), the smart inspection method further includes the steps: (b1) the tool The wearable electronic device with augmented reality function scans one of the plurality of site identifications for the site identification; and (b2) the wearable electronic device with augmented reality function confirms that the site identification corresponds to a station Click and instantly display a name of the site on the wearable electronic device with augmented reality function. For example, the smart inspection method described in item 8 of the scope of patent application, after this step (b2), it further includes the following steps: (b3) real-time display of the inspection standard operating procedure of one of the corresponding sites and one of the corresponding sites Maintenance confirmation items, inspection and teaching at one of the corresponding sites, or navigation at one of the sites corresponding to the wearable electronic device with augmented reality function. For example, the smart inspection method described in item 1 of the scope of patent application, after step (d), further includes the step of: (d1) inputting a control command to the wearable electronic device with augmented reality function by gesture or voice ; (D2) The wearable electronic device with augmented reality function transmits the control command through the instant communication system. The smart inspection method described in item 1 of the scope of patent application, wherein the field has a plurality of site identifications, and after the step (b), the smart inspection method further includes the steps: The wearable electronic device with augmented reality function recognizes a work area in the field; and the wearable electronic device with augmented reality function confirms whether a plurality of audit items in the work area meet the field regulations, And instantly display the confirmation result on the wearable electronic device with augmented reality function. A smart inspection system includes: a gateway; an instant communication system connected to the gateway to receive multiple operation information of multiple devices sent by the gateway, and combine the multiple operation information Converted into a plurality of instant communication messages, where the instant communication system includes a main server, an instant communication server and a client device, and the instant communication system is a system based on an instant communication software, and the gateway is synchronized The plurality of operation information are sent to the main server and the instant communication server of the instant communication system, and the main server uses big data calculations to predict the maintenance of the plurality of devices based on the plurality of operation information Equipment; and a wearable electronic device with augmented reality function, wherein the client device is wirelessly connected with the instant messaging server and the wearable electronic device with augmented reality function, and the client The connection method of the device and the instant communication server is different from the connection method of the client device and the wearable electronic device with augmented reality function; wherein, the instant communication system transmits the plurality of instant communication messages To the client device, the client device transmits the plurality of instant communication messages to the wearable electronic device with augmented reality function, and the wearable electronic device with augmented reality function receives the plurality of instant messages The corresponding position of the plurality of operation information corresponding to the plurality of instant communication messages in the field of view of the plurality of devices is displayed in real time.

Images