JP2017182487A - System, information processing device, head-mounted device, and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To notify a result of operation in realtime when a user performs work using a head-mounted device.SOLUTION: An information processing device includes: a first communication unit that performs communication with a head-mounted device; and a determination unit that determines whether or not information indicating an abnormal state in relation to a device or work associated with the head-mounted device is detected; wherein the first communication unit transmits information indicating a result of the determination detected by the determination unit to the head-mounted device. The head-mounted device includes a display unit and a second communication unit that performs communication with the information processing device; wherein the second communication unit receives the determination result transmitted by the information processing device, and the display unit displays information indicating the determination result received by the second communication unit.SELECTED DRAWING: Figure 4

Description

  Embodiments described herein relate generally to a system, an information processing apparatus, a head mount apparatus, and a method.

In recent years, devices such as sensors have been miniaturized, and wearable head mounted displays (HMDs) such as smart glasses have been developed. In a system using a head-mounted display, there is known a system that displays information by projecting information on a virtual screen such as a spectacle lens, a glass or plastic in front of sunglasses or goggles.
Regarding a technique for performing a fault repair work using a head-mounted display, a technique is known in which even if an unexpected fault occurs, a technique for guiding an optimal fault repair work and providing work support (for example, see Patent Document 1). reference).

JP 2011-248860 A

The worker performs work such as connection to the equipment in accordance with the instruction from the monitoring center. When the work is finished, the worker notifies the monitoring center that the work is finished. The monitoring center that has received the notification of the work completion notifies the worker whether or not an alarm has been issued.
If the monitoring center monitors multiple tasks at the same time and detects multiple alarms at the same time, it cannot respond to multiple alarms at the same time, and notifies that an alarm has been issued. May be delayed.

  The present invention has been made to solve the above problem, and an object of the present invention is to notify the result of the work in real time when the user performs the work using the head mount device.

(1) One aspect of the present invention is a system including a head mount device and an information processing device, wherein the information processing device includes a first communication unit that communicates with the head mount device; A determination unit that determines whether or not information indicating an abnormal state is detected in association with a device or work associated with the head mount device, and the first communication unit is configured to determine whether the determination unit detects Information representing a result is transmitted to the head mount device, and the head mount device includes a display unit and a second communication unit that communicates with the information processing device, and the second communication unit includes: The information processing apparatus receives the determination result, and the display unit displays information indicating the determination result received by the second communication unit.

(2) According to one aspect of the present invention, in the system described in (1) above, the information processing apparatus detects an apparatus associated with the head mount apparatus or information indicating an abnormal state related to the work. Information indicating the abnormal state from the corresponding procedure stored in the first storage unit when the determination unit detects information indicating the abnormal state when the first storage unit stores the corresponding procedure when The first communication unit transmits information representing the corresponding procedure acquired by the acquisition unit to the head mount device, and the second communication unit The communication unit receives information indicating the handling procedure transmitted by the information processing apparatus, and the display unit displays the handling procedure received by the second communication unit.

(3) According to an aspect of the present invention, in the system according to (1), the information processing apparatus stores each of a plurality of pieces of work identification information and the order of the work in association with each other. Work identification information corresponding to the next order of the work order in which the information indicating the abnormal state was not detected by the determination part from the work order stored in the storage unit and the second storage unit An acquisition unit to acquire, wherein the first communication unit transmits work identification information corresponding to the next order acquired by the acquisition unit to the head mount device, and the second communication unit The work identification information corresponding to the next order transmitted by the information processing apparatus is received, and the display unit receives the work identification information corresponding to the next order received by the second communication unit. To display the work corresponding to Is Temu.

(4) One aspect of the present invention is the system according to (3), wherein the first communication unit is configured to identify the work identification information when the determination unit detects information indicating the abnormal state. This is a system that stops sending messages.

(5) In one aspect of the present invention, a communication unit that communicates with the head mount device and whether information indicating an abnormal state is detected in relation to the device or work associated with the head mount device is detected. The communication unit is an information processing device that transmits information representing a result of the determination detected by the determination unit to the head mount device.

(6) One embodiment of the present invention includes a display unit and a communication unit that communicates with an information processing device, and the display unit is a device associated with a head mount device received by the communication unit or The head mount device displays information indicating a result of determination as to whether or not information indicating an abnormal state has been detected in relation to work.

(7) One aspect of the present invention is a method executed by a system including a head mount device and an information processing device, wherein the information processing device is related to a device or work associated with the head mount device. Determining whether or not information indicating an abnormal state has been detected, a step in which the information processing device transmits information indicating a result of the determination to the head mount device, and the head mount device includes the determination Receiving the information indicating the result of the above, and displaying the information indicating the result of the determination by the head mount device.

  According to the embodiment of the present invention, when a user performs an operation using the head mount device, the result of the operation can be notified in real time.

It is a figure which shows the system which concerns on this embodiment. It is a figure which shows an example of the hardware constitutions of the head mount apparatus which concerns on this embodiment. It is a figure which shows an example of the hardware constitutions of the information processing apparatus which concerns on this embodiment. It is a functional block diagram showing an example of a system concerning this embodiment. It is a figure which shows an example of the image projected by the head mount apparatus which concerns on this embodiment. It is a figure which shows an example of a head mount apparatus management table. It is a figure which shows an example of a pairing management table. It is a figure which shows an example of a related apparatus and work management table. It is a sequence diagram which shows an example of operation | movement of the system which concerns on this embodiment. It is a figure which shows an example of a related apparatus and work management table. It is a sequence diagram which shows an example of operation | movement of the system which concerns on this embodiment. It is a functional block diagram showing an example of a system concerning this embodiment. It is a figure which shows an example of an abnormal condition and response procedure corresponding table. It is a sequence diagram which shows an example of operation | movement of the system which concerns on this embodiment.

<First Embodiment>
<System>
The system according to the present embodiment displays an image captured by the imaging unit of the head mount device on the information processing device. Further, the system transmits image data acquired by, for example, imaging by one head mount device to other head mount devices among a plurality of head mount devices. When the other head mount apparatus receives the image data transmitted by the one head mount apparatus, the other head mount apparatus projects the data by processing the data. As a result, the image data can be shared in real time among the plurality of head mount devices.
Further, when the information processing apparatus detects information indicating an abnormal state in association with an apparatus or work associated with the head mount apparatus, the information processing apparatus notifies the head mount apparatus of information indicating that the abnormal state has been detected. When the head mount device receives information indicating that the abnormal state transmitted by the information processing device has been detected, the head mount device processes and displays information indicating that the abnormal state has been detected.

FIG. 1 shows an example of a system. The system includes a head mount device 100a, a head mount device 100b, and an information processing device 200. For example, the information processing apparatus 200 is provided in a monitoring center, and the head mount apparatus 100a and the head mount apparatus 100b are mounted by a worker who performs work on the apparatus. In FIG. 1, two head mount devices are illustrated as an example, but the number of head mount devices may be one or three or more. Further, a terminal device may be provided instead of the head mount device 100b, or a terminal device may be provided together with the head mount device 100b. An example of the terminal device is a mobile phone, a smartphone, a tablet terminal, or the like. Hereinafter, as an example, the case where the system includes the head mount device 100b will be described.
The information processing apparatus 200 is connected to a communication network 20 such as the Internet. The head mount device 100 a and the head mount device 100 b are connected to the communication network 20 via the access point 70.

  In the system according to the present embodiment, as an example, a case will be described in which data obtained by the head mount device 100a is shared between the information processing device 200 and the head mount device 100b. An example of the head mount device 100a and the head mount device 100b is mounted on the operator's head. It is configured such that an image that can be visually recognized by an operator can be presented to an operator wearing the head mount device 100a and the head mount device 100b on the head. For example, the head mount device 100a and the head mount device 100b are transmissive head mount displays. In the example shown in FIG. 1, the head mount device 100 is a glasses-type transmissive head mount display. The head mount device 100a and the head mount device 100b may be a head-mounted head mount display having a shape other than the glasses type.

  The head mount device 100a and the head mount device 100b display information on a virtual screen and display the information superimposed on the background that is the surrounding environment, so that the user can check various information while viewing the background. That is, the head mount device 100a and the head mount device 100b superimpose the information displayed on the virtual screen and the background behind the virtual screen. The head mount device 100a transmits the identification information and authentication information of the head mount device 100a to the information processing device 200.

The information processing apparatus 200 stores the identification information of one or more head mount devices that share data acquired by the head mount device in association with each of the identification information of the plurality of head mount devices. Furthermore, the information processing device 200 stores an IP address for each of the plurality of head mount devices. Further, the information processing apparatus 200 links the identification information of one or a plurality of devices and the identification information of one or a plurality of operations performed on each of the one or more devices for each of the plurality of head mount devices. I remember it.
When the information processing apparatus 200 receives the identification information and authentication information of the head mount apparatus 100a transmitted by the head mount apparatus 100a, the head associated with the identification information of the head mount apparatus 100a as a pairing target The head mount device 100b corresponding to the identification information of the mount device 100b is specified. Furthermore, the information processing apparatus 200 acquires the IP address of the identified head mount apparatus 100b.

  The information processing apparatus 200 makes a connection request by transmitting the authentication information transmitted by the head mount apparatus 100a to the head mount apparatus 100b. When the head mount device 100b receives the authentication information transmitted by the information processing device 200, the head mount device 100b establishes a communication link with the head mount device 100a using the authentication information. Further, when receiving the authentication information transmitted by the information processing apparatus 200, the head mount apparatus 100b may establish a communication link with the information processing apparatus 200 using the authentication information. Hereinafter, when the head mount device 100a and the head mount device 100b are not distinguished, they are referred to as the head mount device 100.

  In the system according to the present embodiment, the head mount device 100 performs data communication with the information processing device 200. For example, the head mount device 100a transmits identification information and authentication information of the head mount device 100a to the information processing device 200, and the information processing device 200 transmits authentication information to the head mount device 100b. Furthermore, the head mount device 100a performs wireless communication with the head mount device 100b. For example, communication may be performed between the head mount device 100a and the head mount device 100b in accordance with a near field communication (NFC) standard such as Bluetooth (registered trademark).

<Configuration of head mount device>
FIG. 2 is a diagram illustrating an example of a head mount device according to the present embodiment. The head mount device 100 includes a CPU (Central Processing Unit) 102, a ROM (Read Only Memory) 104, a RAM (Random Access Memory) 106, an operation unit 107, a nonvolatile memory 108, and a communication I / F unit 110. An imaging unit 112, a projection unit 114, and an internal bus 116. These are arranged in a housing such as a temple of the head mount device 100.

The CPU 102 reads a program 1082 and the like for controlling the operation of the head mount apparatus 100 from the nonvolatile memory 108, expands it in the RAM 106, and executes it.
The operation unit 107 is an input device that receives a user operation, and includes a pointing device such as a touch panel, buttons, a dial, a touch sensor, a touch pad, and the like. In the present embodiment, as an example, the case where the operation unit 107 forms a button will be described.
The non-volatile memory 108 includes a flash memory, an HDD (Hard Disc Drive), an SSD (Solid State Drive), an SD (Secure Digital) card, and the like, and is disposed in the casing of the head mount device 100. The nonvolatile memory 108 associates and stores object data to be projected on the virtual screen by the projection unit 114, setting information for specifying the positions of both eyes, the position of the virtual screen, and the like.

The communication I / F unit 110 is configured by a communication module such as an NFC module or a wireless LAN module, and is disposed in the casing of the head mount device 100. The communication I / F unit 110 communicates with an external device by NFC or wireless LAN.
The imaging unit 112 is configured by a CCD (Charge Coupled Devices) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, or the like, and is fixed to a housing such as a vine (temple) of the head mount device 100. The imaging unit 112 outputs, to the CPU 102, image data obtained by imaging the position or movement of the pupils of both eyes of the user directly or using a half mirror or the like.
The projection unit 114 is disposed in the housing of the head mount device 100. The projection unit 114 displays the object on the virtual screen by processing the image data of the object supplied by the CPU 102.

<Configuration of information processing apparatus>
FIG. 3 shows an example of a hardware configuration of the information processing apparatus 200 according to the present embodiment. The information processing apparatus 200 includes a CPU 202, a memory 204, a nonvolatile memory 206, a communication I / F 208, a display device 209, and an internal bus 210.
For example, the CPU 202 executes a program stored in the non-volatile memory 206 and controls each unit of the information processing apparatus 200 using the memory 204 as a work memory. The memory 204 is configured by a RAM such as a volatile memory using a semiconductor element. The memory 204 is used as a work memory for the CPU 202. The nonvolatile memory 206 is configured by, for example, a hard disk (HD), a ROM, or the like, and stores a program executed by the CPU 202.

  The communication I / F 208 is an interface for communicating with an external device and transmitting / receiving various data such as files and commands. The display device 209 is configured by a liquid crystal display, an organic EL display, or the like, for example, and displays images such as characters and photographs. The internal bus 210 connects the CPU 202, the memory 204, the nonvolatile memory 206, the communication I / F 208, and the display device 209 to each other. Each unit connected to the internal bus 210 can exchange data with each other via the internal bus 210.

<Functional Configuration of System According to this Embodiment>
FIG. 4 is a functional block diagram of the head mount device 100a, the head mount device 100b, and the information processing device 200 that constitute the system of the present embodiment. In FIG. 4, the head mount device 100 a, the head mount device 100 b, and the information processing device 200 are connected so that data communication can be performed via the communication network 20.

<Functional configuration of head mount device>
The head mount device 100a includes a wireless communication unit 152a, a projection processing unit 154a, an imaging processing unit 156a, a connection processing unit 160a, a control unit 162a, a storage unit 164a, and each of the above components as shown in FIG. A bus line 166a such as an address bus or a data bus for connection is provided. Each of these units is a function realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 102 that executes the program 1082 expanded from the nonvolatile memory 108 onto the RAM 106. is there.

<Functional configuration of the head mount device>
Each functional configuration of the head mount device 100a will be described in detail with reference to FIGS. In the following, in describing each functional configuration of the head mount device 100a, among the components illustrated in FIG. 4, the main hardware for realizing each functional configuration of the head mount device 100a is described. Explain the relationship.
The wireless communication unit 152a of the head mount device 100a illustrated in FIG. 4 is realized by a command from the CPU 102 and the communication I / F 110. The wireless communication unit 152 a transmits and receives various data to and from the information processing apparatus 200 via the access point 70 and the communication network 20. Further, the wireless communication unit 152a transmits and receives various data to and from the head mount device 100b. For example, the wireless communication unit 152a transmits / receives various data to / from the information processing apparatus 200 via the access point 70 and the communication network 20 according to the wireless LAN, and various data to / from the head mount apparatus 100b. Send and receive.

The projection processing unit 154a is realized by a command from the CPU 102 and the projection unit 114. The projection processing unit 154a displays the object on the virtual screen based on the object data supplied by the control unit 162a.
FIG. 5 shows an example in which an object is displayed on the virtual screen by the projection processing unit 154a. Here, as an example, an object projected by the head mount device 100a attached by a worker who performs a repair work when a failure occurs in a device or the like is shown. In the example shown in FIG. 5, objects such as “in operation! Attention”, “work target”, “non-target device”, and “non-target device (YYYYMM erroneous work occurrence location)” are superimposed on the background of the device group or the like. Is displayed. Accordingly, the user can perform work while confirming the state of each device included in the device group. Further, the projection processing unit 154a may project a document such as a work instruction sheet, a work check table, and a work manual as an object. As a result, the worker can perform work without bringing a work instruction sheet, a work check table, a work manual, or the like to the place where the failure has occurred. Returning to FIG.

The imaging processing unit 156a is realized by a command from the CPU 102 and the imaging unit 112. The imaging processing unit 156a outputs image data obtained by imaging the positions and movements of the eyes of both eyes of the user wearing the head mount device 100a to the control unit 162a.
The connection processing unit 160a is realized by a command from the CPU 102 and the communication I / F 110. The connection processing unit 160a holds authentication information used for pairing between the head mount device 100a and the head mount device 100b. The connection processing unit 160a may hold authentication information in advance, or when the user performs an operation for sharing data between the head mount device 100a and the head mount device 100b on the operation unit 107. You may make it produce | generate and hold | maintain.

The connection processing unit 160a creates a signal for requesting data sharing (hereinafter referred to as “data sharing request”). The data sharing request is accompanied by identification information of the head mount device 100a (ID of the head mount device 100a) and authentication information. The connection processing unit 160a transmits a data sharing request from the wireless communication unit 152a to the information processing device 200.
Further, when the connection request transmitted from the head mount device 100b is received by the wireless communication unit 152a, the connection processing unit 160a determines whether the authentication information included in the connection request matches the authentication information held. Determine whether. If the connection processing unit 160a determines that the authentication information included in the connection request matches the authentication information held, the connection processing unit 160a performs an encryption process using a link key, and the head mount device 100a and the head mount device A communication link with 100b is established. If the connection processing unit 160a determines that the authentication information included in the connection request and the held authentication information do not match, the connection processing unit 160a does not establish a communication link between the head mount device 100a and the head mount device 100b.

The control unit 162a is realized by a command from the CPU 102. The control unit 162a controls the entire head mount device 100a. For example, when the user performs an operation for sharing data between the head mount device 100a and the head mount device 100b on the operation unit 107, the control unit 162a creates a data sharing request to the connection processing unit 160a. Order to do. In addition, after the communication link is established between the wireless communication unit 152a and the head mount device 100b, the control unit 162a captures the positions and movements of the eyes of both eyes of the user wearing the head mount device 100a. Is transmitted from the wireless communication unit 152a to the information processing apparatus 200 and the head mount apparatus 100b.
Furthermore, when information indicating that the abnormal state transmitted by the information processing device 200 is detected is received by the wireless communication unit 152a, the control unit 162a displays information indicating that the abnormal state has been detected as the projection processing unit 154a. Output to. The storage unit 164a is realized by the nonvolatile memory 108. The storage unit 164a stores the ID 1642a of the head mount device 100a.

The head mount device 100b includes a wireless communication unit 152b, a projection processing unit 154b, an imaging processing unit 156b, a connection processing unit 160b, a control unit 162b, a storage unit 164b, and the above-described components as shown in FIG. A bus line 166b such as an address bus or a data bus for connection is provided. Each of these units is a function realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 102 that executes the program 1082 expanded from the nonvolatile memory 108 onto the RAM 106. is there.
The wireless communication unit 152b, the projection processing unit 154b, and the imaging processing unit 156b can apply the wireless communication unit 152a, the projection processing unit 154a, and the imaging processing unit 156a described above.
When the wireless communication unit 152b receives data sharing information transmitted from the information processing apparatus 200, the connection processing unit 160b acquires authentication information included in the data sharing information. Then, the connection processing unit 160b attaches the authentication information to the connection request and transmits the connection request from the wireless communication unit 152b to the head mount device 100a.
The controller 162b performs overall control of the head mount device 100b. After the communication link is established between the wireless communication unit 152b and the head mount device 100a, the control unit 162b outputs the data received by the wireless communication unit 152b to the projection processing unit 154b. Specifically, when information indicating that the abnormal state transmitted by the information processing device 200 is detected is received by the wireless communication unit 152b, the control unit 162b projects information indicating that the abnormal state has been detected. The data is output to the processing unit 154b.
The storage unit 164b stores the ID 1642b of the head mount device 100b.

<Functional configuration of information processing apparatus>
The information processing apparatus 200 includes a transmission / reception unit 252, a control unit 254, a selection unit 256, an abnormal state determination unit 257, a storage unit 258, a display control unit 259, and the above components as shown in FIG. 4. A bus line 260 such as an address bus or a data bus for connecting to the bus. Each of these units operates in response to an instruction from the CPU 202 that executes the program 2062 expanded from the nonvolatile memory 206 to the memory 204 in the information processing apparatus 200 in any one of the components illustrated in FIG. It is a function realized by this.

<Each functional configuration of information processing apparatus>
Each functional configuration of the information processing apparatus 200 will be described in detail with reference to FIGS. 3 and 4. In the following description, in describing each functional configuration of the information processing apparatus 200, it is assumed that the main hardware for realizing each functional configuration of the information processing apparatus 200 among the respective constituent elements illustrated in FIG. Explain the relationship.
The transmission / reception unit 252 of the information processing apparatus 200 illustrated in FIG. 4 is realized by a command from the CPU 202 and a communication I / F 208. The transmission / reception unit 252 transmits / receives various data to / from the head mount device 100a or the head mount device 100b via the communication network 20 and the access point 70. For example, the transmission / reception unit 252 receives image data transmitted by the head mount device 100a.
The storage unit 258 is realized by the nonvolatile memory 206. The storage unit 258 stores a head mount device management table 2582, a pairing management table 2584, and a related device / work management table 2586.

<Head mount device management table>
FIG. 6 shows an example of the head mount device management table. The head mount device management table manages the IP address of the head mount device in association with each head mount device ID. For example, the head mount device management table shown in FIG. 6 indicates that the IP address of the head mount device whose head mount device ID is “100aa” is “192.168.10.10.”. ing. The ID and IP address of the head mount device shown in FIG. 6 are examples, and the present invention is not limited to this example.

<Pairing management table>
FIG. 7 shows an example of the pairing management table. The pairing management table associates and manages the ID of the head mount device to which data is shared for each head mount device ID. For example, in the pairing management table shown in FIG. 7, the head mount device whose ID of the head mount device is “100aa” has the ID of the head mount device to which the data is shared is “100bb”. It is shown. The ID of the head mount device and the ID of the head mount device to be paired shown in FIG. 7 are examples, and the present invention is not limited to this example.

<Related device / work management table>
FIG. 8 shows an example of a related device / work management table. The related device / work management table 2586 includes, for each head mount device ID, a work date, a work time, a location of the head mount device, an ID of one or more devices related to the head mount device, and the one or more Management is performed by associating IDs of operations performed for each of the device IDs. For example, in the related device / work management table 2586 shown in FIG. 8, the head mount device whose head mount device ID is “100aa” has the work date “2016 aa month bb day” and the work time. Is “2 hours”, the location of the head mount device is “(αα, ββ)”, and the IDs of the related devices are “1111”, “2222”, “3333”,. The ID of the work performed on the device whose identification information is “1111” is “AAA”, and the ID of the work performed on the device whose identification information is “2222” is “BBB”. The ID of the work performed on the device whose identification information is “3333” is “CCC”. Here, an example of the location of the head mount device is represented by “(latitude, longitude)”. The head mount device ID, work date, work time, head mount device location, device ID, and work ID shown in FIG. 8 are examples, and the present invention is not limited to this example.

  The selection unit 256 is realized by an instruction from the CPU 202. When the data sharing request transmitted by the head mount device 100a is received by the transmission / reception unit 252, the selection unit 256 acquires identification information and authentication information of the head mount device 100a from the data sharing request. The selection unit 256 refers to the pairing management table 2584 stored in the storage unit 258 and refers to the head mount device of the sharing destination linked to the ID “100aa” of the head mount device 100a attached to the data sharing request. Get an ID. For example, the selection unit 256 acquires “100bb” as the ID of the head mount device of the sharing destination. The selection unit 256 outputs the ID of the shared head mount device and authentication information to the control unit 254.

  The abnormal state determination unit 257 is realized by a command from the CPU 202. The abnormal state determination unit 257 includes, for each of one or a plurality of head mount devices that have established communication links with the information processing device 200, IDs of one or a plurality of devices associated with the ID of the head mount device, In addition, one or both of one or a plurality of work IDs are extracted from the related apparatus / work management table 2586. Then, the abnormal state determination unit 257 includes one or more devices corresponding to the identification information of one or more devices extracted from the related device / work management table 2586, and one or more corresponding to the IDs of one or more tasks. It is determined whether an abnormal state has been detected for either or both of the operations.

  For example, the abnormal state determination unit 257 determines that an abnormal state has been detected when it is detected that an alarm has been issued from either one or a plurality of devices. The abnormal state determination unit 257 determines that an abnormal state has been detected when it is detected that an alarm has been issued in relation to each of the one or more tasks. When detecting the abnormal state, the abnormal state determination unit 257 outputs information indicating that the abnormal state is detected from the transmission / reception unit 252 to the head mount device 100. Furthermore, the abnormal state determination unit 257 may transmit audio information according to the image when outputting information indicating that an abnormal state has been detected from the transmission / reception unit 252 to the head mount device 100. Further, the abnormal state determination unit 257 may transmit the image information so that the image can be displayed on the place where the abnormal state is detected.

  In addition, when the abnormal state determination unit 257 detects that an alarm has been issued from any of the one or more devices or an alarm has been issued in relation to each of the one or more operations, You may make it notify to the head mount apparatus 100, without determining whether it is an abnormal state. For example, when receiving an operation start notification from the head mount device 100, the abnormal state determination unit 257 activates the monitoring mode and monitors the operation state of one or more devices associated with the head mount device 100. . For example, the abnormal state determination unit 257 detects a state change such as power-off or communication disconnection, or a state change of another device connected to the one or more devices. Then, when detecting the state change of the other device, the abnormal state determination unit 257 may notify the head mount device 100 of the content of the state change of the other device from the transmission / reception unit 252. Thereby, the wearer of the head mount apparatus 100 can determine whether or not it is abnormal based on the notified content of the change. Further, when the abnormal state determination unit 257 detects the state change of the other device, the abnormal state determination unit 257 notifies the head mount device 100 of the content of the specific state change among the state changes of the other device. Also good.

Further, the head mount device 100 may record a past image of a predetermined time such as during the past 5 minutes. Then, when detecting the abnormal state, the abnormal state determination unit 257 may notify the head mount device 100, and may instruct the storage location and upload of the recorded data recorded in the head mount device 100. .
The display control unit 259 is realized by a command from the CPU 202 and the display device 209. When the image data transmitted from the head mount device 100 is received by the transmission / reception unit 252, the display control unit 259 displays an image obtained by processing the image data on the display device 209.

  The control unit 254 is realized by a command from the CPU 202. The control unit 254 performs overall control of the information processing apparatus 200. For example, the control unit 254 refers to the head mount device management table 2582 stored in the storage unit 258 and refers to the IP address “192.100” that is the ID of the shared head mount device supplied by the selection unit 256. 168.10.1 ”is acquired. Then, the control unit 254 transmits the data sharing information attached with the authentication information from the transmission / reception unit 252 with the IP address of the head mount device 100b as the destination. In addition, the control unit 254 determines whether or not the sharing destination head mount device is activated, and when activated, the control unit 254 uses the IP sharing address of the head mount device 100b as the destination. You may make it transmit from the transmission / reception part 252. FIG. In this case, the head mount device 100b may transmit an affirmative response to the pairing request transmitted by the head mount device 100a from the wireless communication unit 152b to the information processing device 200 when activated.

<Operation of System According to this Embodiment>
FIG. 9 shows an example of the operation of the system according to the present embodiment.
In step S902, a communication link is established between the head mount device 100a, the head mount device 100b, and the information processing device 200.
In step S904, the abnormal state determination unit 257 of the information processing device 200 determines whether information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100a. When the abnormal state determination unit 257 does not determine that information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100a, the process proceeds to step S910.

Step S906 is executed when the abnormal state determination unit 257 determines that information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100a. In step S906, the abnormal state determination unit 257 transmits information indicating that an abnormal state is detected from the transmission / reception unit 252 to the head mount apparatus 100a.
In step S908, the projection processing unit 154a of the head mount device 100a detects the abnormal state when the wireless communication unit 152a receives information indicating that the abnormal state transmitted by the information processing device 200 has been detected. It is displayed by projecting. Thereafter, the process proceeds to step S910.

  In step S910, the abnormal state determination unit 257 of the information processing device 200 determines whether information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100b. When the abnormal state determination unit 257 of the information processing apparatus 200 does not detect information indicating an abnormal state related to the device or work associated with the head mount device 100b in step S910, the process proceeds to step S904.

Step S912 is executed when it is determined in step S910 that the abnormal state determination unit 257 has detected information indicating an abnormal state related to the device or work associated with the head mount device 100b. In step S912, the abnormal state determination unit 257 transmits information indicating that an abnormal state is detected from the transmission / reception unit 252 to the head mount device 100b.
In step S914, the projection processing unit 154b of the head mount device 100b detects the abnormal state when the wireless communication unit 152b receives information indicating that the abnormal state transmitted by the information processing device 200 has been detected. It is displayed by projecting. Thereafter, the process proceeds to step S904.

In the embodiment described above, when notifying that the abnormal state is detected from the information processing device 200 to the head mount device 100, in addition to notifying in real time with an image, notification may be made with audio information. . Specifically, the sound information transmitted by the information processing device 200 is output by turning on the microphones of the head mount device 100a and the head mount device 100b. Further, bidirectional telephone communication may be performed between the head mount device 100 and the information processing device 200.
Further, when notifying that an abnormal state has been detected, the information processing apparatus 200 may notify all of the head mounted devices that are activated in the specific area that an abnormality has occurred in which device. .

  According to the system according to the present embodiment, the information processing apparatus 200 is associated with the ID of the head mount apparatus 100, and either one of the work date, the work time, the location of the head mount apparatus, the apparatus ID, and the work ID or Remember both. When the information processing apparatus 200 detects information indicating an abnormal state in association with one or both of the apparatus and the work, the information processing apparatus 200 notifies the head mount device 100 that the abnormal state has been detected. Accordingly, it is possible to notify the head mount device 100 worn by the worker that information indicating the abnormal state has been detected without notifying the monitoring center of the abnormal state by telephone. Furthermore, by associating the head mount devices, when the sharing destination is notified of the abnormal state, information indicating the abnormal state can be displayed on the sharing source. As a result, the operator can be notified of the abnormal state in real time without any manual intervention.

<Modification>
1 to 4 can be applied to the system according to this modification. The system according to the present embodiment is different from the system according to the first embodiment in the related apparatus / work management table 2586 stored in the storage unit 258 of the information processing apparatus 200.
FIG. 10 shows an example of the related device / work management table 2586. The related device / work management table 2586 stored in the storage unit 258 of the information processing apparatus 200 according to the present modification is different from the related device / work management table 2586 shown in FIG. The order of work is added.

  For example, in the related device / work management table shown in FIG. 10, the head mount device whose head mount device ID is “100aa” has the work date “2016 aa month bb day” and the work time “2 hours”, the location of the head mount device is “(αα, ββ)”, and the IDs of the related devices are “1111”, “2222”, “3333”,. The ID of the work performed on the device whose identification information is “1111” is “AAA”, and the ID of the work performed on the device whose identification information is “2222” is “BBB”. The ID of the work performed on the device whose identification information is “3333” is “CCC”. Further, the order of the work indicated by the work ID “AAA” is “1”, the order of the work indicated by the work ID “BBB” is “2”, and the work ID is “BBB”. The order of the work indicated by “3” is “3”. The head mount device ID, work date, work time, head mount device location, device ID, work ID, and work order shown in FIG. 10 are merely examples, and the present invention is not limited to this example. For example, the user ID may be associated with the ID of the head mount device. Further, the user ID may be linked to another related person such as a supervisor, a subordinate, or the same department.

Whether the abnormal state determination unit 257 detects an abnormal state for one or both of the IDs of one or a plurality of devices extracted from the related device / work management table 2586 and the IDs of one or a plurality of tasks. Determine whether or not.
For example, the abnormal state determination unit 257 determines that an abnormal state has been detected when it is detected that an alarm has been issued from either one or a plurality of devices. The abnormal state determination unit 257 determines that an abnormal state has been detected when it is detected that an alarm has been issued in relation to each of the one or more tasks. When the abnormal state determination unit 257 detects an abnormal state, the abnormal state determination unit 257 outputs information indicating that the abnormal state has been detected to the transmission / reception unit 252.
On the other hand, if no abnormal state is detected, the abnormal state determination unit 257 obtains the ID of the work corresponding to the next order of the work corresponding to the current work from the related device / work management table 2586. Extract. When the abnormal state determination unit 257 does not detect the abnormal state, the abnormal state determination unit 257 outputs the work ID corresponding to the next order to the transmission / reception unit 252. When outputting the work ID corresponding to the next order to the transmission / reception unit 252, the display start time and the end time may be stored for each work ID.

<Operation of System According to this Embodiment>
FIG. 11 shows an example of the operation of the system according to the present embodiment.
Steps S <b> 1102 to S <b> 1106 can apply steps S <b> 902 to S <b> 906 in FIG. 9. However, if it is not determined in step S1104 that the abnormal state determination unit 257 has detected information indicating an abnormal state related to the device or work associated with the head mount device 100a, the process proceeds to step S1108.
In step S <b> 1108, the abnormal state determination unit 257 of the information processing apparatus 200 extracts a work ID corresponding to the next order of the work currently being performed from the related apparatus / work management table 2586.

In step S1110, the abnormal state determination unit 257 transmits a work ID corresponding to the next order from the transmission / reception unit 252 to the head mount device 100a.
In step S1112, the projection processing unit 154a of the head mount device 100a receives the information indicating that the abnormal state transmitted by the information processing device 200 has been detected or the work ID corresponding to the next order by the wireless communication unit 152a. Then, it is displayed by projecting the work corresponding to the work ID corresponding to the detection of the abnormal state or the next order. Thereafter, the process proceeds to step S1114.

In step S1114, the abnormal state determination unit 257 of the information processing apparatus 200 determines whether information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100b.
Step S1116 is executed when it is determined by the abnormal state determination unit 257 that information indicating an abnormal state has been detected in relation to the device or work associated with the head mount device 100b. In step S1116, the abnormal state determination unit 257 transmits information indicating that an abnormal state is detected from the transmission / reception unit 252 to the head mount device 100b.

Step S1118 is executed when the abnormal state determination unit 257 does not determine that information indicating an abnormal state has been detected in relation to the device or work associated with the head mount device 100b. In step S <b> 1118, the abnormal state determination unit 257 of the information processing apparatus 200 extracts a work ID corresponding to the next order of the work currently being performed from the related apparatus / work management table 2586.
In step S1120, the abnormal state determination unit 257 transmits a work ID corresponding to the next order from the transmission / reception unit 252 to the head mount device 100b.
In step S1122, the projection processing unit 154b of the head mount device 100b receives the information indicating that the abnormal state transmitted by the information processing device 200 is detected or the work ID corresponding to the next order is received by the wireless communication unit 152b. Then, it is displayed by projecting the work corresponding to the work ID corresponding to the detection of the abnormal state or the next order. Thereafter, the process proceeds to step S1104.
In this modification, a case has been described in which the order of work is notified when the device IDs are different, but the present invention is not limited to this example. For example, a plurality of work IDs may be associated with an ID of a certain device, and the order of the work may be notified for the plurality of works. Furthermore, you may make it notify the content corresponding to work ID.

  According to the system according to the present modification, the information processing apparatus 200 stores an apparatus or work in association with the head mount apparatus 100. Furthermore, the information processing apparatus 200 stores the order of work in association with the apparatus or work. Then, the information processing device 200 notifies the head mount device 100 of work in the next order of the work currently being performed when information indicating an abnormal state related to the device or work is not detected. When the information indicating the abnormal state is detected in relation to the work, the head mount device 100 is stopped from reporting the work in the next order of the work order currently being performed. Accordingly, the next work can be notified to the head mount device 100 worn by the worker without notifying the next work from the monitoring center by telephone.

<Second Embodiment>
1-2 can be applied to the system according to the present embodiment. The system according to the present embodiment includes an information processing device 300 instead of the information processing device 200, as compared with the system according to the first embodiment. FIG. 3 can be applied to the hardware configuration of the information processing apparatus 300.

<Functional Configuration of System According to this Embodiment>
FIG. 12 is a functional block diagram of the head mount device 100a, the head mount device 100b, and the information processing device 300 that constitute the system of the present embodiment.

<Functional configuration of information processing apparatus>
As shown in FIG. 12, the information processing apparatus 300 includes a transmission / reception unit 352, a control unit 354, a selection unit 356, an abnormal state determination unit 357, a storage unit 358, a display control unit 359, and the above-described components. A bus line 360 such as an address bus or a data bus. Each of these units operates in response to an instruction from the CPU 202 that executes the program 2062 expanded from the nonvolatile memory 206 to the memory 204 in the information processing apparatus 300 in any one of the components illustrated in FIG. It is a function realized by this.

<Each functional configuration of information processing apparatus>
Each functional configuration of the information processing apparatus 300 will be described in detail with reference to FIGS. 3 and 12. In the following, in describing each functional configuration of the information processing apparatus 300, among the components illustrated in FIG. 12, main hardware for realizing each functional configuration of the information processing apparatus 300 is described. Explain the relationship.
The transmission / reception unit 352, the control unit 354, the selection unit 356, and the display control unit 359 can apply the transmission / reception unit 252, the control unit 254, the selection unit 256, and the display control unit 259 of the information processing apparatus 200 illustrated in FIG.

  The storage unit 358 is realized by the nonvolatile memory 206. The storage unit 358 stores a head mount device management table 3582, a pairing management table 3584, a related device / work management table 3586, and an abnormal state / corresponding procedure correspondence table 3588. The head mount device management table 2582, the pairing management table 2584, and the related device / work management table 2586 described above can be applied to the head mount device management table 3582, the pairing management table 3584, and the related device / work management table 3586.

<Abnormal state / response procedure correspondence table>
FIG. 13 shows an example of the abnormal state / corresponding procedure correspondence table. The abnormal state / corresponding procedure correspondence table 3588 associates one or a plurality of abnormal state IDs with one or a plurality of abnormal state correspondence procedures represented by each of the one or plural abnormal state IDs. It is a table. For example, in the abnormal state correspondence table shown in FIG. 13, the response procedure for the abnormal state whose abnormal state ID is “11AA” is “unplug the xxx cable”. The abnormal state ID and the response procedure shown in FIG. 13 are examples, and are not limited to this example.

  The abnormal state determination unit 357 is realized by a command from the CPU 202. The abnormal state determination unit 357 includes, for each of one or a plurality of head mount devices that have established communication links with the information processing device 300, the IDs of one or a plurality of devices associated with the ID of the head mount device, In addition, one or both of one or a plurality of work IDs are extracted from the related apparatus / work management table 3586. Then, the abnormal state determination unit 357 includes one or a plurality of devices corresponding to the ID of one or a plurality of devices extracted from the related device / work management table 3586, and one or a plurality of devices corresponding to the ID of the one or a plurality of operations. It is determined whether an abnormal state has been detected for one or both of the operations.

For example, the abnormal state determination unit 357 determines that an abnormal state has been detected when it is detected that an alarm has been issued from the one or more devices. The abnormal state determination unit 357 determines that an abnormal state has been detected when it is detected that an alarm has been issued in relation to the one or more tasks.
Furthermore, the abnormal state determination unit 357 extracts a corresponding procedure corresponding to the detected abnormal state ID from the abnormal state / corresponding procedure correspondence table 3588. When the abnormal state is detected, the abnormal state determination unit 357 outputs information indicating that the abnormal state is detected and information indicating a procedure for handling the abnormal state to the transmission / reception unit 352.

<Operation of System According to this Embodiment>
FIG. 14 shows an example of the operation of the system according to the present embodiment.
Steps S <b> 1402 to S <b> 1404 can apply steps S <b> 902 to S <b> 904 in FIG. 9. However, if it is not determined in step S1404 that the abnormal state determination unit 257 has detected information indicating an abnormal state related to the device or work associated with the head mount device 100a, the process proceeds to step S1412.

Step S1406 is executed when the abnormal state determination unit 357 determines that information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100a. In step S1406, the abnormal state determination unit 357 of the information processing device 300 extracts a corresponding procedure corresponding to the abnormal state ID detected in step S1404 from the abnormal state / corresponding procedure correspondence table 3588.
In step S1408, the abnormal state determination unit 357 transmits information indicating that an abnormal state has been detected and information indicating the handling procedure from the transmission / reception unit 352 to the head mount device 100a.

In step S1410, the projection processing unit 154a of the head mount device 100a receives the information indicating that the abnormal state transmitted by the information processing device 300 is detected and the information indicating the corresponding procedure by the wireless communication unit 152a. And displaying that the abnormal state is detected and the corresponding procedure. Thereafter, the process proceeds to step S1412.
In step S1412, the abnormal state determination unit 357 of the information processing device 300 determines whether information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100b. When the abnormal state determination unit 357 of the information processing device 300 does not detect information indicating an abnormal state related to the device or work associated with the head mount device 100b in step S1412, the process proceeds to step S1404.

  Step S1414 is executed when it is determined by the abnormal state determination unit 357 that information indicating an abnormal state is detected in relation to the device or work associated with the head mount device 100b. In step S1414, the abnormal state determination unit 357 of the information processing device 300 extracts the corresponding procedure corresponding to the abnormal state ID detected in step S1412 from the abnormal state / corresponding procedure correspondence table 3588.

In step S1416, the abnormal state determination unit 357 transmits information indicating that an abnormal state has been detected and information indicating the handling procedure from the transmission / reception unit 352 to the head mount device 100b.
In step S1418, the projection processing unit 154b of the head mount device 100b receives the information indicating that the abnormal state transmitted by the information processing device 300 is detected and the information indicating the corresponding procedure by the wireless communication unit 152b. And displaying that the abnormal state is detected and the corresponding procedure. Thereafter, the process proceeds to step S1404.

  In step S1408, the information processing apparatus 200 may transmit information indicating that an abnormal state has been detected and information indicating a contact information such as a telephone number to the head mount apparatus 100a. In this case, the head mount device 100a displays the contact address transmitted by the information processing device 200. The wearer of the head mount device 100a may make a call to the contact from the head mount device 100a and obtain the corresponding procedure from the contact by, for example, saying “Call”.

  According to the system according to the present embodiment, the information processing device 300 stores the device or work in association with the head mount device 100. Furthermore, the information processing apparatus 300 stores a response procedure in association with the abnormal state. When the information processing apparatus 300 detects information indicating an abnormal state related to the apparatus or the work, the information processing apparatus 300 acquires a response procedure for the abnormal state, and detects that the abnormal state is detected and the corresponding procedure. Notify the mounting apparatus 100. As a result, it is possible to notify the head mount device 100 worn by the worker that the information indicating the abnormal state has been detected without notifying the abnormal procedure by telephone from the monitoring center and not reporting the corresponding procedure. , The corresponding procedure can also be notified. Accordingly, it is possible to notify the worker of the abnormal state in real time without any manual intervention and to notify the corresponding procedure.

  As mentioned above, although embodiment of this invention and its modification were demonstrated, these embodiment and its modification are shown as an example, and are not intending limiting the range of invention. These embodiments and modifications thereof can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and at the same time, are included in the invention described in the claims and the equivalents thereof.

Each of the above devices has a computer inside. The process of each device described above is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer reading and executing the program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  In the embodiment described above, the transmission / reception unit 252 is an example of a first communication unit, the abnormal state determination units 257 and 357 are examples of a determination unit and an acquisition unit, the projection processing unit is an example of a display unit, and wireless The communication unit is an example of a second communication unit, and the storage units 258 and 358 are examples of a first storage unit and a second storage unit.

  DESCRIPTION OF SYMBOLS 20 ... Communication network, 70 ... Access point, 100a, 100b ... Head mount apparatus, 152a, 152b ... Wireless communication part, 154a, 154b ... Projection process part, 156a, 156b ... Imaging process part, 160a, 160b ... Connection process part, 162a, 162b ... control unit, 164a, 164b ... storage unit, 200, 300 ... information processing device, 252, 352 ... transmission / reception unit, 254, 354 ... control unit, 256, 356 ... selection unit, 257, 357 ... abnormal state determination Unit, 258, 358 ... storage unit, 259, 359 ... display control unit

Claims (7)

A system comprising a head mount device and an information processing device,
The information processing apparatus includes:
A first communication unit that communicates with the head mount device;
A determination unit that determines whether or not information indicating an abnormal state is detected in association with a device or work associated with the head mount device, and
The first communication unit transmits information representing a determination result detected by the determination unit to the head mount device,
The head mount device is:
A display unit;
A second communication unit that communicates with the information processing apparatus,
The second communication unit receives the result of the determination transmitted by the information processing apparatus,
The said display part displays the information showing the result of the said determination which the said 2nd communication part received. The information processing apparatus includes:
A first storage unit that stores a procedure associated with a device associated with the head mount device or information indicating an abnormal state associated with the work;
An acquisition unit for acquiring a response procedure when information indicating the abnormal state is detected from a response procedure stored in the first storage unit when the determination unit detects information indicating an abnormal state; Prepared,
The first communication unit transmits information representing the handling procedure acquired by the acquisition unit to the head mount device,
The second communication unit receives information representing the handling procedure transmitted by the information processing apparatus,
The system according to claim 1, wherein the display unit displays the handling procedure received by the second communication unit. The information processing apparatus includes:
A second storage unit that stores each of the plurality of pieces of work identification information in association with the order of the work;
An acquisition unit that acquires, from the work order stored in the second storage unit, work identification information corresponding to the next order of the work order in which the information indicating the abnormal state is not detected by the determination unit. And
The first communication unit transmits work identification information corresponding to the next order acquired by the acquisition unit to the head mount device,
The second communication unit receives work identification information corresponding to the next order transmitted by the information processing apparatus,
The system according to claim 1, wherein the display unit displays work corresponding to work identification information corresponding to the next order received by the second communication unit.   The system according to claim 3, wherein the first communication unit stops transmitting the work identification information when information indicating the abnormal state is detected by the determination unit. A communication unit for communicating with the head mount device;
A determination unit that determines whether or not information indicating an abnormal state is detected in association with a device or work associated with the head mount device, and
The information processing apparatus, wherein the communication unit transmits information representing a determination result detected by the determination unit to the head mount device. A display unit;
A communication unit that communicates with the information processing device,
The display unit displays information representing a result of determination as to whether or not information indicating an abnormal state has been detected in association with a device or work associated with the head mounted device received by the communication unit. . A method executed by a system including a head mount device and an information processing device,
Determining whether the information processing device has detected information indicating an abnormal state in relation to a device or work associated with the head mount device; and
The information processing apparatus transmitting information representing the result of the determination to the head mount apparatus;
The head mount device receiving information representing the result of the determination;
The head mounting device includes a step of displaying information representing a result of the determination.

Images