JP2017175354A - System, information processing device, head mounting device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To verify, when a user works using a head mounting device, the work in real time.SOLUTION: An information processing device includes: an identification information management unit which registers identification information on a plurality of head mounting devices and identification information on one or a plurality of terminal devices to be communication targets with each of the plurality of head mounting devices in association with each other; and a communication unit which, when the identification information on the head mounting devices to be transmitted by the head mounting devices is registered in the identification information management unit, transmits a request for sharing data acquired by the head mounting devices to an activated terminal device which is associated with the identification information on the head mounting devices.SELECTED DRAWING: Figure 8

Description

  Embodiments described herein relate generally to a system, an information processing apparatus, a head mount apparatus, and a program.

In recent years, devices such as sensors have been miniaturized, and wearable head mounted displays (HMDs) such as smart glasses have been developed. In a system using a head-mounted display, there is known a system that displays information by projecting information on a virtual screen such as a spectacle lens, a glass or plastic in front of sunglasses or goggles.
Regarding a technique for performing a fault repair work using a head-mounted display, a technique is known in which even if an unexpected fault occurs, a technique for guiding an optimal fault repair work and providing work support (for example, see Patent Document 1). reference).

JP 2011-248860 A

When a failure occurs in a device or the like, a worker who performs repair work works by bringing a work instruction sheet, a work check table, a work manual, or the like to the place where the failure occurs. For this reason, there is a risk of forgetting to bring the document to the place where the failure occurred or the document being lost. Furthermore, when a work is performed by a beginner who is unfamiliar with the work, a work mistake may occur. In this case, it is preferable that the work can be confirmed by an expert in real time. The same applies to normal work, not only when a failure occurs in a device or the like.
The present invention has been made to solve the above-described problem, and an object of the present invention is to confirm the work in real time when the user performs the work using the head mount device.

(1) According to one embodiment of the present invention, an information processing apparatus includes one or more identification information of one or more head mount devices and one or more target objects of communication between the one or more head mount devices. An identification information management unit that associates and registers identification information of a terminal device, and the head mount device when the identification information of the head mount device transmitted by the head mount device is registered in the identification information management unit A first communication unit that transmits a request to share the data acquired by the head mount device to a terminal device that is associated with the identification information and is activated, and the head mount device includes: A second communication unit that communicates with one communication unit, and a first control unit that performs a process of sharing data acquired by the head mount device with a terminal device, Communication unit, when the processing for sharing the data by the first control unit is executed, transmits the identification information of the head-mounted device to the information processing apparatus is a system.

(2) One aspect of the present invention is the system according to (1), in which the terminal device receives the third communication unit that communicates with the first communication unit, and the third communication unit. In response to a request to share the received data, it is connected to both or one of the information processing device and the head mount device, and performs communication control between the information processing device and both or one of the head mount devices. And a second control unit.

(3) One aspect of the present invention is the system according to (1) or (2) above, in which the identification information management unit is configured to provide a terminal device for each of the identification information of the one or more head mounted devices. The head mount device includes a positioning unit that performs positioning of the head mount device, and the second communication unit is positioned by the positioning unit. The obtained position information is transmitted to the information processing apparatus, and the first communication unit moves the head mount within a predetermined range from position information corresponding to the identification information of the head mount apparatus transmitted by the head mount apparatus. When the position information transmitted by a device is included, the system transmits a request for sharing data acquired by the head mount device to the terminal device.

(4) One aspect of the present invention is the system according to any one of (1) to (3), wherein the identification information management unit includes identification information of the one or more head mount devices, Time when each of the one or a plurality of head mount devices is registered in association with information indicating the use time, and the first communication unit receives the identification information of the head mount device from the head mount device Is acquired by the head mount device to the terminal device when the information indicating the same as or used within the time corresponding to the identification information of the head mount device stored in the identification information management unit. This is a system that sends a request to share data.

(5) According to one aspect of the present invention, in the system according to any one of (1) to (4), the information processing apparatus identifies a terminal device that is authorized to communicate with a head mounted device. A communication authority management unit for registering information to be registered and information for identifying the head mount device transmitted by the head mount device are registered in the identification information management unit. And a determination unit that determines that the terminal device shares data acquired by the head mount device when the terminal device that is attached and activated is registered in the communication authority management unit, When the determination unit determines that the terminal unit shares data acquired by the head mount device, the first communication unit Sending a request to share data that is acquired by the the location head-mounted device, a system.

(6) According to one aspect of the present invention, identification information of one or a plurality of head mount devices and identification information of one or a plurality of terminal devices to be communicated with each of the one or more head mount devices And the identification information management unit that registers the identification information of the head mount device transmitted by the head mount device in the identification information management unit. And a communication unit that transmits a request for sharing the data acquired by the head mount device to the activated terminal device.

(7) According to one aspect of the present invention, in the head mount device, a communication unit that communicates with the information processing device, and a control unit that performs processing for sharing data acquired by the head mount device with the terminal device The communication unit is a head mount device that transmits identification information of the head mount device to the information processing device when processing for sharing the data is performed by the control unit.

(8) According to one embodiment of the present invention, information identifying the head mount device transmitted by the head mount device to the computer of the information processing device includes identification information of one or more head mount devices, and the one or more head mount devices. A step of determining whether or not the identification information is registered in an identification information management unit that associates and registers identification information of one or a plurality of terminal devices to be communicated with each of the head mounted devices; and the identification information A step of transmitting a request to share the data acquired by the head mount device to the terminal device linked to the identification information of the head mount device and being activated when registered in the management unit; A program to be executed.

(9) According to one aspect of the present invention, there is provided a step of causing a computer of a head mount device to perform processing for sharing data acquired by the head mount device with a terminal device; and And a step of transmitting identification information of the head mount device to the information processing device when processing to be shared is performed.

  According to the embodiment of the present invention, when a user performs an operation using the head mount device, the operation can be confirmed in real time.

It is a figure which shows the system which concerns on this embodiment. It is a figure which shows an example of the hardware constitutions of the head mount apparatus which concerns on this embodiment. It is a figure which shows an example of the hardware constitutions of the information processing apparatus which concerns on this embodiment. It is a functional block diagram showing an example of a system concerning this embodiment. It is a figure which shows an example of the image projected by the head mount apparatus which concerns on this embodiment. It is a figure which shows an example of a head mount apparatus management table. It is a figure which shows an example of a pairing management table. It is a sequence diagram which shows an example of operation | movement of the system which concerns on this embodiment. It is a figure which shows an example of the hardware constitutions of the head mount apparatus which concerns on this embodiment. It is a functional block diagram showing an example of a system concerning this embodiment. It is a figure which shows an example of a pairing management table. It is a sequence diagram which shows an example of operation | movement of the system which concerns on this embodiment. It is a figure which shows an example of a communication authority management table.

<First Embodiment>
<System>
The system according to the present embodiment displays an image captured by the imaging unit of the head mount device on the information processing device. Further, the system transmits data obtained by, for example, imaging by one head mount device to the other head mount devices among a plurality of head mount devices. When the other head mounting apparatus receives the data transmitted by the one head mounting apparatus, the other head mounting apparatus processes and projects the data. Thereby, data can be shared in real time among a plurality of head mount devices.

FIG. 1 shows an example of a system. The system includes a head mount device 100a, a head mount device 100b, and an information processing device 200. In FIG. 1, two head mount devices are illustrated as an example, but the number of head mount devices may be one or three or more. Further, a terminal device may be provided instead of the head mount device 100b, or a terminal device may be provided together with the head mount device 100b. An example of the terminal device is a mobile phone, a smartphone, a tablet terminal, or the like. Hereinafter, as an example, the case where the system includes the head mount device 100b will be described.
The information processing apparatus 200 is connected to a communication network 20 such as the Internet. The head mount device 100 a and the head mount device 100 b are connected to the communication network 20 via the access point 70.

  In the system according to the present embodiment, as an example, a case will be described in which data obtained by the head mount device 100a is shared between the information processing device 200 and the head mount device 100b. An example of the head mount device 100a and the head mount device 100b is mounted on the operator's head. It is configured such that an image that can be visually recognized by an operator can be presented to an operator wearing the head mount device 100a and the head mount device 100b on the head. For example, the head mount device 100a and the head mount device 100b are transmissive head mount displays. In the example shown in FIG. 1, the head mount device 100 is a glasses-type transmissive head mount display. The head mount device 100a and the head mount device 100b may be a head-mounted head mount display having a shape other than the glasses type. The head mount device 100a and the head mount device 100b display information on a virtual screen and display the information superimposed on the background that is the surrounding environment, so that the user can check various information while viewing the background. That is, the head mount device 100a and the head mount device 100b superimpose the information displayed on the virtual screen and the background behind the virtual screen. The head mount device 100a transmits information for identifying the head mount device 100a and authentication information to the information processing device 200.

The information processing apparatus 200 stores the identification information of one or more head mount devices that share data acquired by the head mount device in association with each of the identification information of the plurality of head mount devices. Furthermore, the information processing device 200 stores an IP address for each of the plurality of head mount devices. When the information processing apparatus 200 receives the identification information and authentication information of the head mount apparatus 100a transmitted by the head mount apparatus 100a, the head associated with the identification information of the head mount apparatus 100a as a pairing target The head mount device 100b corresponding to the identification information of the mount device is specified. Furthermore, the information processing apparatus 200 acquires the IP address of the identified head mount apparatus 100b.
The information processing apparatus 200 makes a connection request by transmitting the authentication information transmitted by the head mount apparatus 100a to the head mount apparatus 100b. When the head mount device 100b receives the authentication information transmitted by the information processing device 200, the head mount device 100b establishes a communication link with the head mount device 100a using the authentication information.
Hereinafter, when the head mount device 100a and the head mount device 100b are not distinguished, they are referred to as the head mount device 100.

  In the system according to the present embodiment, the head mount device 100 performs data communication with the information processing device 200. For example, the head mount device 100a transmits identification information and authentication information of the head mount device 100a to the information processing device 200, and the information processing device 200 transmits authentication information to the head mount device 100b. Furthermore, the head mount device 100a performs wireless communication with the head mount device 100b. For example, communication may be performed between the head mount device 100a and the head mount device 100b in accordance with a near field communication (NFC) standard such as Bluetooth (registered trademark).

<Configuration of head mount device>
FIG. 2 is a diagram illustrating an example of a head mount device according to the present embodiment.
The head mount device 100 includes a CPU (Central Processing Unit) 102, a ROM (Read Only Memory) 104, a RAM (Random Access Memory) 106, an operation unit 107, a nonvolatile memory 108, and a communication I / F unit 110. An imaging unit 112, a projection unit 114, and an internal bus 116. These are arranged in a housing such as a temple of the head mount device 100.

The CPU 102 reads a program 1082 and the like for controlling the operation of the head mount apparatus 100 from the nonvolatile memory 108, expands it in the RAM 106, and executes it.
The operation unit 107 is an input device that receives a user operation, and includes a pointing device such as a touch panel, buttons, a dial, a touch sensor, a touch pad, and the like. In the present embodiment, as an example, the case where the operation unit 107 forms a button will be described.
The non-volatile memory 108 includes a flash memory, an HDD (Hard Disc Drive), an SSD (Solid State Drive), an SD (Secure Digital) card, and the like, and is disposed in the casing of the head mount device 100. The nonvolatile memory 108 associates and stores object data to be projected on the virtual screen by the projection unit 114, setting information for specifying the positions of both eyes, the position of the virtual screen, and the like.
The communication I / F unit 110 is configured by a communication module such as an NFC module or a wireless LAN module, and is disposed in the casing of the head mount device 100. The communication I / F unit 110 communicates with an external device by NFC or wireless LAN.
The imaging unit 112 is configured by a CCD (Charge Coupled Devices) camera, a CMOS (Complementary Metal Oxide Semiconductor) camera, or the like, and is fixed to a housing such as a vine (temple) of the head mount device 100. The imaging unit 112 outputs, to the CPU 102, image data obtained by imaging the position or movement of the pupils of both eyes of the user directly or using a half mirror or the like.
The projection unit 114 is disposed in the housing of the head mount device 100. The projection unit 114 displays the object on the virtual screen by processing the image data of the object supplied by the CPU 102.

<Configuration of information processing apparatus>
FIG. 3 shows an example of a hardware configuration of the information processing apparatus 200 according to the present embodiment. The information processing apparatus 200 includes a CPU 202, a memory 204, a nonvolatile memory 206, a communication I / F 208, a display device 209, and an internal bus 210.
For example, the CPU 202 executes a program stored in the non-volatile memory 206 and controls each unit of the information processing apparatus 200 using the memory 204 as a work memory.
The memory 204 is configured by a RAM such as a volatile memory using a semiconductor element. The memory 204 is used as a work memory for the CPU 202.
The nonvolatile memory 206 is configured by, for example, a hard disk (HD), a ROM, or the like, and stores a program executed by the CPU 202.

The communication I / F 208 is an interface for communicating with an external device and transmitting / receiving various data such as files and commands.
The display device 209 is configured by a liquid crystal display, an organic EL display, or the like, for example, and displays images such as characters and photographs.
The internal bus 210 connects the CPU 202, the memory 204, the nonvolatile memory 206, the communication I / F 208, and the display device 209 to each other. Each unit connected to the internal bus 210 can exchange data with each other via the internal bus 210.

<Functional Configuration of System According to this Embodiment>
Next, the functional configuration of this embodiment will be described. FIG. 4 is a functional block diagram of the head mount device 100a, the head mount device 100b, and the information processing device 200 that constitute the system of the present embodiment. In FIG. 4, the head mount device 100 a, the head mount device 100 b, and the information processing device 200 are connected so that data communication can be performed via the communication network 20.

<Functional configuration of head mount device>
The head mount device 100a includes a wireless communication unit 152a, a projection processing unit 154a, an imaging processing unit 156a, a connection processing unit 160a, a control unit 162a, a storage unit 164a, and each of the above components as shown in FIG. A bus line 166a such as an address bus or a data bus for connection is provided. Each of these units is a function realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 102 that executes the program 1082 expanded from the nonvolatile memory 108 onto the RAM 106. is there.

<Functional configuration of the head mount device>
Each functional configuration of the head mount device 100a will be described in detail with reference to FIGS. In the following, in describing each functional configuration of the head mount device 100a, among the components illustrated in FIG. 4, the main hardware for realizing each functional configuration of the head mount device 100a is described. Explain the relationship.
The wireless communication unit 152a of the head mount device 100a illustrated in FIG. 4 is realized by a command from the CPU 102 and the communication I / F 110. The wireless communication unit 152 a transmits and receives various data to and from the information processing apparatus 200 via the access point 70 and the communication network 20. Further, the wireless communication unit 152a transmits and receives various data to and from the head mount device 100b. For example, the wireless communication unit 152a transmits and receives various data to and from the information processing apparatus 200 via the access point 70 and the communication network 20 according to the wireless LAN, and performs various kinds of data to and from the head mount apparatus 100b. Send and receive data.

The projection processing unit 154a is realized by a command from the CPU 102 and the projection unit 114. The projection processing unit 154a displays the object on the virtual screen based on the object data supplied by the control unit 162a.
FIG. 5 shows an example in which an object is displayed on the virtual screen by the projection processing unit 154a. Here, as an example, an object projected by the head mount device 100a attached by a worker who performs a repair work when a failure occurs in a device or the like is shown. In the example shown in FIG. 5, objects such as “in operation! Attention”, “work target”, “non-target device”, and “non-target device (YYYYMM erroneous work occurrence location)” are superimposed on the background of the device group or the like. Is displayed. Accordingly, the user can perform work while confirming the state of each device included in the device group. Further, the projection processing unit 154a may project a document such as a work instruction sheet, a work check table, and a work manual as an object. As a result, the worker can perform work without bringing a work instruction sheet, a work check table, a work manual, or the like to the place where the failure has occurred. Returning to FIG.

The imaging processing unit 156a is realized by a command from the CPU 102 and the imaging unit 112. The imaging processing unit 156a outputs image data obtained by imaging the positions and movements of the eyes of both eyes of the user wearing the head mount device 100a to the control unit 162a.
The connection processing unit 160a is realized by a command from the CPU 102 and the communication I / F 110. The connection processing unit 160a holds authentication information used for pairing between the head mount device 100a and the head mount device 100b. The connection processing unit 160a may hold authentication information in advance, or when the user performs an operation for sharing data between the head mount device 100a and the head mount device 100b on the operation unit 107. You may make it produce | generate and hold | maintain.
The connection processing unit 160a creates a signal for requesting data sharing (hereinafter referred to as “data sharing request”). The data sharing request is accompanied by identification information of the head mount device 100a (ID of the head mount device 100a) and authentication information. The connection processing unit 160a transmits a data sharing request from the wireless communication unit 152a to the information processing device 200.
Further, when the connection request transmitted from the head mount device 100b is received by the wireless communication unit 152a, the connection processing unit 160a determines whether the authentication information included in the connection request matches the authentication information held. Determine whether. If the connection processing unit 160a determines that the authentication information included in the connection request matches the authentication information held, the connection processing unit 160a performs an encryption process using a link key, and the head mount device 100a and the head mount device A communication link with 100b is established. If the connection processing unit 160a determines that the authentication information included in the connection request and the held authentication information do not match, the connection processing unit 160a does not establish a communication link between the head mount device 100a and the head mount device 100b.

The control unit 162a is realized by a command from the CPU 102. The control unit 162a controls the entire head mount device 100a. For example, when the user performs an operation for sharing data between the head mount device 100a and the head mount device 100b on the operation unit 107, the control unit 162a creates a data sharing request to the connection processing unit 160a. Order to do. In addition, after the communication link is established between the wireless communication unit 152a and the head mount device 100b, the control unit 162a captures the positions and movements of the eyes of both eyes of the user wearing the head mount device 100a. Is transmitted from the wireless communication unit 152a to the information processing apparatus 200 and the head mount apparatus 100b.
The storage unit 164a is realized by the nonvolatile memory 108. The storage unit 164a stores the ID 1642a of the head mount device 100a.

The head mount device 100b includes a wireless communication unit 152b, a projection processing unit 154b, an imaging processing unit 156b, a connection processing unit 160b, a control unit 162b, a storage unit 164b, and the above-described components as shown in FIG. A bus line 166b such as an address bus or a data bus for connection is provided. Each of these units is a function realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 102 that executes the program 1082 expanded from the nonvolatile memory 108 onto the RAM 106. is there.
The wireless communication unit 152b, the projection processing unit 154b, and the imaging processing unit 156b can apply the wireless communication unit 152a, the projection processing unit 154a, and the imaging processing unit 156a described above.
When the wireless communication unit 152b receives data sharing information transmitted from the information processing apparatus 200, the connection processing unit 160b acquires authentication information included in the data sharing information. Then, the connection processing unit 160a attaches the authentication information to the connection request and transmits it from the wireless communication unit 152b to the head mount device 100a.
The controller 162b performs overall control of the head mount device 100b. After the communication link is established between the wireless communication unit 152b and the head mount device 100a, the control unit 162b outputs the data received by the wireless communication unit 152b to the projection processing unit 154b.
The storage unit 164b stores the ID 1642b of the head mount device 100b.

<Functional configuration of information processing apparatus>
The information processing apparatus 200 has an address for electrically connecting the transmission / reception unit 252, the control unit 254, the selection unit 256, the storage unit 258, the display control unit 259, and the above components as illustrated in FIG. 4. A bus line 260 such as a bus or a data bus is provided. Each of these units operates in response to an instruction from the CPU 202 that executes the program 2062 expanded from the nonvolatile memory 206 to the memory 204 in the information processing apparatus 200 in any one of the components illustrated in FIG. It is a function realized by this.

<Each functional configuration of information processing apparatus>
Each functional configuration of the information processing apparatus 200 will be described in detail with reference to FIGS. 3 and 4. In the following description, in describing each functional configuration of the information processing apparatus 200, it is assumed that the main hardware for realizing each functional configuration of the information processing apparatus 200 among the respective constituent elements illustrated in FIG. Explain the relationship.
The transmission / reception unit 252 of the information processing apparatus 200 illustrated in FIG. 4 is realized by a command from the CPU 202 and a communication I / F 208. The transmission / reception unit 252 transmits / receives various data to / from the head mount device 100a or the head mount device 100b via the communication network 20 and the access point 70. For example, the transmission / reception unit 252 receives image data transmitted by the head mount device 100a.
The storage unit 258 is realized by the nonvolatile memory 206. The storage unit 258 stores a head mount device management table 2582 and a pairing management table 2584.

<Head mount device management table>
FIG. 6 shows an example of the head mount device management table. The head mount device management table 2582 manages the IP address of the head mount device in association with each head mount device ID. For example, the head mount device management table 2582 shown in FIG. 6 indicates that the IP address of the head mount device with the head mount device ID “100aa” is “192.168.10.10.”. Has been. The ID and IP address of the head mount device shown in FIG. 6 are examples, and the present invention is not limited to this example.

<Pairing management table>
FIG. 7 shows an example of the pairing management table. The pairing management table 2584 manages, for each head mount device ID, the ID of the head mount device to which the data is shared and the time to share the data in association with each other. For example, in the pairing management table 2584 shown in FIG. 7, the head mount device whose ID of the head mount device is “100aa” has the ID of the head mount device to which data is shared is “100bb”, It is shown that the time for sharing data is “9 o'clock to 12 o'clock”. The ID of the head mount device, the ID of the head mount device that shares the data, and the time for sharing the data shown in FIG. 7 are examples, and are not limited to this example.
Hereinafter, it is assumed that the ID of the head mount device 100a is “100aa” and the ID of the head mount device 100b is “100bb”, and the description will be continued by returning to FIG.

  The selection unit 256 is realized by an instruction from the CPU 202. When the data sharing request transmitted by the head mount device 100a is received by the transmission / reception unit 252, the selection unit 256 acquires identification information and authentication information of the head mount device 100a from the data sharing request. The selection unit 256 refers to the pairing management table 2584 stored in the storage unit 258, and shares the head mount device that is linked to the identification information “100aa” of the head mount device 100a attached to the data sharing request. Get the ID. The selection unit 256 outputs the ID of the head mount device to be paired and the authentication information to the control unit 254. Further, the selection unit 256 refers to the pairing management table 2584 stored in the storage unit 258, and based on the time when the data sharing request is received by the transmission / reception unit 252, the time is the identification information “ When the data associated with “100aa” is coincident with or included in the time of sharing, the ID of the head mount device of the sharing destination associated with the identification information “100aa” of the head mount device 100a is acquired. You may make it do. For example, when the time when the data sharing request is received by the transmission / reception unit 252 is 9:00, the selection unit 256 is a time for sharing the data associated with the identification information “100aa” of the head mount device 100a. Since “9 o'clock to 12 o'clock” is included, “100bb” is acquired as the ID of the head mount device to be paired. The selection unit 256 outputs the ID of the head mount device to be paired and the authentication information to the control unit 254.

The display control unit 259 is realized by a command from the CPU 202 and the display device 209. The display control unit 259 displays an image obtained by processing the image data received by the transmission / reception unit 252 on the display device 209.
The control unit 254 is realized by a command from the CPU 202. The control unit 254 performs overall control of the information processing apparatus 200. For example, the control unit 254 refers to the head mount device management table 2582 stored in the storage unit 258, and the IP address “192” of “100bb” that is the ID of the head mount device to be paired supplied by the selection unit 256. .168.11.11 ”. Then, the control unit 254 transmits the data sharing information attached with the authentication information from the transmission / reception unit 252 with the IP address of the head mount device 100b as the destination. Further, the control unit 254 determines whether or not the head mount device to be paired is activated, and when it is activated, the data sharing information accompanied by the authentication information is sent to the IP address of the head mount device 100b. It is also possible to transmit from the transmission / reception unit 252. In this case, the head mount device 100b may transmit an affirmative response to the pairing request transmitted by the head mount device 100a from the wireless communication unit 152b to the information processing device 200 when activated.

<Operation of System According to this Embodiment>
FIG. 8 shows an example of the operation of the system according to the present embodiment. Here, the operation after the communication link is established between the head mount device 100a and the information processing device 200 and between the head mount device 100b and the information processing device 200 will be described.
In step S802, the connection processing unit 160a of the head mount device 100a creates a data sharing request with the ID of the head mount device 100a and the authentication information held in the connection processing unit 160a.
In step S804, the wireless communication unit 152a of the head mount device 100a transmits a data sharing request to the information processing device 200.
In step S806, the transmission / reception unit 252 of the information processing device 200 receives the data sharing request transmitted by the head mount device 100a. The selection unit 256 of the information processing device 200 acquires the ID of the head mount device 100a attached to the data sharing request. The selection unit 256 refers to the pairing management table 2584 in the storage unit 258, and acquires the ID of the shared head mount device linked to the ID of the head mount device 100a.

In step S808, the control unit 254 of the information processing device 200 refers to the head mount device management table 2582 and acquires an IP address associated with the ID of the head mount device that is the sharing destination. Further, the control unit 254 acquires authentication information attached to the data sharing request. The control unit 254 uses the IP address linked to the ID of the head mount device as the sharing destination as the destination, and creates data sharing information accompanied with the authentication information.
In step S810, the transmission / reception unit 252 of the information processing apparatus 200 transmits the data sharing information created by the control unit 254. Here, before transmitting the data sharing information, the information processing apparatus 200 transmits a pairing request to the head mount apparatus 100b that is the sharing destination, and receives a response to the pairing request to activate the head mount apparatus 100b. You may judge that it is in the state. The information processing apparatus 200 may determine that data sharing is possible when the head mount apparatus 100b is activated. Then, the information processing apparatus 200 transmits data sharing information when it is determined that data sharing is possible, and with the head mount apparatus other than the head mount apparatus 100b when it is determined that data sharing is impossible. You may make it perform pairing between. Further, a priority order may be set for each pairing destination. Then, the information processing apparatus 200 may try pairing according to the priority order.
In step S812, the wireless communication unit 152b of the head mount device 100b receives the data sharing information transmitted by the information processing device 200. The connection processing unit 160b of the head mount device 100b acquires authentication information attached to the data sharing information. Then, the connection processing unit 160b creates a connection request accompanied with the authentication information.
In step S814, the wireless communication unit 152b of the head mount device 100b transmits a connection request to the head mount device 100a.

In step S816, the wireless communication unit 152a of the head mount device 100a receives the connection request transmitted by the head mount device 100b. The connection processing unit 160a of the head mount device 100a acquires authentication information attached to the connection request received by the wireless communication unit 152a. The connection processing unit 160a determines whether or not the authentication information acquired from the connection request matches the stored authentication information.
In step S818, when the connection processing unit 160a of the head mount device 100a determines that the authentication information acquired from the connection request matches the held authentication information, the connection processing unit 160b of the head mount device 100b communicates with the connection processing unit 160b. The process of establishing a communication link is performed.
After the communication link is established between the head mount device 100a and the head mount device 100b, the control unit 162a of the head mount device 100a acquires image data obtained by the imaging processing unit 156a and performs wireless communication. It transmits from the part 152a.
The image data transmitted by the head mount device 100a is received by the wireless communication unit 152b of the head mount device 100b and the transmission / reception unit 252 of the information processing device 200.
The control unit 162b of the head mount device 100b acquires the image data received by the wireless communication unit 152b and outputs it to the projection processing unit 154b. The projection processing unit 154b performs projection by processing the image data supplied by the control unit 162b. As a result, the image visually recognized by the user of the head mount device 100a is projected onto the virtual screen by the projection processing unit 154b of the head mount device 100b.
In addition, the display control unit 259 of the information processing device 200 processes the image data received by the transmission / reception unit 252 to display on the display device 209. As a result, the image visually recognized by the user of the head mount device 100a is displayed on the display device 209 by the display control unit 259 of the information processing device 200.

In the embodiment described above, in addition to transmitting images in real time between the head mount device 100a and the head mount device 100b, audio information may also be transmitted. More specifically, audio information is transmitted by turning on the microphones of the head mount device 100a and the head mount device 100b. Further, bidirectionally between the head mount device 100a and the head mount device 100b. It may be possible to perform telephone communication.
In the above-described embodiment, it is assumed that a communication link is established between the head mount device 100a and the information processing device 200, and between the head mount device 100b and the information processing device 200. Although the case where the data sharing request is transmitted to the information processing apparatus 200 has been described, the present invention is not limited to this example.
For example, the head mount device 100b transmits the captured image to the server, and the server stores the image. When the information processing apparatus 200 receives a data sharing request (pairing request) transmitted by the head mount apparatus 100a that is the data sharing destination, the information processing apparatus 200 pushes the data sharing request to the pairing source head mount apparatus 100b. It may be. When the information processing apparatus 200 receives an affirmative response indicating OK transmitted by the head mount apparatus 100b, the information processing apparatus 200 acquires an image captured by the head mount apparatus 100b from the server and transmits the image to the head mount apparatus 100a. You may do it.
Further, the head mount device 100a and the head mount device 100b transmit the recorded image to the information processing device 200, and the information processing device 200 records the recorded image transmitted by the head mount device 100a and the head mount device 100b. You may do it. Then, the head mount device 100a and the head mount device 100b may view a recorded image stored in the information processing device 200 with a time difference.

  According to the system according to the present embodiment, data acquired by one head mount device can be shared between a terminal device such as a head mount device other than the one head mount device and an information processing device. . For example, among a plurality of head mount devices, an image picked up by an image pickup unit of one head mount device can be projected onto a virtual screen of another head mount device and displayed on an information processing device or a terminal device. The information processing apparatus selects a head mount apparatus that shares an image captured by the one head mount apparatus based on information transmitted by the one head mount apparatus. For this reason, the work performed by the user wearing the one head mount device can be confirmed in real time by a user who wears another head mount device or a user other than the user who wears the head mount device. .

<Second Embodiment>
FIG. 1 can be applied to the system according to the present embodiment. The system according to the present embodiment includes a head mount device 300a, a head mount device 300b, and an information processing device 200. The system according to the first embodiment, the configuration of the head mount device, and the functions of the information processing device. Is different.
FIG. 9 shows a head mount device 300 according to this embodiment. The head mount device 300 includes a CPU 302, a ROM 304, a RAM 306, an operation unit 307, a nonvolatile memory 308, a communication I / F unit 310, an imaging unit 312, a projection unit 314, a positioning unit 315, an internal And a bus 316. These are fixed or arranged in a housing such as a temple of the head mount device 300.
The CPU 302, ROM 304, RAM 306, operation unit 307, nonvolatile memory 308, communication I / F unit 310, imaging unit 312, projection unit 314, and internal bus 316 are the same as the CPU 102, ROM 104, RAM 106, and operation unit 107 shown in FIG. The nonvolatile memory 108, the communication I / F unit 110, the imaging unit 112, the projection unit 114, and the internal bus 116 can be applied.

  The positioning unit 315 is a device for positioning the head mount device 300, and performs positioning by a global navigation satellite system (Global Navigation Satellite System (s): GNSS) or pedestrian dead reckoning (PDR). . When performing positioning by autonomous navigation for pedestrians, the positioning unit 315 may include an acceleration sensor, a gyro sensor, a geomagnetic sensor, an atmospheric pressure sensor, a proximity sensor, an illuminance sensor, and the like.

<Functional Configuration of System According to this Embodiment>
Next, the functional configuration of this embodiment will be described. FIG. 10 is a functional block diagram of the head mount device 300a, the head mount device 300b, and the information processing device 200 that constitute the system of the present embodiment. In FIG. 10, the head mount device 300 a, the head mount device 300 b, and the information processing device 200 are connected so that data communication can be performed via the communication network 20.

<Functional configuration of head mount device>
The head mount device 300a includes a wireless communication unit 352a, a projection processing unit 354a, an imaging processing unit 356a, a connection processing unit 360a, a control unit 362a, a positioning processing unit 363a, a storage unit 364a, and the above-described components shown in FIG. As shown in the figure, a bus line 366a such as an address bus or a data bus for electrical connection is provided. Each of these units is a function realized by any one of the constituent elements shown in FIG. 10 operating according to a command from the CPU 302 that executes the program 3082 expanded from the nonvolatile memory 308 to the RAM 306. is there.
As the wireless communication unit 352a, the projection processing unit 354a, the imaging processing unit 356a, and the storage unit 364a, the wireless communication unit 152a, the projection processing unit 154a, the imaging processing unit 156a, and the storage unit 164a described with reference to FIG. 4 are applied. it can.
The positioning processing unit 363a is realized by the positioning unit 315, and calculates position information by positioning the head mount device 300a. The positioning processing unit 363a outputs the position information to the control unit 362a.

The control unit 362a is realized by a command from the CPU 302. The control unit 362a controls the entire head mount device 300a. For example, the control unit 362a connects the connection processing unit 360a when image data obtained by imaging the positions and movements of the eyes of both eyes of the user wearing the head mount device 300a is input by the imaging processing unit 356a. To create a data sharing request.
In addition, when the position information is input from the positioning processing unit 363a, the control unit 362a outputs the position information to the connection processing unit 360a and instructs to create a data sharing request.
The connection processing unit 360a is realized by a command from the CPU 302 and a communication I / F 310. The connection processing unit 360a holds authentication information used for pairing between the head mount device 300a and the head mount device 300b. The connection processing unit 360a may hold authentication information in advance, or when the user performs an operation for sharing data between the head mount device 300a and the head mount device 300b on the operation unit 307. You may make it produce | generate and hold | maintain.

The connection processing unit 360a creates a data sharing request. The data sharing request is accompanied by the ID of the head mount device 100a, authentication information, and position information of the head mount device 300a. The connection processing unit 360a transmits a data sharing request from the wireless communication unit 352a to the information processing device 200. The data sharing request may include image data supplied by the control unit 362a. As a result, the work status can be notified to the information processing apparatus 200.
In addition, when the connection request transmitted by the head mount device 300b is received by the wireless communication unit 352a, the connection processing unit 360a determines whether the authentication information included in the connection request matches the authentication information held. Determine whether. When the connection processing unit 360a determines that the authentication information included in the connection request matches the authentication information held, the connection processing unit 360a performs an encryption process using a link key, and the head mount device 300a and the head mount device A communication link is established with 300b. If the connection processing unit 360a determines that the authentication information included in the connection request does not match the authentication information held, the connection processing unit 360a does not establish a communication link between the head mount device 300a and the head mount device 300b.

The head mount device 300b includes a wireless communication unit 352b, a projection processing unit 354b, an imaging processing unit 356b, a connection processing unit 360b, a control unit 362b, a positioning processing unit 363b, a storage unit 364b, and the above-described components shown in FIG. As described above, a bus line 366b such as an address bus or a data bus for electrical connection is provided. Each of these units is a function realized by any one of the constituent elements shown in FIG. 10 operating according to a command from the CPU 302 that executes the program 3082 expanded from the nonvolatile memory 308 to the RAM 306. is there.
The wireless communication unit 352b, the projection processing unit 354b, the imaging processing unit 356b, the connection processing unit 360b, the control unit 362b, and the storage unit 364b are the wireless communication unit 152b, the projection processing unit 154b, the imaging processing unit 156b, and the connection processing unit described above. 160b, control unit 162b, and storage unit 164b can be applied.
The positioning processing unit 363b is realized by the positioning unit 315, and calculates position information by positioning the head mount device 300b. The positioning processing unit 363b outputs the position information to the control unit 362b. In this case, the control unit 362b may output the position information supplied by the positioning processing unit 363b to the projection processing unit 354b, and the projection processing unit 354b may project the position information supplied by the connection processing unit 360b. .
The information processing apparatus 200 described with reference to FIG. 4 can be applied to the information processing apparatus according to the present embodiment. However, the function of the pairing management table 2584 and the selection unit 256 is different.

<Pairing management table>
FIG. 11 shows an example of the pairing management table 2584. The pairing management table 2584 uses, for each head mount device ID, the ID of the head mount device to be paired with which data is to be shared, the position where the data is shared, and the head mount device that shares the data. Manage time by linking. For example, in the pairing management table shown in FIG. 11, the head mount device whose ID of the head mount device is “300aa” has the ID of the head mount device of the pairing target to which data is shared is “300bb”. The data sharing position is “(111, 222)”, and the time for using the head mount device sharing the data is “9 o'clock to 12 o'clock”. The ID of the head mount device, the ID of the head mount device that shares the data, the position of sharing the data, and the time of sharing the data shown in FIG. 11 are examples, and the present invention is not limited to this example.
When the data sharing request transmitted by the head mount device 300a is received by the transmission / reception unit 252, the selection unit 256 acquires identification information, authentication information, and position information of the head mount device 300a from the data sharing request. The selection unit 256 refers to the pairing management table 2584 stored in the storage unit 258, and the position represented by the position information of the head mount device 300a attached to the data sharing request is the identification information of the head mount device 300a. If it is within a predetermined range from the sharing position associated with “300aa”, the head mounting device of the sharing destination associated with the identification information “300aa” of the head mounting device 300a attached to the data sharing request Get an ID. The selection unit 256 outputs the ID of the head mount device to be paired and the authentication information to the control unit 254. Further, the selection unit 256 refers to the pairing management table 2584 stored in the storage unit 258, and based on the time when the data sharing request is received by the transmission / reception unit 252, the time is the identification information “of the head mount device 300a”. The position information of the head mount device 300a included in the time for sharing the data associated with “300aa” and attached to the data sharing request is the share associated with the identification information “300aa” of the head mount device 300a. If it is within a predetermined range from the position, the ID of the shared head mount device associated with the identification information “300aa” of the head mount device 300a may be acquired.

<Operation of System According to this Embodiment>
FIG. 12 shows an example of the operation of the system according to the present embodiment.
In step S1202, the connection processing unit 360a of the head mount device 300a receives the ID of the head mount device 300a, the authentication information held in the connection processing unit 360a, and the position information obtained by positioning by the positioning processing unit 363a. And create a data sharing request.
In step S1204, the wireless communication unit 352a of the head mounted device 300a transmits a data sharing request to the information processing device 200.
In step S1206, the transmission / reception unit 252 of the information processing device 200 receives the data sharing request transmitted by the head mount device 300a. The selection unit 256 of the information processing device 200 acquires the ID of the head mount device 300a attached to the data sharing request and the position information. The selection unit 256 refers to the pairing management table 2584 of the storage unit 258, and the position represented by the position information of the head mount device 300a is determined from the shared position linked to the identification information “300aa” of the head mount device 300a. If it is within the predetermined range, the ID of the head mount device of the sharing destination linked to the identification information “300aa” of the head mount device 300a attached to the data sharing request is acquired.
Steps S808 to S818 in FIG. 8 can be applied to steps S1208 to S1218.

The image data transmitted by the head mount device 300a is received by the wireless communication unit 352b of the head mount device 300b and the transmission / reception unit 252 of the information processing device 200.
The control unit 362b of the head mount device 300b acquires the image data received by the wireless communication unit 352b and outputs it to the projection processing unit 354b. The projection processing unit 354b projects an image obtained by processing the image data supplied by the control unit 362b. As a result, the image visually recognized by the user of the head mount device 300a is projected on the virtual screen by the projection processing unit 354b of the head mount device 300b.
Further, the display control unit 259 of the information processing device 200 displays an image obtained by processing the image data received by the transmission / reception unit 252 on the display device 209. As a result, the image visually recognized by the user of the head mount device 300a is displayed on the display device 209 by the display control unit 259 of the information processing device 200.
According to the system according to the present embodiment, a data sharing request accompanied by position information of the head mount device is transmitted from the head mount device, and the position information is within a predetermined range from the positions of other head mount devices. In addition, the image data acquired by the head mount device can be shared between the other head mount device and the information processing device.

<Modification>
The first embodiment can be applied to a system according to this modification.
In the system according to this modification, an authority management table is stored in the storage unit 258 of the information processing apparatus 200 in addition to the head mount device management table 2582 and the pairing management table 2584.

<Authority management table>
FIG. 13 shows an example of the authority management table. The authority management table associates and manages the ID of the head mount apparatus with the authority to communicate for each ID of the head mount apparatus. For example, in the device authority management table shown in FIG. 13, the ID of the head mount device that is authorized to communicate with the head mount device whose head mount device ID is “100aa” is “100bb”. It is shown. The ID of the head mount device and the ID of the head mount device with the authority to communicate shown in FIG. 13 are examples and are not limited to this example.
When the data sharing request transmitted by the head mount device 100a is received by the transmission / reception unit 252, the selection unit 256 acquires identification information and authentication information of the head mount device 100a from the data sharing request. The selection unit 256 refers to the pairing management table stored in the storage unit 258, and acquires the ID of the head mount device to be paired associated with the identification information “100aa” of the head mount device 100a. Further, the selection unit 256 refers to the authority management table, and determines whether the head mount apparatus corresponding to the ID of the head mount apparatus to be paired has the authority to communicate with the head mount apparatus 100a that transmitted the data sharing request. judge. When the selection unit 256 determines that the head mount device corresponding to the ID of the head mount device to be paired has the authority to communicate with the head mount device 100a that has transmitted the data sharing request, the selection target head mount device 256 And the authentication information are output to the control unit 254.

<Operation of system according to this modification>
FIG. 8 can be applied to an example of the operation of the system according to this modification.
However, in step S808, the selection unit 256 of the information processing device 200 has an authority to communicate with the head mount device 100a to which the head mount device corresponding to the ID of the head mount device to be paired has transmitted the data sharing request. Determine whether. Further, when the control unit 254 determines that the head mount device corresponding to the ID of the head mount device to be paired has the authority to communicate with the head mount device 100a that has transmitted the data sharing request, the control unit 254 sets the destination as the pair target. The data sharing information accompanied by the authentication information is created with the IP address associated with the ID of the head mount device.

  According to the system according to the present modification, the data acquired by one head mounting device among a plurality of head mounting devices is used as the one head among other head mounting devices other than the one head mounting device. It can be shared with the head mount device having the authority to communicate with the mount device.

  As mentioned above, although embodiment of this invention and its modification were demonstrated, these embodiment and its modification are shown as an example, and are not intending limiting the range of invention. These embodiments and modifications thereof can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the spirit of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and at the same time, are included in the invention described in the claims and the equivalents thereof.

  Each of the above devices has a computer inside. The process of each device described above is stored in a computer-readable recording medium in the form of a program, and the above-described processing is performed by the computer reading and executing the program. Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  In the above-described embodiment, the head mount device 100a is an example of a head mount device, the head mount device 100b is an example of a terminal device, the pairing management table is an example of an identification information management unit, and the data sharing information is data The transmission / reception unit 252 is an example of a first communication unit, the wireless communication unit 152b is an example of a second communication unit, and the wireless communication unit 152b is an example of a third communication unit. It is. The positioning processing unit 363a is an example of a positioning unit, and the communication authority management table is an example of a communication authority management unit.

  DESCRIPTION OF SYMBOLS 20 ... Communication network, 70 ... Access point, 100a, 100b ... Head mount apparatus, 152a, 152b ... Wireless communication part, 154a, 154b ... Projection process part, 156a, 156b ... Imaging process part, 160a, 160b ... Connection process part, 162a, 162b ... control unit, 164a, 164b ... storage unit, 200 ... information processing device, 252 ... transmission / reception unit, 254 ... control unit, 256 ... selection unit, 258 ... storage unit, 259 ... display control unit, 300a, 300b ... Head mount device, 352a, 352b ... wireless communication unit, 354a, 354b ... projection processing unit, 356a, 356b ... imaging processing unit, 360a, 360b ... connection processing unit, 362a, 362b ... control unit, 364a, 364b ... storage unit

Claims (9)

Information processing device
Identification information management for registering identification information of one or a plurality of head mount devices and identification information of one or a plurality of terminal devices to be communicated with each of the one or more head mount devices And
When the identification information of the head mount device transmitted by the head mount device is registered in the identification information management unit, the terminal device linked to the identification information of the head mount device and activated A first communication unit that transmits a request to share data acquired by the head mount device,
The head mount device is:
A second communication unit that communicates with the first communication unit;
A first control unit that performs processing for sharing data acquired by the head mount device with a terminal device;
The second communication unit is a system that transmits identification information of the head mount device to the information processing device when processing for sharing the data is performed by the first control unit. The terminal device
A third communication unit that communicates with the first communication unit;
In response to a request to share the data received by the third communication unit, the information processing apparatus and the head mount apparatus are connected to either or both of the information processing apparatus and the head mount apparatus. The system of Claim 1 provided with the 2nd control part which performs communication control between these. The identification information management unit registers and registers position information with which data can be shared with a terminal device for each of the identification information of the one or more head mounted devices,
The head mount device is:
A positioning unit for positioning the head mount device;
The second communication unit transmits position information obtained by positioning by the positioning unit to the information processing device,
The first communication unit includes the position information transmitted by the head mount device within a predetermined range from position information corresponding to the identification information of the head mount device transmitted by the head mount device. The system according to claim 1 or 2, wherein a request for sharing the data acquired by the head mount device is transmitted to the terminal device. The identification information management unit registers the identification information of the one or more head mount devices and information indicating the time each of the one or more head mount devices is used,
The first communication unit is used corresponding to the identification information of the head mount device in which information indicating the time when the identification information of the head mount device is received from the head mount device is stored in the identification information management unit. The system according to any one of claims 1 to 3, wherein a request for sharing data acquired by the head mounted device is transmitted to the terminal device when the time matches or is included in the time. . The information processing apparatus includes a communication authority management unit that registers identification information of a terminal apparatus that is authorized to communicate with a head mounted apparatus;
When the identification information of the head mount device transmitted by the head mount device is registered in the identification information management unit, the terminal device linked to the identification information of the head mount device and activated A determination unit that determines to share data acquired by the head mount device with the terminal device when registered in the authority management unit;
The first communication unit shares data acquired by the head mount device to the terminal device when the determination unit determines that the terminal device shares data acquired by the head mount device. The system according to any one of claims 1 to 4, wherein the system transmits a request to be transmitted. Identification information management for registering identification information of one or a plurality of head mount devices and identification information of one or a plurality of terminal devices to be communicated with each of the one or more head mount devices And
When the identification information of the head mount device transmitted by the head mount device is registered in the identification information management unit, the terminal device linked to the identification information of the head mount device and activated An information processing apparatus comprising: a communication unit that transmits a request to share data acquired by the head mount device. In the head mount device,
A communication unit that communicates with the information processing apparatus;
A control unit that performs processing for sharing data acquired by the head mount device with the terminal device,
The said communication part is a head mount apparatus which transmits the identification information of the said head mount apparatus to the said information processing apparatus, when the process which shares the said data by the said control part is performed. In the computer of the information processing device,
The identification information of the head mount device transmitted by the head mount device is the identification information of one or a plurality of head mount devices and one or a plurality of targets to be communicated with each of the one or more head mount devices. Determining whether or not it is registered in an identification information management unit that registers and associates identification information of a terminal device;
When registered in the identification information management unit, a request to share the data acquired by the head mount device is transmitted to the terminal device linked to the identification information of the head mount device and activated. A program that executes steps and. To the computer of the head mount device,
Communicating with the information processing apparatus;
Performing a process of sharing data acquired by the head mount device with a terminal device;
A program for executing the step of transmitting the identification information of the head mount device to the information processing device when processing for sharing the data with the terminal device is performed.

Images