JP7247532B2 - Controllers, programs and systems - Google Patents

Description

The present invention relates to control devices, programs and systems.

A technology of mixed reality (MR) is known. The MR technology is a technology for superimposing virtual digital three-dimensional information on the real space and visualizing the real object and the virtual three-dimensional information while interacting with each other.

In the MR technology, for example, it is possible to superimpose virtual three-dimensional information data on an object and draw it, or to enable intuitive manipulation by the user's hand. Such usage of MR technology can be said to be usage in which augmented reality (AR) and virtual reality (VR) are combined. Here, the AR technology is a technology for superimposing and expanding information on the real space. Also, the VR technology is a technology that makes it possible to experience a virtual world in a digital space.

MR technology uses a device that can be worn by the user (wearable). As such a device, for example, there is a display device that is a head-mounted display. The display device is a spectacle-like device that is worn on the user's head. When the display device is worn on the user's head, the display device has a transmissive display section at a position facing the user's eyes, and is controlled by the control device to display virtual information on the display section. indicate. Thereby, the display device provides both eyes of the user with visual information that interacts with real objects and virtual three-dimensional information.

As an example of MR technology, the welding inspection method described in Patent Document 1 uses an information processing device that presents MR images. Alignment with the model is performed (see Patent Document 1).

JP 2017-44496

However, in the control device for the wearable display device as described above, data search keys are accepted in accordance with the movement of the user wearing the display device, which may require a great deal of effort on the part of the user. rice field.
For example, the control device accepts input such as characters using a virtual keyboard and a virtual mouse. At this time, the control device determines the operation of the virtual keyboard or the virtual mouse according to the change in the direction of the head of the user wearing the display device, and inputs characters and the like according to the operation. accept. In this way, the user inputs characters by moving his or her head, which sometimes requires a great deal of effort.

In such a control device, an input method of recognizing the voice of the user wearing the display device and accepting a data search key according to the recognition result of the voice is also conceivable. It is conceivable that the display device will be used in places where it cannot be detected with accuracy. For example, when the user is working in a predetermined workplace, there may be times when mechanical operation sounds around the user and the display device are loud, or there may be times when noise is loud in the workplace.

SUMMARY OF THE INVENTION The present invention has been made in consideration of such circumstances, and enables a user to easily accept a key for data retrieval when an MR is displayed by a display device worn by the user. An object of the present invention is to provide a control device, a program, and a system that can

As one configuration example, an ambient information acquisition device that acquires information around a display device that is worn by a user and is capable of transmitting light in a real space to a display unit and displaying a virtual image on the display unit. a space recognizing unit that recognizes a space based on the acquired information; an associating unit that associates the space recognized by the space recognizing unit with predetermined map information; a result of the associating by the associating unit; A keyword acquisition unit for acquiring a keyword that hierarchically changes according to the situation of the display device based on the situation of the display device ; and display control for causing the display device to display the keyword acquired by the keyword acquisition unit. and a control device.

As one configuration example, in the control device, the status of the display device includes one or more of the position of the display device, the orientation of the display device, or the information acquired by the ambient information acquisition device .
As one configuration example, the control device includes a search unit that performs a search using the keyword, and the display control unit causes the display device to display information searched by the search unit.
As one configuration example, in the control device, the display control unit displays at least one of the keyword and the information searched by the search unit pasted at a predetermined location in accordance with an instruction given by the user. I do.

As one configuration example, an ambient information acquisition device that acquires information around a display device that is worn by a user and is capable of transmitting light in a real space to a display unit and displaying a virtual image on the display unit. a function of recognizing a space based on the acquired information; a function of associating the recognized space with predetermined map information ; A program for causing a computer to realize a function of acquiring a keyword that hierarchically changes according to a situation and a function of displaying the acquired keyword on the display device.

As one configuration example, a system comprising a display device worn by a user and capable of transmitting light in a real space to a display unit and displaying a virtual image on the display unit, and a server device, a space recognition unit that recognizes a space based on the information acquired by a surrounding information acquisition device that acquires information around the display device; and the space recognized by the space recognition unit and predetermined map information. a matching unit for matching, a keyword acquiring unit for acquiring a keyword that hierarchically changes according to the status of the display device based on the result of matching by the matching unit and the status of the display device , and the keyword acquisition. a display control unit that causes the display device to display the keyword acquired by the unit.

According to the present invention, when MR is displayed by the display device worn by the user, it is possible for the user to easily accept a data search key.

1 is a diagram showing a schematic configuration of an MR system according to one embodiment of the present invention; FIG. 1 is a diagram showing a schematic configuration of a display device according to an embodiment of the invention; FIG. 1 is a diagram showing a schematic configuration of a mobile device according to one embodiment of the present invention; FIG. It is a figure showing a schematic structure of a control device concerning one embodiment of the present invention. FIG. 2 is a diagram showing an example of floor layout according to an embodiment of the present invention; (A) to (E) are diagrams showing examples of narrowing down keywords according to an embodiment of the present invention. (A) to (E) are diagrams showing other examples of narrowing down keywords according to an embodiment of the present invention. FIG. 4 is a diagram showing an example of information displayed on the display device according to one embodiment of the present invention; FIG. 5 is a diagram showing an example of pasting selection information according to an embodiment of the present invention; It is a figure which shows the example of the search-result information which concerns on one Embodiment of this invention. It is a figure which shows the example of the procedure of the process performed in the control apparatus which concerns on one Embodiment of this invention. It is a figure which shows the example of the procedure of the process performed in the control apparatus which concerns on one Embodiment of this invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
A first embodiment will be described.
<MR system>
FIG. 1 is a diagram showing a schematic configuration of an MR system 1 according to one embodiment of the invention.
The MR system 1 includes a display device 11 , a mobile device 12 , a server device 21 , an information processing device 22 , a display device 23 , a database 24 , a router device 25 and a network 31 .
Also shown in FIG. 1 are a user 51 and an operator 61 . In this embodiment, user 51 and operator 61 are each human.

The display device 11 is a wearable device, and is a head-mounted display device in this embodiment, and can be attached to and detached from the head of the user 51 . In this embodiment, the display device 11 is worn on the head of the user 51 .
Further, the display device 11 has, for example, a transmissive display function. With the transmissive display function, a virtual image can be displayed on the display section covering both eyes of the user 51 while allowing external light to pass through the display section. As a result, the transmissive display function can provide the user 51 with visual information in which the real world and virtual images are combined. A virtual image may be called a hologram image, for example.
In this manner, the display device 11 is worn by the user 51 and can transmit light in the real space to the display section to display a virtual image on the display section.

Portable device 12 can be carried by user 51 . The mobile device 12 is, for example, a mobile phone, a smart phone, or a personal digital assistant (PDA: Personal Digital Assistant). In this embodiment, the mobile device 12 is carried by the user 51 .

The server device 21 has a control device 41 . The server device 21 and the control device 41 are each, for example, computers. Note that the server device 21 and the control device 41 may be configured using, for example, a common computer, or may be different computers.
The information processing device 22 is, for example, a computer.
The information processing device 22 includes a display device 23 .

Here, in the present embodiment, the information processing device 22 and the display device 23 are communicably connected. This connection may be made using, for example, a wired line or a wireless line.
An information processing device 22 and a server device 21 are also connected. This connection may be made using, for example, a wired line or a wireless line.

Server device 21 , database 24 , and router device 25 are each connected to network 31 and can communicate with each other via network 31 . This connection may be made using, for example, a wired line or a wireless line. That is, the network 31 may be a wired network, a wireless network, or a combination of wired and wireless.
Also, the router device 25 is compatible with, for example, Wi-Fi (registered trademark).

The display device 11 and the mobile device 12 are connected to the router device 25 so as to communicate wirelessly.
In this embodiment, this connection is made through a wireless line, but as another example, this connection may be made through a wired line.

<Overview of operation of MR system>
In this embodiment, in the server device 21 , the control device 41 communicates with the display device 11 and the mobile device 12 via the network 31 and the router device 25 . The control device 41 controls the display device 11 so that the display device 11 displays MR. The control device 41 can use the information received from the display device 11 or the portable device 12 to display the MR.
Also, the control device 41 communicates with the database 24 via the network 31 . The database 24 stores various information. The control device 41 searches the information stored in the database 24 and receives the obtained information. The control device 41 can use the information obtained by the search to display MR.
The information stored in the database 24 may be arbitrary information, for example, information obtained as a result of monitoring a monitoring target. The monitored object may be, for example, the temperature or pressure of predetermined equipment or equipment.

The router device 25 relays a signal transmitted from the control device 41 and transmits the signal to the display device 11 or the mobile device 12 . Also, the router device 25 relays a signal transmitted from the display device 11 and a signal transmitted from the mobile device 12 and transmits the signals to the control device 41 .
The information processing device 22 communicates with the control device 41 of the server device 21 . The information processing device 22 transmits signals such as instructions to the control device 41 and receives signals transmitted from the control device 41 . The information processing device 22 displays information on the screen of the display device 23 . In this embodiment, an operator 61 at the central management location operates the information processing device 22 and sees information displayed on the screen of the display device 23 . In this embodiment, for example, the operator 61 operates the information processing device 22 to control the control device 41, thereby performing remote control at a location remote from the display device 11, and the display device 11 exists. On-site assistance may be provided. Further, the information processing device 22 may display, for example, one or both of a real space image and a virtual image viewed by the user 51 wearing the display device 11 on the screen of the display device 23. , whereby monitoring of work performed by user 51 may be performed.
Note that each signal may include various types of information.

The display device 11 is controlled by the control device 41 to display MR to the user 51 .
The mobile device 12 detects the position of the mobile device 12 and transmits information on the detected position to the control device 41 .
In this embodiment, the display device 11 is worn by the user 51, the portable device 12 is carried by the user 51, and the position of the display device 11, the position of the portable device 12, and the position of the user 51 are the same. Consider. In other words, the position of the display device 11, the position of the portable device 12, and the position of the user 51 may differ in practice, but in this embodiment, they are maintained at substantially the same positions. Therefore, in the following description, the position of the display device 11, the position of the portable device 12, and the position of the user 51 are all considered to be the same position. Detailed description is omitted in the embodiment.) is performed, each position is detected as a different position.
Note that the display device 11 and the mobile device 12 may communicate directly by wire or wirelessly, or may communicate via the router device 25, for example.

Here, the information processing device 22 and the display device 23 may not necessarily be separate units, and may be integrated.
Further, the information processing device 22, the database 24, and the router device do not necessarily have to be separate from the server device 21, and any one or more of the information processing device 22, the database 24, and the router device It may be integrated with the server device 21 .
Further, the MR system 1 according to the present embodiment includes the information processing device 22 and the display device 23 used by the operator 61, but as another example, these devices may not be provided.

Further, in the present embodiment, the mobile device 12 has the function of detecting the position, but as another example, the display device 11 may have the function of detecting the position. may not be provided. In this embodiment, the mobile device 12 is primarily used for position detection.
Further, in the present embodiment, the control device 41 generates an image for MR display, transmits the image to the display device 11, and the display device 11 receives and displays the information.

<Display device>
FIG. 2 is a diagram showing a schematic configuration of the display device 11 according to one embodiment of the invention.
The display device 11 includes a communication section 111 , a storage section 112 , a display section 113 , a detection section 114 and a control section 115 .
Communication unit 111 communicates with control device 41 of server device 21 via router device 25 and network 31 .
The storage unit 112 stores various information.
The display unit 113 displays information. The display unit 113 may have, for example, a display portion for the right eye of the user 51 and a display portion for the left eye.

The detection unit 114 detects predetermined information.
In this embodiment, the detection unit 114 includes an ambient information acquisition device that acquires ambient information. The ambient information acquisition device detects information about the surroundings of the display device 11 and acquires the information.
In addition, in the present embodiment, the ambient information acquisition device includes an imaging device, which captures an image of the outside world of the display device 11 and acquires information of the image.
In the present embodiment, in the display device 11, the imaging device is provided at a position for capturing an image of the surroundings in the line-of-sight direction of the user 51 wearing the display device 11 as an image of the outside world. Specifically, when the display device 11 is worn by the user 51, the imaging device is provided at a position corresponding to the center of both eyes of the user 51, and captures an image in the direction opposite to the head of the user 51. are arranged to

Further, in the present embodiment, the ambient information acquisition device may include a device that acquires information other than images. For example, the ambient information acquisition device is a device that detects ultrasonic waves around the display device 11 and acquires information about the ultrasonic waves, or detects the temperature around the display device 11 and acquires information about the temperature. It may also include devices and the like. For example, a sonar may be used as a device that detects ultrasonic waves and acquires information on the ultrasonic waves. A temperature sensor, for example, may be used as a device that detects temperature and acquires temperature information.

In addition, in this embodiment, when using the information of the image captured by the imaging device included in the ambient information acquisition device, it is possible to further use information such as ultrasonic wave or temperature to improve the overall accuracy. However, the image information does not necessarily have to be acquired by the ambient information acquisition device.
For example, the ambient information acquisition device may include a device that acquires one or more information other than an image, such as ultrasonic waves or temperature, without including an imaging device that captures an image. In this case, information other than the image is used as the surrounding information.

Here, the detection unit 114 may include a sensor that detects arbitrary information in addition to the ambient information acquisition device. In other words, the present embodiment includes a configuration in which the detection unit 114 is provided with another sensor in addition to the ambient information acquisition device, and includes a configuration in which the detection result of the other sensor is arbitrarily used. Note that when the detection unit 114 includes only the ambient information acquisition device, the detection unit 114 may be regarded as the ambient information acquisition device.
As another example, the detection unit 114 may be provided separately from the display device 11 . For example, when the detection unit 114 includes a plurality of information acquisition devices, some of the plurality of information acquisition devices may be separately provided. As the information acquisition device, for example, a device that acquires various types of information included in the surrounding information acquisition device (for example, an imaging device that acquires image information, a device that acquires ultrasound information, and a device that acquires temperature information device, etc.), or the entire ambient information acquisition device, or various other sensors.

The control unit 115 performs various controls in the display device 11 .
The control unit 115 includes a display control unit 131 and a detection information transmission control unit 132 as examples of functional units.
The display control unit 131 controls image display by the display unit 113 . In this embodiment, in the display device 11, the communication unit 111 receives image information transmitted from the control device 41, and the display control unit 131 displays the image on the display unit 113 based on the received image information. control to allow
The detected information transmission control section 132 transmits the information detected by the detection section 114 to the control device 41 through the communication section 111 . In this embodiment, the information includes information acquired by the ambient information acquisition device included in the detection unit 114 .

<Portable device>
FIG. 3 is a diagram showing a schematic configuration of the portable device 12 according to one embodiment of the invention.
Portable device 12 includes input unit 211 , output unit 212 , communication unit 213 , storage unit 214 , position detection unit 215 , and control unit 216 .
The input unit 211 has a function of receiving and inputting the contents of operations performed by the user 51 and a function of inputting information output from an external device connected by wire.
The output unit 212 has a function of outputting information to a display screen and a function of outputting information to an external device connected by wire.
Here, the external device may be, for example, a portable recording medium such as a USB memory.

The communication unit 213 wirelessly communicates with an external device. In this embodiment, the communication unit 213 communicates with the control device 41 of the server device 21 via the router device 25 and the network 31 .
The storage unit 214 stores various information.
The position detection unit 215 detects the position of the mobile device 12 . In this embodiment, the position detection unit 215 has a GPS (Global Positioning System) function, receives signals wirelessly transmitted from GPS satellites, and detects a position based on the received signals.

The control unit 216 performs various controls in the mobile device 12 .
The control unit 216 includes a position information transmission control unit 231 as an example of a functional unit.
The position information transmission control section 231 transmits the information of the position detected by the position detection section 215 to the control device 41 through the communication section 213 .

<Control device>
FIG. 4 is a diagram showing a schematic configuration of the control device 41 according to one embodiment of the invention.
The control device 41 includes an input section 311 , an output section 312 , a communication section 313 , a storage section 314 and a control section 315 .
The input unit 311 has a function of receiving and inputting details of an operation performed by a person who operates the control device 41 and a function of inputting information output from an external device connected by wire.
The output unit 312 has a function of outputting information to a display screen and a function of outputting information to an external device connected by wire.

The communication unit 313 wirelessly communicates with an external device. In this embodiment, the communication unit 313 communicates with the display device 11 and the mobile device 12 via the router device 25 and the network 31 . Also, in this embodiment, the communication unit 313 communicates with the database 24 via the network 31 .
The storage unit 314 stores various information. In this embodiment, the storage unit 314 stores map information 331 .

Here, the map information 331 includes at least map information of a place that is assumed to be imaged by the imaging device of the display device 11 .
The map information 331 may include information on various areas, information on facilities or equipment, and the like.
It should be noted that the terms "equipment" and "equipment" are examples and may be called by other arbitrary names. Also, "facility" and "equipment" do not necessarily have to be distinguished from each other, and both may be considered to represent the same thing.

For example, in this embodiment, the map information 331 includes identification information of one or more predetermined areas and information specifying the area. As the area identification information, for example, information such as the name of the area is used. A region may be associated with, for example, equipment or equipment.
For example, in this embodiment, the map information 331 includes identification information of one or more predetermined facilities or devices and information on the arrangement of the facilities or devices. Information such as the name of the facility or device is used as the identification information of the facility or device, for example.

The map information 331 may include, for example, map information of the site where the user 51 who is the worker is working.
Also, in the map information 331, facilities or equipment may be represented using simplified solids such as rectangular parallelepipeds or cylinders. The simplified solid in virtual space and the corresponding location in real space may be registered.

In this embodiment, it is possible to acquire keywords related to areas or devices based on information included in the map information 331 . Such keywords may be included in the map information 331 as, for example, a list of one or more keywords, or may be included in the map information 331 as information related to the equipment, such as a description of the equipment. may be As described above, in the present embodiment, in the map information 331, devices and the like are associated with information from which keywords can be acquired.
The map information 331 is, for example, created in advance by a person or a device and stored in the storage unit 314 . As an example, the map information 331 may be generated by the information processing device 22 or the server device 21 .
In this embodiment, the control device 41 can acquire keywords based on the map information 331. As another example, the control device 41 acquires keywords related to areas or devices. Information for this may be stored in the storage unit 314 as information separate from the map information 331 .

The control unit 315 performs various controls in the control device 41 .
The control unit 315 includes, as examples of functional units, a space recognition unit 351, a map acquisition unit 352, an equipment layout association unit 353, a position/orientation acquisition unit 354, a keyword acquisition unit 355, a search unit 356, a display A control unit 357 is provided.

The space recognition unit 351 acquires information around the display device 11 transmitted from the display device 11 and received by the communication unit 313 . The space recognition unit 351 recognizes a three-dimensional space based on the acquired information. For example, when the information about the surroundings of the display device 11 includes the information of the image of the outside world of the display device 11, the space recognition section 351 may recognize the three-dimensional space shown in the image. The method for recognizing space is not particularly limited, and for example, a method for recognizing space using a three-dimensional point group may be used.

The map acquisition section 352 acquires the map information 331 stored in the storage section 314 .
In this embodiment, the map acquisition unit 352 detects information on a predetermined marker included in information transmitted from the display device 11 and received by the communication unit 313 . For example, when the information on the surroundings of the display device 11 includes information on an image of the outside world of the display device 11, the information on the marker may be visually recognizable information included in the image. As the marker, for example, an AR marker may be used.
In this embodiment, the map information 331 includes map information of the space recognized by the space recognition unit 351 .
In this embodiment, the map information 331 is associated with predetermined markers. Then, the map acquisition unit 352 acquires the map information 331 associated with the detected marker.
In addition, when the space recognized by the space recognition unit 351 is predetermined and the map information 331 corresponding to the space is specified in advance, the predetermined marker may not be used.

Further, the map acquisition unit 352 may acquire the position information of the display device 11 based on the position information transmitted from the portable device 12 and received by the communication unit 313, for example.
As an example, the map acquisition unit 352 acquires the map information 331 corresponding to the space based on the marker in the space where the predetermined marker is detected, and based on the position information in the space where the predetermined marker is not detected. You may acquire the map information 331 corresponding to the said position by pressing.

Note that the control unit 315 may follow the position of the moving display device 11 or detect the speed of the moving display device 11 based on the acquired position information of the display device 11, for example. may be performed.
Further, for example, if the position information detected by the portable device 12 is not used, the MR system 1 may not be provided with the portable device 12, and the control device 41 has the function of acquiring the position information. may not be provided.

Based on the space recognized by the space recognition unit 351 and information on the placement of the devices included in the map information acquired by the map acquisition unit 352, the device placement association unit 353 associates the space with the placement of the devices. Associate.
Here, information on the dimensions of each device is set in the device layout information. As the dimensions, for example, one, two, or three axial dimensions of an XYZ coordinate system, which is a three-dimensional orthogonal coordinate system, may be used. As an example, when dimensions are defined by width, depth, and height, only width, width and depth, width and height, or width, depth, and height may be used, for example.

In addition, the equipment arrangement association unit 353 may further perform processing for improving the accuracy of associating the space and the arrangement of the equipment, for example.
As an example, the device layout association unit 353 detects the finger position when the user 51 wearing the display device 11 points to a predetermined position of a predetermined device, and based on the detection result, determines the space and the device. may be fine-tuned.

The position/orientation acquisition unit 354 acquires position information and orientation information of the display device 11 .
Here, the position/orientation acquisition unit 354 acquires the position and orientation of the display device 11 based on the result of the association performed by the device layout association unit 353 and the information transmitted from the display device 11 and received by the communication unit 313, for example. Get location information for .
Also, the position/orientation acquisition unit 354 acquires orientation information of the display device 11 based on information transmitted from the display device 11 and received by the communication unit 313 . That is, the posture of the display device 11 changes according to the change in the posture of the user's 51 head.

In this embodiment, the control unit 315 generates a virtual space corresponding to the real space based on the processing results of the space recognition unit 351 , the map acquisition unit 352 and the device layout association unit 353 . Then, the position and orientation acquisition unit 354 detects the position and orientation of the display device 11 in the virtual space, and regards the detection result as the position and orientation in the real space.
Here, the real space means a real space, and the virtual space means a virtual space captured based on the real space.

Further, the position/orientation acquisition unit 354 detects the direction of the line of sight of the user 51, detects a change in the direction of the line of sight of the user 51, or detects the direction of the line of sight of the user 51, or It is possible to detect the movement of the head, and so on.
For example, the control unit 315 may accept input of information such as characters or instructions based on the direction of the user's 51 line of sight, changes in the direction, or movement of the user's 51 head.
Further, in this embodiment, the orientation of the display device 11 is determined based on the information acquired by the display device 11 . As an example, a configuration may be used in which the orientation of the display device 11 is determined based on an image captured by the display device 11 . As another example, a configuration in which the display device 11 is provided with a sensor capable of detecting the orientation such as a gyro sensor and the orientation of the display device 11 is determined based on the detection result of the sensor may be used. good.

As a configuration for receiving characters or instructions from the user 51 on the display device 11, for example, a virtual keyboard or a virtual mouse is displayed on the display device 11, and characters or instructions are displayed according to the movement of the user 51. Any configuration that accepts may be used.
Note that when the orientation information of the display device 11 is not used, the control device 41 may not be provided with the function of acquiring the orientation information.

Further, as a configuration for receiving characters or instructions from the user 51 in the display device 11, for example, a microphone for detecting voice is provided in the detection unit 114, and the voice emitted from the user 51 is detected by the microphone or the like, and the detected voice is detected. A configuration may also be used in which characters or instructions are received by recognizing the voice.

A keyword acquisition unit 355 acquires a keyword based on the position and orientation detected by the position/orientation acquisition unit 354 and the map information 331 . That is, in the present embodiment, the keyword acquisition unit 355 automatically detects keywords based on the position and orientation of the user 51 wearing the display device 11 and the map information 331 .
Note that the keyword acquisition unit 355 may acquire keywords based on the position detected by the position/orientation acquisition unit 354 and the map information 331 . In other words, posture information may not be used to acquire keywords.

Here, the keyword acquisition unit 355 acquires, for example, a keyword related to an area or device corresponding to the position where the user 51 wearing the display device 11 is present. The keyword acquisition unit 355 also obtains keywords related to devices existing around the line of sight of the user 51 wearing the display device 11 based on the position and posture of the user 51 wearing the display device 11, for example. may be obtained. For example, the field of view of the user 51 may be used as the surroundings of the line of sight of the user 51 . The field of view of user 51 is specified based on the position and orientation of display device 11 . For example, the keyword acquisition unit 355 may acquire keywords related to equipment determined to be within the field of view of the user 51 , or may acquire keywords related to devices in the vicinity of the user 51 regardless of whether they are within the field of view of the user 51 . Keywords related to devices in
Further, the keyword acquisition unit 355 may acquire the keyword using the name of the area, the name of the device, or the like.

As described above, for example, when the user 51 wearing the display device 11 is present in a predetermined area, the keyword acquisition unit 355 generates a Alternatively, even if the user 51 wearing the display device 11 is present in a predetermined area, a keyword related to the device or the like present in the predetermined area is necessarily obtained. However, keywords related to devices existing in the line of sight or line of sight of the user 51 may be acquired.
As another example, the keyword acquisition unit 355 does not determine whether or not the user 51 wearing the display device 11 exists in a predetermined area. You can get keywords related to
As another example, when the distance between the position of the user 51 wearing the display device 11 and the position of the predetermined device or the like is equal to or less than a predetermined threshold, the keyword acquisition unit 355 may obtain a keyword related to the predetermined device or the like. You may detect keywords that

In this embodiment, the keyword acquisition unit 355 acquires keywords based on the status of the display device 11, for example. The status of the display device 11 may include, for example, one or more of the position of the display device 11, the orientation of the display device 11, and information about the surroundings of the display device 11 (information obtained by the surrounding information obtaining device). good.

Search unit 356 searches for information using some or all of the keywords acquired by keyword acquisition unit 355 .
Here, the search unit 356 may accept a keyword selected by the user 51 from among the keywords acquired by the keyword acquisition unit 355 as a keyword for search. In this case, for example, the keywords acquired by the keyword acquisition unit 355 are displayed by the display device 11, and selection of one or more keywords from the displayed keywords is accepted according to the user's 51's actions or the like.
In this embodiment, the search unit 356 searches information stored in the database 24 using a keyword via the network 31 by the communication unit 313 .
Then, the search unit 356 acquires the information of the searched result.

The display control unit 357 controls the display of MR on the display device 11 . Specifically, the display control unit 357 generates information of an image to be displayed by the display device 11 and transmits the information of the image to the display device 11 through the communication unit 313 .
The display control unit 357 selects an image to be displayed by the display device 11 based on one or more of the map information 331, the keyword acquired by the keyword acquisition unit 355, or the search result of the search unit 356, for example. Generate information. In other words, the display control unit 357 has a function of causing the display device 11 to display any one or more pieces of information such as a map, keywords, search results, and the like.

Here, in this embodiment, in order to simplify the explanation, a case where one display device 11 is controlled and monitored by one control device 41 will be described.
As another example, when a plurality of different display devices 11 are monitored by one control device 41, for example, identification information is set for each display device 11, and the control device 41 and each display device 11 are Communicate using each identification information. Further, the information processing device 22 and the server device 21, for example, manage information about the respective display devices 11 using their respective identification information.

Further, in this embodiment, the map information 331 is stored in the storage unit 314 of the control device 41, but as another example, the map information 331 may be stored in the storage unit of any device other than the control device 41. may be stored. In this case, the map acquisition unit 352 accesses the external device from the control device 41 and acquires the map information 331 stored in the device. The device may be, for example, a database, by way of example database 24 .

<Example of operation in MR system>
5 to 10, examples of operations in the MR system 1 are shown.
FIG. 5 is a diagram illustrating an example layout of the floor 411 according to one embodiment of the present invention.
Floor 411 represents the second floor of the building where the power plant is located. Although one floor 411 is shown in the example of FIG. 5, if the building has two or more floors, similar floors may be recognized for each floor.

In the example of FIG. 5, on the floor 411, the device group of machine No. 8 is arranged. Here, the 8th machine is an example, and one or more of the 1st to 7th machines and the 9th and subsequent machines may be arranged at other locations. In addition, "machine number" is an example of a name, and any other name may be used.
In the 8th machine, three shafts, namely, a 1st shaft 431, a 2nd shaft 432 and a 3rd shaft 433 are arranged. Here, the 1-axis 431 to 3-axis 433 are examples, and the number of axes may be 1, 2, or 4 or more. Also, "axis" is an example of a name, and any other name may be used.
Two devices 451 and 452 are arranged in the area of one axis 431 . One device 453 is arranged in the area of the two axes 432 . Two devices 454 and 455 are arranged in the area of the three axes 433 . Note that the number or types of devices arranged on each axis may be arbitrary.

For the eighth car on the floor 411, a car area 421 determined to be the area of the eighth car is set.
A 1-axis region 441 that is determined to be the region of the 1-axis 431 is set on the 1-axis 431 .
A biaxial region 442 that is determined to be the region of the biaxial 432 is set on the biaxial 432 .
A 3-axis region 443 that is determined to be the 3-axis 433 region is set in the 3-axis 433 .
Device regions 461 to 465 that are determined to be regions of the devices 451 to 455 are set for the devices 451 to 455, respectively.

Here, the floor 411, the first axis 431, the second axis 432, the third axis 433, and the devices 451 to 455 exist in real space. Items in the virtual space associated with the space will also be described using the same reference numerals.
Also, the machine number area 421, the 1-axis area 441, the 2-axis area 442, the 3-axis area 443, and the equipment areas 461 to 465 are in the virtual space that the control device 41 associates with the real space.
Things in these virtual spaces are defined by the map information 331 .

An AR marker 471 is installed on the floor 411 . AR marker 471 may be, for example, a three-dimensional marker or a two-dimensional marker.
In the example of FIG. 5, an AR marker 471 is installed near the position where the user 51 wearing the display device 11 enters the floor 411 .
The control device 41 grasps the situation of the floor 411 by superimposing the map information 331 on the space of the floor 411 based on the detection of the AR marker 471 .

In this embodiment, hierarchical areas are defined. Specifically, the floor 411 includes the 8th machine. A 1-axis area 431, a 2-axis area 432, and a 3-axis area 433 are included in the car area 421 of the 8th car. Two device regions 461 and 462 are included in the one-axis region 441 . One device region 463 is included in the two-axis region 442 . Two instrument regions 464 and 465 are included in the triaxial region 443 .
In other words, the hierarchies are arranged in the order of "machine number", "axis", and "equipment" from the higher to the lower hierarchies.

In this embodiment, as a specific example, the case where the ambient information acquisition device of the display device 11 includes an imaging device that captures an image of the outside world of the display device 11 will be described. For this reason, as a specific example, a case where image information around the display device 11 is used as an example of information around the display device 11 will be described. Further, it may be used to improve accuracy. As another example, the information about the surroundings of the display device 11 may be configured to use detection information other than images, such as ultrasonic waves or temperature, instead of using image information.

When the user 51 wearing the display device 11 comes near the AR marker 471 on the floor 411 , the image of the AR marker 471 is captured by the display device 11 . Based on the image, the control device 41 recognizes the AR marker 471 and sets a virtual space on the floor 411 as shown in FIG. At this stage, the control device 41 determines that the position of the user 51 is in the car area 421 of car No. 8, and sets "car No. 8" or a related phrase as a keyword.

Assume that the user 51 then moves and enters the 1-axis area 441 . At this stage, the control device 41 determines that the position of the user 51 exists in the machine area 421 of the 8th machine and also exists in the 1-axis area 441. Set the phrases related to when you do or the phrases related to these as keywords.

Assume that the user 51 then moves and enters the device area 461 . At this stage, the control device 41 determines that the position of the user 51 exists in the machine region 421 of the 8th car, in the 1-axis region 441, and in the device region 461, and determines that "8th car" and " Axis 1” and the “equipment” of the equipment 451 are set as keywords.

In this way, the user 51 exists on the floor 411 and does not exist in a more detailed area than that, to the state where the user 51 exists in the car number 8 area 421 on the floor 411 but does not exist in a more detailed area. state, then to a state that exists in the machine number area 421 and the 1-axis area 441 but does not exist in the more detailed area, and then to a state that exists in the machine number area 421, the 1-axis area 441, and the device 451 Keywords related to more detailed areas are automatically narrowed down and set as the state shifts to the state where the
Here, in the present embodiment, keywords are narrowed down with an AND condition when going from a wide area to a detailed area, but the keywords may be set with other conditions.

Although the case where the user 51 moves from a wide area to a detailed area is shown here, conversely, when the user 51 moves from a detailed area to a wider area, the keywords are gradually narrowed down. is released. That is, in this case, the state in which the keywords are narrowed down in detail is shifted to the state in which the keywords are not so narrowed down.
In addition, although the movement of the user 51 is specified here, keywords are narrowed down according to the area including the position where the user 51 exists regardless of how the user 51 moves. becomes.

Here, various keywords may be used as keywords in each state.
For example, in a state where the user 51 exists on the floor 411 and does not exist in a more detailed area, the control device 41 sets keywords related to the floor 411 based on the map information 331 .
For example, when the user 51 exists in the car area 421 of car No. 8 on the floor 411 and does not exist in a more detailed area, the control device 41 detects car No. 8 on the floor 411 based on the map information 331. set keywords for In this state, the control device 41 may or may not set a keyword related to the floor 411, which is a wider area.

For example, when the user 51 exists in the machine area 421 and the 1-axis area 441 and does not exist in the more detailed area, the control device 41, based on the map information 331, moves to the 8th machine on the floor 411. A keyword related to a certain axis 431 is set. In this state, the control device 41 may or may not set one or more of the keywords related to the floor 411, which is a wider area, or the keywords related to the No. 8 car on the floor 411. .
For example, when the user 51 exists in the machine area 421, the 1st axis area 441, and the equipment area 461, the control device 41, based on the map information 331, controls the equipment on the 1st axis 431 in the 8th machine on the floor 411. Set keywords for 451. Note that in this state, the control device 41 does not specify any one or more of the keywords related to the floor 411 which is a wider area, the keywords related to the 8th car on the floor 411, or the keywords related to the 1st axis 431 in the 8th car on the floor 411. may be set or may not be set.

FIGS. 6A to 6E are diagrams showing examples of narrowing down keywords according to an embodiment of the present invention.
FIG. 6A shows keywords related to floors. In this example, there are "1FL" representing the first floor, "2FL" representing the second floor, and "3FL" representing the third floor.
FIG. 6(B) shows keywords related to the car on the second floor. In this example, there is a "2FL No. 7" representing the second floor, and a "2FL No. 8" representing the second floor, the No. 8.
FIG. 6C shows keywords related to axes. In this example, there are "single-axis", "two-axis", "three-axis", and "common" indicating common to all axes.
FIG. 6D shows keywords related to equipment. In this example, there are "Gland Steam Condenser Fan", "ST Main Valve", "Power Panel", and "Other".

FIG. 6(E) shows a state in which information retrieved using an arbitrary keyword is displayed after keywords are obtained as a result of gradually specifying a hierarchy such as floor, machine number, axis, and equipment. be.
In this embodiment, the control device 41 automatically sets a keyword based on the position and posture of the user 51 wearing the display device 11 and causes the display device 11 to display information on the set keyword. Such keyword information changes hierarchically, for example, as the user 51 moves through areas or devices hierarchically provided in the space.
Further, the control device 41 acquires the keyword selected by the user 51 based on the movement of the user's 51 head detected by the display device 11 . Then, the control device 41 searches the database 24 using the acquired keyword, and causes the display device 11 to display the information of the searched result.

Here, in this embodiment, the control device 41 accepts the selection of a keyword based on the movement of the head of the user 51, etc. As another example, the position and posture of the user 51 automatically select a keyword to be used for searching. may be set (selected).

FIGS. 7A to 7E are diagrams showing other examples of narrowing down keywords according to an embodiment of the present invention.
This example shows a case where the user 51 selects a keyword different from the keyword displayed by the display device 11 .

FIG. 7A shows keywords related to axes. In this example, there are "single-axis", "two-axis", "three-axis", and "common" indicating common to all axes.
For example, at this time, assume that the user 51 desires to set a keyword different from the keyword displayed in FIG. 7(A). In this example, the user 51 instructs to display the information shown in FIG. 7B by moving the head or the like.
FIG. 7B shows keywords displayed according to the user's 51 instructions. In this example, there are "turbine building", "HRSG", and "outdoor".

For example, at this time, assume that the user 51 instructs to select "turbine building".
FIG. 7C shows keywords displayed according to the user's 51 instructions. In this example, "1FL Turbine" for first floor turbines, "1FL other" for first floor others, "2FL Turbine" for second floor turbines, "2FL other" for second floor others , "3FL Turbine" for third floor turbines, and "3FL other" for others on third floor.

For example, at this time, assume that the user 51 instructs to select "2FL Turbine".
FIG. 7D shows keywords displayed according to the user's 51 instruction. In this example, there are "Gland Steam Condenser Fan", "ST Main Valve", "Power Panel", and "Other".

For example, at this time, assume that the user 51 instructs to perform a search using a desired keyword.
FIG. 7(E) shows a state in which information retrieved using the specified keyword is displayed.
In this embodiment, when a keyword is selected by movement of the head of the user 51 wearing the display device 11, the control device 41 searches the database 24 using the selected keyword, and searches the database 24. The resulting information is displayed by the display device 11 .

FIG. 8 is a diagram showing an example of information (display information 1011) displayed on the display device 11 according to one embodiment of the present invention.
The display information 1011 may be displayed at any time, for example, when keywords are being narrowed down.
In this example, for convenience of explanation, the top, bottom, left, and right when viewing the drawing will be used for explanation.
In the display information 1011, words associated with various instructions are displayed in the leftmost upper and lower columns. In the example of FIG. 8, "select" instructs to select desired information, "another device" instructs to select another device, and "peel off" instructs to deselect desired information. ', 'return' for returning to the previous state, and 'saisho e' for returning to the initial state.
Here, when the user 51 selects "another device", for example, as shown in FIG. make it possible.

Further, in the display information 1011, information regarding various characteristic values is displayed in the upper and lower columns from the second column onward from the left.
In this example, a plurality of sets of information consisting of an identifier, a name, a numerical value, and a unit are stored in the vertical direction. Specifically, the identifier "1A0000", the name "water supply temperature", the numerical value "40", and the unit "°C" are a set of information, and the identifier "1A0001", the name "water supply pressure", the numerical value "4", and the unit "MPa" as a set.

FIG. 9 is a diagram showing an example of pasting selection information according to one embodiment of the present invention.
FIG. 9 shows a water supply pipe 2011 and selection information 2021 .
First, it is assumed that the user 51 wearing the display device 11 can see the water supply pipe 2011 existing in the real world. Note that the water supply pipe 2011 may not exist in the real world, and as another example, it may exist in the virtual world.

Suppose that when the display information 1011 shown in FIG. 8 is displayed, the user 51 selects "select" and further selects a set of information "1A000 water supply temperature 40°C". . Then, the control device 41 causes the display device 11 to display selection information 2021 having the content of the selected set of information, as shown in FIG.

The user 51 can instruct to paste the selection information 2021 on or near the water supply pipe 2011 by moving the head or the like. In response to receiving such an instruction, the control device 41 pastes the selection information 2021 on or near the water supply pipe 2011 . In this example, pasting the selection information 2021 on or near the water supply pipe 2011 means displaying while maintaining the relative arrangement of the selection information 2021 and the water supply pipe 2011 .
Any operation may be used as the operation performed by the user 51 to attach the selection information 2021 to the water supply pipe 2011 or its vicinity.

In addition, the user 51 can instruct to peel off the selection information 2021 pasted on or near the water supply pipe 2011 by moving the head or the like. In this example, when the display information 1011 shown in FIG. 8 is displayed, it is assumed that the user 51 selects “peel off” and further selects the selection information 2021 . Then, the control device 41 erases the selection information 2021 pasted on or near the water supply pipe 2011 so that it is not displayed on the display device 11 .

FIG. 10 is a diagram showing an example of search result information 1021 according to one embodiment of the present invention.
In this example, for convenience of explanation, the top, bottom, left, and right when viewing the drawing will be used for explanation.
The search result information 1021 represents information of results of a search using a certain keyword.
In this example, a plurality of sets of information consisting of an identifier, a name, a unit, a date and a numerical value are stored in the horizontal direction.
Specifically, the identifier "2B0000", the name "fan inlet flow rate", the unit "%", the date "2018/8/26 17:00", and the numerical value "70 .1", which is a set of information including one or more vertical columns in which dates and numerical values are associated with each other.

Here, in the present embodiment, various processes may be performed according to the contents of instructions given by the user 51 on the display device 11 . For example, processing to delete some or all of the displayed keywords, processing to adjust the number of displayed keywords or the number of characters, processing to increase or decrease the number of displayed keywords, processing of part or all of the search result information as new keywords A process of executing a search, a process of switching between a state in which a list of keywords is displayed and a state in which it is not displayed, and the like may be performed.

<Example of procedure of processing performed in MR system>
FIG. 11 is a diagram showing an example of the procedure of processing performed in the control device 41 according to one embodiment of the present invention.
In this flow, processing for spatial recognition and association with equipment placement is performed.

(Step S1)
The control device 41 uses the space recognition section 351 to recognize the three-dimensional space around the display device 11 based on the surrounding information acquired by the display device 11 . Then, the process proceeds to step S2.
When the information of the image of the three-dimensional space captured by the display device 11 is used as the surrounding information, the three-dimensional space is recognized by the space recognition unit 351, for example.

(Step S2)
Based on the map information 331 , the control device 41 causes the device layout association unit 353 to associate the recognized three-dimensional space with the layout of the devices. Then, the processing of this flow ends.

FIG. 12 is a diagram showing an example of the procedure of processing performed by the control device 41 according to one embodiment of the present invention.
In this flow, processing using keywords is performed.

(Step S11)
The control device 41 acquires the position and posture of the user 51 wearing the display device 11 by the position/posture acquisition unit 354 . Then, the process proceeds to step S12.

(Step S12)
The control device 41 acquires the keyword by the keyword acquisition section 355 based on the position and posture of the user 51 . Then, the process proceeds to step S13.
Here, the acquired keyword may be displayed on the display device 11 by the display control unit 357 . Moreover, the keyword selected by the user 51 from among the displayed keywords may be accepted as a keyword for search by the search unit 356 .

(Step S13)
The control device 41 searches the database 24 using the search unit 356 using the acquired keyword. Then, the process proceeds to step S14.

(Step S14)
The control device 41 causes the display control section 357 to display information based on the search result. Then, the process proceeds to step S15.

(Step S15)
The control device 41 performs predetermined processing on the displayed information. Then, the processing of this flow ends.
Here, the predetermined processing may be arbitrary processing. Also, the predetermined process may be, for example, a predetermined process or a process instructed by the user 51 .

<Regarding the first embodiment>
As described above, in the MR system 1 according to this embodiment, for example, when MR is displayed by the display device 11 worn by the user 51, a data search key (in this embodiment, a keyword) is accepted. can be easily performed by the user 51 concerned. In the MR system 1 according to this embodiment, for example, when MR technology is applied to a vast area, it is possible to simplify retrieval of information for monitoring.

In the MR system 1 according to the present embodiment, the control device 41 performs spatial recognition of the floor or the like based on the surrounding information acquired by the display device 11 . Next, the control device 41 associates the layout of equipment and the like with the floor and the like obtained by spatial recognition. Then, the control device 41 determines the position of the user 51 inside the floor or the like by matching the reference point inside the floor or the like with the result of the space recognition. Then, the control device 41 causes the display device 11 to display the keyword information based on the position of the user 51 . In this case, when the user 51 approaches a specific device or the like, the control device 41 causes the display device 11 to display information such as keywords related to the device or the like. The control device 41 performs a search using a keyword based on an instruction given by the user 51 or the like, and causes the display device 11 to display the information of the search result.

As a specific example, in the MR system 1 according to this embodiment, the user 51 wearing the display device 11 departs from a central monitoring place or the like, arrives at a predetermined floor, and approaches a predetermined area or predetermined equipment. Keywords related to devices in the vicinity are automatically narrowed down as the search progresses.
In addition, in the MR system 1 according to the present embodiment, the user 51 wearing the display device 11 only performs a simple operation, thereby enabling automatic keyword narrowing down and automatic retrieval using the keyword. is. Further, the MR system 1 according to the present embodiment can be configured so that a search is automatically performed by narrowing down the keywords even if the user 51 wearing the display device 11 does not perform an operation.
In addition, in the MR system 1 according to the present embodiment, for example, since it is not necessary to install an AR marker for each device or the like, the number of AR markers can be reduced.

Further, in the MR system 1 according to the present embodiment, in accordance with an operation performed by the user 51 wearing the display device 11, a part or all of the information such as the keyword displayed by the display device 11 is transmitted to a predetermined device or the like. It is possible to paste the information on a predetermined place as shown in FIG. Therefore, in the MR system 1 according to the present embodiment, for example, when a predetermined work is performed by the user 51, information related to the work is attached to predetermined equipment or the like in the workplace and is visually recognized by the user 51. By setting the state, it is possible to facilitate the work for the user 51 .

[Second embodiment]
A second embodiment will be described.
In this embodiment, for convenience of explanation, the same reference numerals as those shown in FIG. 1 according to the first embodiment are used for explanation.

<Display device>
In the first embodiment, the MR system 1 is provided with the display device 11 and the control device 41 separately, and the control device 41 controls the display device 11 to display MR on the display device 11 .
In the present embodiment, the display device 11 is provided with functions similar to some or all of the functions of the control device 41 according to the first embodiment, so that the display device 11 alone can display MR. and

Schematically, the display device 11 according to the present embodiment has functions similar to those of the control unit 315 of the control device 41 according to the first embodiment, in addition to the functions of the first embodiment.
Further, in the display device 11 according to this embodiment, information similar to the map information 331 is stored in the storage unit 112 . As another example, the map information 331 may be stored in a storage unit (not shown) at a location other than the display device 11. In this case, the display device 11 may be stored in the storage unit as necessary. get the information
However, the function of performing communication between the display device 11 and the control device 41 performed in the first embodiment does not have to be provided in the display device 11 in the present embodiment.

As described above, in the display device 11 according to the present embodiment, as in the case of the first embodiment, for example, when the display device 11 worn by the user 51 displays MR, the data search key The user 51 can easily accept (in this embodiment, a keyword). In the display device 11 according to this embodiment, for example, when the MR technology is applied to a vast area, it is possible to simplify the retrieval of information for monitoring.

[Third embodiment]
A third embodiment will be described.
In this embodiment, for convenience of explanation, the same reference numerals as those shown in FIG. 1 according to the first embodiment are used for explanation.

<MR system>
In the MR system 1 according to this embodiment, as a whole, processing similar to that of the MR system 1 according to the first embodiment is performed.
In this embodiment, each of the functional units provided in each of the display device 11, the portable device 12, and the control device 41 of the server device 21 according to the first embodiment is provided in an arbitrary device in the MR system 1. may In other words, even if the MR system 1 is provided with each functional unit at an arbitrary location, it is sufficient that the same processing as the MR system 1 according to the first embodiment can be performed as a whole. If it is necessary to communicate between functional units provided in a plurality of different devices, necessary information is communicated between the plurality of different devices.

As described above, in the MR system 1 according to the present embodiment, as in the case of the first embodiment, for example, when the display device 11 worn by the user 51 displays the MR, the data search key The user 51 can easily accept (in this embodiment, a keyword). In the MR system 1 according to this embodiment, for example, when MR technology is applied to a vast area, it is possible to simplify retrieval of information for monitoring.

<Configuration example related to the above embodiment>
As one configuration example, the control device 41 receives information about the surroundings of the display device 11 which is worn by the user 51 and is capable of transmitting light in the real space to the display section and displaying a virtual image on the display section. A space recognition unit 351 that recognizes a space based on information acquired by a surrounding information acquisition device (in this embodiment, a surrounding information acquisition device included in the detection unit 114), and a space recognized by the space recognition unit 351 and predetermined map information 331; and a display control unit 357 that causes the display device 11 to display the keyword acquired by the keyword acquisition unit 355 .

As one configuration example, in the control device 41, the status of the display device 11 is the position of the display device 11 (in this embodiment, the position acquired by the position/orientation acquisition unit 354), the orientation of the display device 11 (in this embodiment, , orientation acquired by the position and orientation acquisition unit 354), or information acquired by the surrounding information acquisition device.
As one configuration example, in the control device 41 , the keyword changes hierarchically according to the status of the display device 11 .
As one configuration example, the control device 41 includes a search section 356 that searches using a keyword, and the display control section 357 causes the display device 11 to display information searched by the search section 356 .
As one configuration example, in the control device 41, the display control unit 357 selects information (selected information 2021 in the example of FIG. 9) searched by the keyword or the search unit 356 in accordance with an instruction given by the user 51. At least one of them is attached to a predetermined location (the water supply pipe 2011 in the example of FIG. 9) for display.

As one configuration example, it is possible to provide a program.
That is, the program acquires information about the surroundings of the display device 11 which is worn by the user 51 and is capable of transmitting light in the real space to the display unit and displaying a virtual image on the display unit. A function of recognizing a space based on information acquired by the device, a function of associating the recognized space with predetermined map information 331, and a function of acquiring a keyword based on the result of the association and the status of the display device 11. and a function of displaying the acquired keyword on the display device 11, in a computer.

As one configuration example, the system (the MR system 1 in this embodiment) is worn by the user 51 and can transmit light in the real space to the display unit to display a virtual image on the display unit. It has a display device 11 and a server device 21 .
The system includes a space recognition unit (in this embodiment, space recognition unit 351) that recognizes a space based on information acquired by an ambient information acquisition device that acquires information around the display device 11, and a space recognition unit. A correlating unit (equipment arrangement correlating unit 353 in this embodiment) that correlates the recognized space with predetermined map information 331, and a keyword based on the result of the correlating by the correlating unit and the status of the display device 11. A keyword acquisition unit (in this embodiment, the keyword acquisition unit 355) that acquires a keyword, and a display control unit (in this embodiment, the display control unit 357) that causes the display device 11 to display the keyword acquired by the keyword acquisition unit. Prepare.
Note that each functional unit provided in the system may be provided in either the display device 11 or the server device 21, for example. For example, some functional units may be provided in the display device 11 and other functional units may be provided in the server device 21 .

<Regarding the above embodiment>
Here, a program for realizing the function of each device (for example, the display device 11, the mobile device 12, the server device 21, the information processing device 22, the router device 25, the control device 41, etc.) according to the above-described embodiments may be recorded (stored) in a computer-readable recording medium, and the program recorded in the recording medium may be loaded into a computer system and executed to perform processing.
The “computer system” referred to here may include an operating system (OS) or hardware such as peripheral devices.
In addition, "computer-readable recording medium" includes a flexible disk, a magneto-optical disk, a ROM (Read Only Memory), a writable non-volatile memory such as a flash memory, a portable medium such as a DVD (Digital Versatile Disc), It refers to a storage device such as a hard disk built into a computer system.
Also, the computer-readable recording medium is, for example, a non-transitory recording medium.

Furthermore, "computer-readable recording medium" means a volatile memory (e.g., DRAM ( It also includes those that hold programs for a certain period of time, such as Dynamic Random Access Memory)).
Moreover, the above program may be transmitted from a computer system storing this program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in a transmission medium. Here, the "transmission medium" for transmitting the program means a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
Also, the above program may be for realizing part of the functions described above. Furthermore, the above program may be a so-called difference file (difference program) that can realize the functions described above in combination with a program already recorded in the computer system.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like are included within the scope of the present invention.

Reference Signs List 1 MR system 11 display device 12 portable device 21 server device 22 information processing device 23 display device 24 database 25 router device 31 network 51 user 61 Operator 111, 213, 313...Communication unit 112, 214, 314...Storage unit 113...Display unit 114...Detection unit 115, 216, 315...Control unit 131, 357...Display control unit 132...Detection Information transmission control unit 211, 311 input unit 212, 312 output unit 215 position detection unit 231 position information transmission control unit 331 map information 351 space recognition unit 352 map acquisition unit 353...Equipment layout association unit, 354...Position and orientation acquisition unit, 355...Keyword acquisition unit, 356...Search unit, 411...Floor, 421...Unit area, 431...1st axis, 432...2nd axis, 433...3rd axis, 441... 1 axis area, 442... 2 axis area, 443... 3 axis area, 451 to 455... equipment, 461 to 465... equipment area, 1011... display information, 1021... search result information, 2011... water supply pipe, 2021... selection information

Claims (6)

The information acquired by the ambient information acquisition device that acquires information around a display device that is worn by a user and is capable of transmitting light in a real space to a display unit and displaying a virtual image on the display unit. a space recognition unit that recognizes a space based on
an associating unit that associates the space recognized by the space recognition unit with predetermined map information;
a keyword acquiring unit that acquires a keyword that hierarchically changes according to the state of the display device based on the result of the association by the associating unit and the state of the display device ;
a display control unit that causes the display device to display the keyword acquired by the keyword acquisition unit;
A control device comprising: The status of the display device includes one or more of the position of the display device, the orientation of the display device, or the information acquired by the ambient information acquisition device,
A control device according to claim 1 . A search unit that performs a search using the keyword,
The display control unit causes the display device to display the information searched by the search unit.
The control device according to claim 1 or 2 . The display control unit performs display in which at least one of the keyword and the information searched by the search unit is pasted at a predetermined location in accordance with an instruction given by the user.
4. A control device according to claim 3 . The information acquired by the ambient information acquisition device that acquires information around a display device that is worn by a user and is capable of transmitting light in a real space to a display unit and displaying a virtual image on the display unit. and the ability to perceive space based on
a function of associating the recognized space with predetermined map information;
A function of acquiring keywords that hierarchically change according to the status of the display device based on the result of the association and the status of the display device ;
a function of displaying the acquired keyword on the display device;
A program for realizing on a computer. A system comprising: a display device worn by a user and capable of transmitting light in a real space to a display unit to display a virtual image on the display unit; and a server device,
a space recognition unit that recognizes a space based on the information acquired by a surrounding information acquisition device that acquires information around the display device;
an associating unit that associates the space recognized by the space recognition unit with predetermined map information;
a keyword acquiring unit that acquires a keyword that hierarchically changes according to the state of the display device based on the result of the association by the associating unit and the state of the display device ;
a display control unit that causes the display device to display the keyword acquired by the keyword acquisition unit;
comprising a
system.

Images