JP2017148373A - Detection system, communication system, and control method for communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible to accurately detect an activity state and a physical state of a person and to provide information.SOLUTION: A detection system 100 comprises: an HMD 20B worn around a head of a manager UM; and a terminal device 10 worn by the manager UM or users U1, U2. The terminal device 10 comprise: an activity amount detection unit for detecting activity amount information indicating an activity amount of the user U; a physical information detection unit for detecting physical information indicating a physical state; and a notification unit for making notification. The HMD 20B includes a display unit for allowing the manager UM to visually recognize the information. The detection system 100 performs at least one of notification by the notification unit and display by the display unit based on the activity amount information and physical information detected by the terminal device 10.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a detection system, a communication system, and a control method for the communication system.

  2. Description of the Related Art Conventionally, devices that monitor or detect the state of a human body and perform display or the like as necessary are known (see, for example, Patent Documents 1 and 2). An apparatus for the purpose of managing the body during diving is typical, and the underwater mask described in Patent Document 1 has a function of detecting an environmental value such as a water depth, measuring time, and displaying it. It also has a function to issue an alarm. If the underwater mask of patent document 1 is used, diving time, decompression time, etc. can be known. Moreover, the diving computer of patent document 2 utilizes heart rate information.

JP-A-58-221796 JP 2008-18928 A

There is a great need for providing information and warnings about the physical condition and mental effects of people. However, conventionally, as described in Patent Document 2, for example, only limited information such as heart rate is detected and provided.
The present invention has been made in view of the above circumstances, and provides a detection system, a communication system, and a control method for a communication system that can accurately detect the state of human activity and the state of the body and provide information. With the goal.

In order to achieve the above object, the present invention includes a display device worn on the head of a first user and a terminal device worn by the first user or the second user as the wearer. The terminal device includes: an activity amount detection unit that detects activity amount information indicating an activity amount of the wearer; a body information detection unit that detects body information indicating a state of the wearer's body; And the display device includes a display unit that allows the first user to visually recognize the information. Based on the activity amount information and the physical information, notification by the notification unit, and by the display unit It is characterized by performing at least one of the displays.
According to this configuration, notification or display is performed based on the information regarding the activity amount and the physical state of the wearer wearing the terminal device. Thereby, it is possible to promptly provide information to the user who uses the display device regarding the activity amount and physical condition of the wearer. For example, if the wearer's activity level or physical condition becomes a state requiring attention, this can be promptly notified to the user.

In the detection system, the present invention further includes a determination unit that determines whether or not notification is performed based on the activity amount information and the body information, and the notification unit performs the determination based on the determination result of the determination unit. Notification is performed.
According to this configuration, it is possible to appropriately determine whether or not the activity amount and / or the state of the body of the wearer is a state in which notification is to be performed and perform notification.

Further, according to the present invention, in the detection system, the determination unit determines whether at least one of a physical state and a mental state of the wearer is informed based on the activity amount information and the physical information. It is characterized by determining.
According to this configuration, it is possible to appropriately determine whether or not it is a state in which at least one of the body state and the mental state of the wearer wearing the terminal device should be notified. Thereby, for example, when the wearer's mental and physical disorder occurs, notification and display are performed, and notification and display can be performed at a stage where there is a high possibility of malfunction.

In the detection system, the terminal device includes a first communication unit, the determination unit, and a terminal control unit that transmits a determination result of the determination unit to the display device by the first communication unit. And the display device causes the display unit to perform display based on the determination result when the second communication unit and the determination result transmitted by the terminal device are received by the second communication unit. And a control unit.
According to this configuration, when it is determined that the terminal device determines the amount of activity or the state of the wearer wearing the terminal device and performs notification, the display device performs display based on the determination result. In this case, in the terminal device, detection and determination of information about the wearer is executed, and the display device performs display based on the determination result. Therefore, the amount of information transmitted from the terminal device to the display device and the frequency of transmission can be suppressed. In addition, since the processing load on the display device is small, it is possible to realize a configuration in which the display device receives and displays determination results from a plurality of terminal devices, for example.

In the detection system, the terminal device may include a first communication unit, the activity amount information detected by the activity amount detection unit, and the body information detected by the body information detection unit. A terminal control unit that transmits by one communication unit, and the display device receives a second communication unit and the activity amount information and the body information transmitted from the terminal device by the second communication unit. The terminal unit comprising: the determination unit that performs determination based on the activity amount information and the physical information; and a display device control unit that transmits a determination result of the determination unit to the terminal device by the second communication unit. The terminal control unit performs notification by the notification unit when the first communication unit receives a determination result transmitted from the display device.
According to this configuration, the display device receives and detects information detected by the terminal device regarding the activity amount and body condition of the wearer wearing the terminal device, and the determination result is received by the terminal device. Do. In this configuration, since the processing load on the terminal device is small, the terminal device can be easily downsized.

Moreover, the present invention is characterized in that, in the detection system, the physical information detection unit includes a heart rate sensor that detects a heart rate of the wearer.
According to this configuration, it is possible to perform notification and display that accurately reflects the state of the wearer using physical information including the heart rate of the wearer. Further, by detecting the heart rate affected by the psychological state of the wearer, it is possible to determine or estimate the mental state of the wearer, and to perform notification and display more accurately.

Moreover, the present invention is characterized in that, in the detection system, the body information detection unit includes a respiration rate sensor for detecting a respiration rate of the wearer.
According to this configuration, it is possible to perform notification and display that accurately reflects the state of the wearer using physical information including the breathing rate of the wearer. Further, by detecting the respiratory rate affected by the psychological state of the wearer, it is possible to determine or estimate the mental state of the wearer, and to perform notification and display more accurately.

In the detection system according to the present invention, the body information detection unit is a pressure sensor that detects an occlusal pressure of the wearer.
According to this configuration, it is possible to perform notification and display that accurately reflects the state of the wearer using the body information including the wearer's occlusal pressure. In particular, by detecting the occlusal pressure that reflects the psychological tension state of the wearer, it is possible to more accurately notify and display the wearer's mental state.

Moreover, the present invention is characterized in that, in the detection system, the notification unit includes a vibrator that generates vibration.
According to this configuration, it is possible to notify the wearer wearing the terminal device so that it can be easily noticed even in water or in a dark place.

In the detection system according to the present invention, the display device is configured integrally with goggles worn by the first user.
According to this configuration, in the environment where the goggles are worn, the information can be displayed so that the first user can easily recognize the goggles while wearing the goggles.

In the detection system according to the present invention, the display device can be moved to a position overlapping the goggles worn by the first user and outside the range visually recognized by the first user through the goggles. It is mounted.
According to this configuration, in the environment in which the goggles are worn, the state in which the first user can easily view the information displayed by the display device and the state in which the display device does not interfere with the visual field of the first user. realizable.

In order to achieve the above object, the present invention provides a plurality of users with a detection system including a display device worn on a user's head and a terminal device worn by the user. And each of the terminal devices detects an activity amount detection unit that detects activity amount information indicating the activity amount of the user and body information indicating a state of the user's body. A determination unit that determines whether or not to perform a notification based on the activity amount information and the physical information, a notification unit that performs notification, a first communication unit, and a determination by the determination unit A terminal control unit that causes the notification unit to perform notification based on a result, and transmits the determination result of the determination unit to the display device designated as a transmission destination among the plurality of display devices by the first communication unit. And comprising the above The display device is configured to display the display unit based on the determination result when the second communication unit receives the determination result transmitted from the display unit, the second communication unit, and the terminal device that allows the user to visually recognize the information. And a display device control unit that causes display to be performed.
According to this configuration, in a state where a plurality of users wear terminal devices and wear display devices, the respective terminal devices are based on the activity amount information and the physical information detected by the respective terminal devices as necessary. Notification is performed, and one of the display devices performs display. As a result, each user can be notified of the amount of activity and physical condition of the user wearing the terminal device, and information is provided by the display device used by any user. it can. Accordingly, any of the plurality of users can know the state of the body or activity. In addition, each user can quickly know when the amount of activity or physical condition of the user is in a state that requires attention. Further, since the processing load on the display device is small, even when the display device receives determination results from, for example, a plurality of terminal devices, it is possible to promptly display.

In order to achieve the above object, the present invention provides a plurality of users with a detection system including a display device worn on a user's head and a terminal device worn by the user. And each of the terminal devices detects an activity amount detection unit that detects activity amount information indicating the activity amount of the user and body information indicating a state of the user's body. A body information detection unit that performs notification, a notification unit that performs notification, a first communication unit, the activity amount information detected by the activity amount detection unit, and the body information detected by the body information detection unit, A terminal control unit that transmits to the display device specified as a transmission destination among the plurality of display devices by a communication unit, and at least one of the display devices causes the user to visually recognize information And second communication And determining whether or not it is in a state of performing notification based on the activity amount information and the physical information when the activity amount information and the physical information transmitted by the terminal device are received by the second communication unit And a display device control unit that displays on the display unit based on the determination result of the determination unit, and transmits the determination result of the determination unit to the terminal device by the second communication unit, the terminal device The terminal control unit performs notification by the notification unit when the first communication unit receives a determination result transmitted from the display device.
According to this configuration, in a state where a plurality of users wear terminal devices and wear display devices, the respective terminal devices are based on the activity amount information and the physical information detected by the respective terminal devices as necessary. Notification is performed, and one of the display devices performs display. As a result, each user can be notified of the amount of activity and physical condition of the user wearing the terminal device, and information is provided by the display device used by any user. it can. Accordingly, any of the plurality of users can know the state of the body or activity. In addition, each user can quickly know when the amount of activity or physical condition of the user is in a state that requires attention. Further, since the display device makes the determination based on the activity amount information and the physical information detected by the terminal device, the processing load on the terminal device is small and the terminal device can be easily downsized.

In order to achieve the above object, the present invention provides a plurality of users with a detection system including a display device worn on a user's head and a terminal device worn by the user. A plurality of communication system control methods, wherein each terminal device detects activity amount information indicating the activity amount of the user, and detects body information indicating a state of the user's body, Based on the activity amount information and the physical information, it is determined whether or not notification is in effect, notification is performed based on a determination result, and the determination result is designated as a transmission destination among the plurality of display devices. When the determination result transmitted to the display device and received by the terminal device is received by the display device, display is performed based on the determination result.
According to this configuration, in a state where a plurality of users wear terminal devices and wear display devices, the respective terminal devices are based on the activity amount information and the physical information detected by the respective terminal devices as necessary. Notification is performed, and one of the display devices performs display. As a result, each user can be notified of the amount of activity and physical condition of the user wearing the terminal device, and information is provided by the display device used by any user. it can. Accordingly, any of the plurality of users can know the state of the body or activity. In addition, each user can quickly know when the amount of activity or physical condition of the user is in a state that requires attention. Further, since the processing load on the display device is small, even when the display device receives determination results from, for example, a plurality of terminal devices, it is possible to promptly display.

In order to achieve the above object, the present invention provides a plurality of users with a detection system including a display device worn on a user's head and a terminal device worn by the user. A plurality of communication system control methods, wherein each terminal device detects activity amount information indicating the activity amount of the user, and detects body information indicating a state of the user's body, The detected activity amount information and the physical information are transmitted to the display device specified as a transmission destination among the plurality of display devices, and at least one of the display devices is transmitted by the terminal device. When activity amount information and physical information are received, it is determined based on the activity amount information and physical information whether or not a notification is made, and display is performed based on the determination result. Send device, by the terminal device, when the display device receives the determination result to send, it performs notification, characterized in that.
According to this configuration, in a state where a plurality of users wear terminal devices and wear display devices, the respective terminal devices are based on the activity amount information and the physical information detected by the respective terminal devices as necessary. Notification is performed, and one of the display devices performs display. As a result, each user can be notified of the amount of activity and physical condition of the user wearing the terminal device, and information is provided by the display device used by any user. it can. Accordingly, any of the plurality of users can know the state of the body or activity. In addition, each user can quickly know when the amount of activity or physical condition of the user is in a state that requires attention. Further, since the display device makes the determination based on the activity amount information and the physical information detected by the terminal device, the processing load on the terminal device is small and the terminal device can be easily downsized.

  The present invention can be realized as a program executed by each or at least one of the terminal device and the display device in order to realize the above-described control method of the communication system. Further, for example, the present invention can be realized in various forms such as a recording medium in which the program is recorded, a server device that distributes the program, a transmission medium that transmits the program, and a data signal in which the program is embodied in a carrier wave.

The figure which shows the example of application of the communication system in 1st Embodiment. The external appearance perspective view of a terminal device. The block diagram of a terminal device. The functional block diagram of the control part and memory | storage part of a terminal device. FIG. The block diagram of HMD. The schematic block diagram of the optical system of HMD. The functional block diagram of the control part and memory | storage part of HMD. The flowchart which shows operation | movement of a terminal device. The figure which shows the example of determination data typically. The flowchart which shows operation | movement of HMD. The sequence diagram which shows operation | movement of a communication system. The sequence diagram which shows operation | movement of a communication system. The figure which shows the example of a display in HMD. The flowchart which shows operation | movement of the terminal device in 2nd Embodiment. The flowchart which shows operation | movement of HMD in 2nd Embodiment. The sequence diagram which shows operation | movement of the communication system in 2nd Embodiment. The sequence diagram which shows operation | movement of the communication system in 2nd Embodiment. The principal part side view which shows the structure of HMD of 3rd Embodiment. The perspective view which shows the structure of HMD of 3rd Embodiment. The principal part external view of the detection apparatus of 4th Embodiment. The block diagram of the detection apparatus of 4th Embodiment.

[First Embodiment]
FIG. 1 is a schematic diagram showing an application example of a communication system 1 according to the first embodiment to which the present invention is applied.
The communication system 1 includes a plurality of terminal devices 10 worn by each of a plurality of users (users and wearers) and a plurality of HMDs 20 (Head Mounted Display). A pair of terminal devices 10 and the HMD 20 constitute a detection system 100. The communication system 1 is configured by each of a plurality of users wearing the detection system 100.

  FIG. 1 shows an application example of the communication system 1 of the first embodiment. In this application example, the user U1, UM, U2 wears the detection system 100 when carrying out scuba diving with the diving equipment. Examples of the diving equipment include a diving suit, an air tank, a regulator, a mask, and fins. The user UM is an administrator who leads diving, and plays the role of a leader or an instructor. The users U1 and U2 perform diving under the management of the user UM. In the following description, the user UM may be called an administrator. The administrator UM corresponds to the first user of the present invention, and the users U1 and U2 correspond to the second user.

  In the application example of FIG. 1, an example will be described in which the user U1 wears the detection system 100A, the user UM wears the detection system 100B, and the user U2 wears the detection system 100C. The detection system 100A includes a terminal device 10A and an HMD 20A, the detection system 100B includes a terminal device 10B and an HMD 20B, and the detection system 100 includes a terminal device 10C and an HMD 20C. In the following description, when the detection systems 100A, 100B, and 100C are not distinguished, they are referred to as the detection system 100. Similarly, when the terminal devices 10A, 10B, and 10C are not distinguished from each other, they are represented as the terminal device 10, and when the HMDs 20A, 20B, and 20C are not distinguished from each other, they are represented as the HMD 20. Moreover, when not distinguishing user U1, UM, and U2, it describes with the user U.

  The terminal device 10 functions as a dive computer. The terminal device 10 has a wristwatch shape, and is used by the user U wearing on the arm. That is, the terminal device 10 functions as a timer counter that measures the time from the start of diving and a water depth meter that detects and displays the water depth. Further, the terminal device 10 may have a function of calculating and displaying a no-decompression diving time during diving.

  The HMD 20 has a function of a mask that is a diving equipment. That is, it has a glass or resin shield that secures a field of view, a frame that holds the shield, and a skirt that is in close contact with the face of the user U to make the space inside the frame watertight. The HMD 20 functions as a display device that displays an image (including a virtual image to be described later) on at least a part of the shield so that the user U can visually recognize the image. The configuration and function as a display device will be described later.

  In the communication system 1, a plurality of terminal devices 10A, 10B, and 10C worn by the users U1, UM, and U2 can communicate with the HMDs 20A, 20B, and 20C worn by the users U1, UM, and U2, respectively. In the present embodiment, as an example, an example will be described in which the terminal device 10A communicates with the MDs 20A and 20B, the terminal device 10B communicates with the HMDs 20A and 20B, and the terminal device 10C communicates with the HMD 20B.

FIG. 2 is an external perspective view of the terminal device 10. Since the terminal devices 10A, 10B, and 10C all have a common configuration, the terminal device 10 will be described here.
The terminal device 10 includes a disk-shaped case 11 and a belt 12 that fixes the case 11 to the arms of the users U1, UM, and U2. A display 13 for displaying characters and images is disposed on the surface of the case 11. In the example of FIG. 2, the display 13 is provided with a user status display unit 131 described later. An operation button 14 is disposed on the side surface of the case 11.

FIG. 3 is a block diagram of the terminal device 10.
The terminal device 10 includes a main processor 101 that executes a program and controls the terminal device 10. A memory 103 and a non-volatile storage unit 102 are connected to the main processor 101. The nonvolatile storage unit 102 includes a semiconductor storage device such as an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, an eMMC (embedded Multi Media Card), or other nonvolatile storage devices. The nonvolatile storage unit 102 stores a program executed by the main processor 101 and various data processed by the main processor 101 executing the program. The memory 103 constitutes a work area that temporarily stores a program to be executed and data to be processed when the main processor 101 executes the program.

  An input unit 112 is connected to the main processor 101. The input unit 112 detects the operation of the button 14 and outputs operation data to the main processor 101. Here, the number of buttons 14 is not limited, and a plurality of buttons 14 may be arranged in the case 11 (FIG. 2). In addition, an operation unit such as a switch or a touch panel other than the button 14 can be provided in the terminal device 10. In this case, these operation units are connected to the input unit 112, and the input unit 112 detects the operation of the operation unit. .

  A communication unit 111 is connected to the main processor 101. The communication unit 111 performs wireless communication with an external device. The communication unit 111 according to the present embodiment includes a configuration capable of performing wireless communication using a communication method suitable for underwater wireless communication. For example, the communication unit 111 can be configured to perform wireless communication using sound waves or ultrasonic waves. In this case, the communication unit 111 includes a sound output unit, a microphone, an encoding / decoding unit, and the like (not shown). Alternatively, the communication unit 111 can be configured to execute wireless communication using visible light or infrared light. In this case, the communication unit 111 includes a light emitting unit, a light receiving unit, an encoding / decoding unit, and the like.

  In addition, the communication unit 111 may perform wireless communication using radio waves. In this case, the communication unit 111 includes an RF unit, a baseband unit, an encode / decode unit, and the like (not shown). The communication unit 111 transmits and receives a radio signal having a frequency suitable for wireless communication in water (for example, 10 kHz or less, 100 k to 200 kHz, 10 MHz or less, etc.).

  In addition, the communication unit 111 includes a configuration that performs wireless communication according to standards such as Bluetooth (registered trademark) and wireless LAN (including Wi-Fi (registered trademark)) as a communication method suitable for wireless communication in the atmosphere. Also good. For example, the communication unit 111 may be configured to be able to switch between a communication method suitable for wireless communication in water and a communication method suitable for wireless communication in the atmosphere.

  A display 13 is connected to the main processor 101. The display 13 includes a segment display panel, a liquid crystal display panel, an OLED (Organic Light Emitting Diode) display panel, and the like. The display 13 may include a drive circuit that drives the display panel, and displays characters, symbols, figures, images, and the like under the control of the main processor 101.

  A vibrator 15 is connected to the main processor 101. The vibrator 15 includes a motor (not shown), an eccentric rotor (not shown), and may have other necessary configurations. The vibrator 15 generates vibration by rotating the motor according to the control of the main processor 101. Since the vibration of the vibrator 15 is transmitted from the case 11 to the user U's arm, the terminal device 10 can reliably notify the user U.

The main processor 101 is connected with a motion sensor 113, a pulse sensor 114, and a pressure sensor 115 as sensors.
The motion sensor 113 (activity amount information detection unit) is a sensor that detects the motion of the case 11, and is specifically an acceleration sensor or a gyro (angular velocity) sensor. The motion sensor 113 may be a six-axis sensor including a three-axis acceleration sensor and a three-axis gyro sensor, for example. The motion sensor 113 may be an IMU (Inertial Measurement Unit) in which an acceleration sensor and / or a gyro sensor is modularized.

The pulse sensor 114 (body information detection unit) is a sensor that detects the pulse wave of the user U wearing the terminal device 10. The pulse sensor 114 is provided on, for example, the bottom surface of the case 11, that is, the surface that contacts the user U's body. The pulse sensor 114 may be, for example, a light detection type pulse wave sensor that emits light from the bottom surface of the case 11 and detects a pulse wave by detecting reflected light of the light. In this case, the pulse sensor 114 includes a light source such as an LED and a light receiving unit.
Further, the pulse sensor 114 functions as an oxygen saturation measuring device (pulse oximeter) that detects blood oxygen saturation (SpO ₂ ) by irradiating the epidermis of the user U's arm with red light or infrared light. It may be. In this case, the pulse sensor 114 can detect and measure the pulse rate (heart rate) and blood oxygen saturation of the user U.

  The pressure sensor 115 is a sensor that detects atmospheric pressure and / or water pressure around the case 11.

  The main processor 101 detects the movement of the case 11 based on the detection value of the motion sensor 113. Based on the movement of the case 11, the main processor 101 obtains the movement of the user U wearing the terminal device 10. Further, the main processor 101 calculates the pulse rate per minute of the user U wearing the terminal device 10 based on the detection value of the pulse sensor 114. Further, the main processor 101 determines whether the terminal device 10 is in the air or in the water based on the detection value of the pressure sensor 115, and calculates the water depth when determining that the terminal device 10 is in the water.

  The terminal device 10 includes a power supply unit 116 and operates with power supplied from the power supply unit 116. The power supply unit 116 includes a rechargeable battery 118 and a power supply control circuit 117 that detects the remaining capacity of the battery 118 and controls charging of the battery 118. The power control circuit 117 is connected to the main processor 101, and outputs a detected value of the remaining capacity of the battery 118 or a detected value of the voltage to the main processor 101. The power supply state from the power supply unit 116 to each unit of the terminal device 10 may be configured to be controllable by the main processor 101.

  FIG. 4 is a functional block diagram of the storage unit 32 and the control unit 31 constituting the control system of the terminal device 10. The storage unit 32 illustrated in FIG. 4 is a logical storage unit configured by the nonvolatile storage unit 102 (FIG. 3). The control unit 31 and various functional units included in the control unit 31 are formed by the cooperation of software and hardware when the main processor 101 executes a program. The control unit 31 and each functional unit constituting the control unit 31 include, for example, a main processor 101, a nonvolatile storage unit 102, and a memory 103.

  The control unit 31 (terminal control unit) executes various processes using data stored in the storage unit 32 and controls the terminal device 10. The storage unit 32 stores various data processed by the control unit 31. The storage unit 32 stores setting data 321, communication setting data 323, and determination data 325. The setting data 321 includes various setting values related to the operation of the terminal device 10. When the control unit 31 controls the terminal device 10, parameters, determinants, arithmetic expressions, LUTs (LookUp Tables), and the like may be included in the setting data 321.

  The communication setting data 323 includes information specifying the counterpart device with which the terminal device 10 communicates, and information necessary for communicating with the counterpart device. In the present embodiment, the terminal device 10A communicates with the HMDs 20A and 20B. Further, the terminal device 10B communicates with the HMDs 20A and 20B, and the terminal device 10C communicates with the HMD 20B. The communication setting data 323 includes, for example, a communication ID and name of the counterpart device as information for specifying the counterpart device with which the communication is performed. Further, the communication setting data 323 may include information specifying a communication method used when communicating with each counterpart device, and information such as a password and an ID for performing authentication with the counterpart device. .

The determination data 325 includes information related to processing for determining the state of the user U wearing the terminal device 10. The communication system 1 monitors or manages the states of a plurality of users U using the detection systems 100A, 100B, and 100C. In the communication system 1, it is determined whether or not the state of each user U is a state that requires notification, and notification is performed when it is determined that the state requires notification. Here, the state requiring notification indicates a state corresponding to a preset condition. Specifically, for the corresponding user U, a state in which a medical or scientific abnormality has occurred, a state in which the possibility of occurrence of this abnormality has become higher than normal, a state in which the occurrence of an abnormality is a concern, or an abnormality has occurred This is a state where other users U should pay attention to whether or not to do so. In addition, regardless of the abnormality, the user U may simply include a state in which another user U should pay more attention than usual.
The determination data 325 is information used when the terminal device 10 determines whether or not the state of the user U wearing the terminal device 10 is a state requiring notification. Contains information on standards. Further, the determination data 325 may include a determination result determined for the user U wearing the terminal device 10.

  The control unit 31 has functions of an operating system (OS) 311, a detection control unit 313, a communication control unit 315, and a display control unit 317. The functions of the operating system 311 are functions of a control program stored in the storage unit 32, and the other units are functions of application programs executed on the operating system 311.

The detection control unit 313 executes detection by the motion sensor 113, the pulse sensor 114, and the pressure sensor 115, and acquires a detection value.
The detection control unit 313 obtains the motion of the case 11 based on the detection value of the motion sensor 113, and calculates the activity amount of the user U from the obtained motion. The detection control unit 313 outputs activity amount data indicating the activity amount of the user U. Further, the detection control unit 313 calculates the pulse rate per minute of the user U wearing the terminal device 10 based on the detection value of the pulse sensor 114. Further, the detection control unit 313 determines whether the terminal device 10 is in the air or in the water based on the detection value of the pressure sensor 115, and calculates the water depth when determining that the terminal device 10 is in the water. Here, the detection control unit 313 may generate activity data of the user U, pulse data indicating the pulse rate per minute of the user U, water depth data indicating the water depth, and the like, and store the generated data in the storage unit 32.

  The detection control unit 313 makes a determination regarding the state of the user U based on detection values of the motion sensor 113, the pulse sensor 114, and the pressure sensor 115, or data obtained from these detection values (hereinafter referred to as determination data). Do. The detection control unit 313 determines that the state of the user U is a state to be notified when the determination data corresponds to a condition or a standard set in the determination data 325. In the present embodiment, the detection control unit 313 corresponds to a determination unit.

  The determination data may include data regarding the mental state of the user U. In this case, the detection control unit 313 obtains, for example, the strength of the stress felt by the user U, and outputs stress level data indicating the obtained strength of the stress stepwise. The detection control unit 313 performs determination based on the detection values of the motion sensor 113, the pulse sensor 114, and the pressure sensor 115, and the stress level data. In the process for obtaining the strength of the stress, the blood oxygen saturation or the respiratory rate described later may be used.

  When the detection control unit 313 determines that the state of the user U is a state to be notified, the detection control unit 313 performs a notification operation. In the notification operation, the detection control unit 313 causes the display control unit 317 described later to execute display for notification. Moreover, the detection control part 313 vibrates the vibrator 15 in alerting | reporting operation | movement. Here, the vibrator 15 corresponds to a notification unit. Moreover, when the detection control part 313 alert | reports by the display of the display 13, the display 13 is corresponded to an alerting | reporting part.

  The communication control unit 315 controls the communication unit 111 based on the communication setting data 323 and executes communication with an external device. For example, the communication control unit 315 transmits the determination data determined by the detection control unit 313 and data including the determination result to the other terminal device 10 or the HMD 20. Further, the communication control unit 315 receives data from, for example, another terminal device 10 or the HMD 20 and stores the data in the storage unit 32.

  The display control unit 317 displays characters, images, and the like on the display 13 (FIG. 2) based on various data output from the detection control unit 313. In the example of FIG. 2, under the control of the detection control unit 313, the pulse rate per minute, the amount of activity, and the strength of stress of the user U wearing the terminal device 10 are displayed on the user state display unit 131. Is done. In addition, when the detection control unit 313 determines that the state of the user U is a state to be notified, display for notification is performed according to the control of the detection control unit 313.

  FIG. 5 is an external view of the HMD 20 and is a front view showing a main part of the HMD 20. Since the HMDs 20A, 20B, and 20C all have a common configuration, the HMD 20 will be described here.

As described above, the HMD 20 has a structure as a mask (also referred to as goggles) that secures a field of view in water. The HMD 20 includes a spectacle-shaped frame 21 and a skirt 23 attached around the frame 21. The frame 21 of this embodiment is a two-lens type in which a frame corresponding to the right eye of the user U and a frame corresponding to the left eye are divided. A right light guide plate 26 and a left light guide plate 28 are fitted into the frame 21.
The right light guide plate 26 and the left light guide plate 28 are transparent or translucent members so as to secure the field of view of the user U. The right light guide plate 26 and the left light guide plate 28 ensure the airtightness of the space surrounded by the frame 21, the skirt 23, the right light guide plate 26, the left light guide plate 28, and the face of the user U. Further, the right light guide plate 26 and the left light guide plate 28 display images as will be described later. An area where the user U visually recognizes an image displayed by the right light guide plate 26 is a display area 26V, and an area where the user U visually recognizes an image displayed by the left light guide plate 28 is a display area 28V. The display area 26V and the display area 28V are areas where the user U recognizes images, and do not mean that images are formed in the display areas 26V and 28V.

FIG. 6 is a block diagram of the HMD 20.
The HMD 20 includes a main processor 201 that executes a program and controls the HMD 20. A memory 205 and a nonvolatile storage unit 203 are connected to the main processor 201. The nonvolatile storage unit 203 is configured by a semiconductor storage device such as an EEPROM, a flash memory, or eMMC, or other nonvolatile storage devices. The nonvolatile storage unit 203 stores a program executed by the main processor 201 and various data processed by the main processor 201 executing the program.

  An input unit 213 is connected to the main processor 201. The input unit 213 detects an input operation by the user U and outputs operation data to the main processor 201. When the HMD 20 includes an operation unit such as a button or a switch (not shown), the input unit 213 detects an operation of the operation unit.

  A communication unit 211 is connected to the main processor 201. The communication unit 211 performs wireless communication with an external device. The communication unit 211 of the present embodiment is configured to be capable of wireless communication with the communication unit 111 provided in the terminal device 10. For this reason, the communication unit 211 has a configuration common to or compatible with the communication unit 111. The communication unit 211 includes a configuration capable of performing wireless communication with a communication method suitable for underwater wireless communication. For example, the communication unit 211 can be configured to execute wireless communication using sound waves or ultrasonic waves. In this case, the communication unit 211 includes a sound output unit, a microphone, an encoding / decoding unit, and the like (not shown). Alternatively, the communication unit 211 can be configured to execute wireless communication using visible light or infrared light. In this case, the communication unit 211 includes a light emitting unit, a light receiving unit, an encoding / decoding unit, and the like.

  In addition, the communication unit 211 may execute wireless communication using radio waves. In this case, the communication unit 211 includes an RF unit, a baseband unit, an encoding / decoding unit, and the like (not shown). The communication unit 211 transmits and receives a radio signal having a frequency suitable for wireless communication in water (for example, 10 kHz or less, 100 k to 200 kHz, 10 MHz or less, etc.).

  Further, the communication unit 211 may include a configuration for performing wireless communication according to standards such as Bluetooth and wireless LAN (including Wi-Fi) as a communication method suitable for wireless communication in the atmosphere. For example, the communication unit 211 may be configured to be able to switch between a communication method suitable for underwater wireless communication and a communication method suitable for wireless communication in the atmosphere.

  The illuminance sensor 214 is provided on the front surface of the frame 21 of the HMD 20, for example, and is arranged to receive external light from the front of the user U wearing the HMD 20. The illuminance sensor 214 outputs a detection value corresponding to the amount of received light (received light intensity).

  The HMD 20 includes a power supply unit 248 and operates with power supplied from the power supply unit 248. The power supply unit 248 includes a rechargeable battery 249 and a power supply control circuit 250 that detects the remaining capacity of the battery 249 and controls charging of the battery 249. The power control circuit 250 is connected to the main processor 201 and outputs a detected value of the remaining capacity of the battery 249 or a detected value of the voltage to the main processor 201. The power supply state from the power supply unit 248 to each unit of the HMD 20 may be configured to be controllable by the main processor 201.

  The main processor 201 is connected to the right display unit 22 and the left display unit 24. The right display unit 22 is a unit related to image display by the right light guide plate 26 (FIG. 5), and is located in the vicinity of the right side of the user when the HMD 20 is worn. The left display unit 24 is a unit related to image display by the left light guide plate 28 and is located in the vicinity of the user's left head when the HMD 20 is worn. The right display unit 22 and the left display unit 24 are also simply referred to as “display drive unit”.

The right light guide plate 26 and the left light guide plate 28 of the present embodiment are optical components formed of a light transmissive resin or the like, and are prisms, for example. The right light guide plate 26 and the left light guide plate 28 guide the image light output from the right display unit 22 and the left display unit 24 to the eyes of the user.
The configuration of the right display unit 22 and the left display unit 24 will be described with reference to FIG.

FIG. 7 is a plan view of a principal part showing a configuration of an optical system provided in the HMD 20. FIG. 7 shows the user's left eye LE and right eye RE for explanation.
As shown in FIG. 7, the right display unit 22 and the left display unit 24 are configured symmetrically. The right display unit 22 has a configuration that allows the user's right eye RE to visually recognize an image. Specifically, the right display unit 22 includes an OLED unit 221 that emits image light and a right optical system 251. The right optical system 251 includes an optical member that guides the image light L emitted from the OLED unit 221. The image light L is guided to the right light guide plate 26 by the right optical system 251. Note that the video projection units of the right display unit 22 and the left display unit 24 are not limited to OLED units, and liquid crystal, DMD (Digital Micromirror Device), or the like may be used.

The OLED unit 221 includes an OLED panel 223 and an OLED drive circuit 225 that drives the OLED panel 223. The OLED panel 223 is a self-luminous display configured by arranging light emitting elements that emit light of R (red), G (green), and B (blue) light by organic electroluminescence in a matrix. It is a panel. The OLED panel 223 includes a plurality of pixels, each of which includes one R, G, and B element, and forms an image with pixels arranged in a matrix. The OLED drive circuit 225 causes the light emitting elements of the OLED panel 223 to emit light by selecting the light emitting elements included in the OLED panel 223 and energizing the light emitting elements under the control of the control unit 51. The OLED drive circuit 225 is fixed to the back surface of the OLED panel 223, that is, the back surface of the light emitting surface by bonding or the like. The OLED drive circuit 225 may be formed of a semiconductor device that drives the OLED panel 223, for example, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 223. A temperature sensor 217 is mounted on this substrate.
Note that the OLED panel 223 may have a configuration in which light emitting elements that emit white light are arranged in a matrix, and color filters corresponding to R, G, and B colors are stacked. Further, an OLED panel 223 having a WRGB configuration provided with a light emitting element that emits W (white) light in addition to the light emitting elements that respectively emit R, G, and B color light may be used.

  The right optical system 251 includes a collimator lens that converts the image light L emitted from the OLED panel 223 into a parallel light flux. The image light L converted into a parallel light beam by the collimator lens enters the right light guide plate 26. A plurality of reflecting surfaces that reflect the image light L are formed in an optical path that guides light inside the right light guide plate 26. The image light L is guided to the right eye RE side through a plurality of reflections inside the right light guide plate 26. A half mirror 261 (reflection surface) located in front of the right eye RE is formed on the right light guide plate 26. The image light L is reflected by the half mirror 261 and emitted from the right light guide plate 26 toward the right eye RE. The image light L forms an image on the retina of the right eye RE, and allows the user to visually recognize the image.

  The left display unit 24 has a configuration for allowing the user's left eye LE to visually recognize an image. Specifically, the left display unit 24 includes an OLED unit 241 that emits image light, and a left optical system 252 that includes an optical component that guides the image light L emitted from the OLED unit 241. The image light L is guided to the left light guide plate 28 by the left optical system 252.

  The OLED unit 241 includes an OLED panel 243 and an OLED drive circuit 245 that drives the OLED panel 243. The OLED panel 243 is a self-luminous display panel configured similarly to the OLED panel 223. The OLED drive circuit 245 causes the light emitting elements of the OLED panel 243 to emit light by selecting the light emitting elements included in the OLED panel 243 and energizing the light emitting elements under the control of the control unit 51. The OLED drive circuit 245 is fixed to the back surface of the OLED panel 243, that is, the back surface of the light emitting surface by bonding or the like. The OLED drive circuit 245 may be configured by a semiconductor device that drives the OLED panel 243, for example, and may be mounted on a substrate (not shown) fixed to the back surface of the OLED panel 243.

  The left optical system 252 includes a collimating lens that converts the image light L emitted from the OLED panel 243 into a light beam in a parallel state. The image light L converted into a parallel light beam by the collimator lens enters the left light guide plate 28. The left light guide plate 28 is an optical element formed with a plurality of reflecting surfaces that reflect the image light L, and is, for example, a prism. The image light L is guided to the left eye LE side through a plurality of reflections inside the left light guide plate 28. The left light guide plate 28 is formed with a half mirror 281 (reflection surface) located in front of the left eye LE. The image light L is reflected by the half mirror 281 and emitted from the left light guide plate 28 toward the left eye LE. The image light L forms an image on the retina of the left eye LE, and allows the user to visually recognize the image. In addition, the reflective surface of the right light guide plate 26 and the left light guide plate 28 on the side of the external light OL may be provided with an air or vacuum layer (not shown) inside the light guide plate. By providing an air or vacuum layer inside the light guide plate, it is possible to prevent the reflective surface from coming into contact with water, and to eliminate a decrease in the refractive index difference of the reflective surface and the influence of water flow on the reflective surface.

  The HMD 20 functions as a see-through display device. That is, the image light L reflected by the half mirror 261 and the external light OL transmitted through the right light guide plate 26 are incident on the user's right eye RE. Image light L reflected by the half mirror 281 and external light OL transmitted through the half mirror 281 are incident on the left eye LE. In this manner, the HMD 20 causes the image light L of the internally processed image and the external light OL to overlap each other and enter the user's eyes, and for the user, the outside light can be seen through the right light guide plate 26 and the left light guide plate 28. The image by the image light L is visually recognized over the outside scene.

  The half mirrors 261 and 281 are image extraction units that extract the image by reflecting the image light output from the right display unit 22 and the left display unit 24, respectively. Although it can be said that the right display unit 22 and the left display unit 24 correspond to the display unit of the present invention, the half mirrors 261 and 281 can also be called display units.

  FIG. 8 is a functional block diagram of the storage unit 53 and the control unit 51 that constitute the control system of the terminal device 10. The storage unit 53 illustrated in FIG. 8 is a logical storage unit configured by the nonvolatile storage unit 203 (FIG. 6). The control unit 51 and various functional units included in the control unit 51 are formed by cooperation of software and hardware when the main processor 201 executes a program. The control unit 51 and each functional unit constituting the control unit 51 include, for example, a main processor 201, a nonvolatile storage unit 203, and a memory 205.

  The control unit 51 (display device control unit) executes various processes using data stored in the storage unit 53 and controls the HMD 20. The storage unit 53 stores various data processed by the control unit 51. The storage unit 53 stores setting data 531, communication setting data 533, and display setting data 535. The setting data 531 includes various setting values related to the operation of the HMD 20. When the controller 51 controls the HMD 20, parameters, determinants, arithmetic expressions, LUTs, and the like may be included in the setting data 531.

  The communication setting data 533 includes information specifying the counterpart device with which the HMD 20 communicates, and information necessary for communicating with the counterpart device. In the present embodiment, the HMD 20A communicates with the terminal devices 10A and 10B. The HMD 20B communicates with the terminal devices 10A, 10B, and 10C, and the HMD 20C communicates with the terminal devices 10B and 10C. The communication setting data 533 includes, for example, a communication ID and name of the counterpart device as information for specifying the counterpart device with which the communication is performed. Further, the communication setting data 533 may include information specifying a communication method used when communicating with each counterpart device, and information such as a password and an ID for performing authentication with the counterpart device. .

  The display setting data 535 includes information related to settings when displaying an image on the HMD 20. Specifically, the display setting data 535 includes information that specifies the display position, display size, brightness, and the like of the image in the display area of the HMD 20 for each attribute of the image to be displayed. Further, the display setting data 535 includes an image that is displayed when notification is performed, and information that specifies a display position and a display size of the notification image.

  The control unit 51 has functions of an operating system (OS) 511, a communication control unit 513, and a display control unit 515. The functions of the operating system 511 are functions of a control program stored in the storage unit 53, and the other units are functions of application programs executed on the operating system 511.

The communication control unit 513 controls the communication unit 211 based on the communication setting data 533 and executes communication with an external device. For example, the communication control unit 515 performs wireless communication with the terminal devices 10A, 10B, and 10C, and receives data including a determination result. The communication control unit 513 stores the received data in the storage unit 53.
The communication control unit 513 causes the display control unit 515 to display the determination data included in the data received from the terminal devices 10A, 10B, and 10C in association with each terminal device 10. In addition, when the determination result included in the data received from any of the terminal devices 10A, 10B, and 10C indicates that the communication control unit 513 is to be notified, the communication control unit 513 uses the display setting data 535 to display an image for notification. Is displayed.

  The display controller 515 causes the right display unit 22 and the left display unit 24 to display characters, images, and the like. The display control unit 515 displays the determination data received by the communication control unit 513 by the right display unit 22 and the left display unit 24 in association with the terminal device 10 that transmitted the determination data. The display control unit 515 displays an image for notification when the determination result included in the data received by the communication control unit 513 indicates that the notification should be performed.

FIG. 9 is a flowchart showing the operation of the terminal device 10.
Since the operation of FIG. 9 is common to the terminal devices 10A, 10B, and 10C, the operation of the terminal device 10 will be described here.
When the terminal device 10 is powered on, the control unit 31 starts operating (step S11). The control unit 31 acquires detection values of the motion sensor 113, the pulse sensor 114, and the pressure sensor 115 (step S12). In step S12, the control unit 31 may generate the stress level data of the user U based on the acquired detection value. The control unit 31 refers to the determination data 325 (step S13) and makes a determination based on the determination criteria or conditions included in the determination data 325 (step S14). The control unit 31 determines whether or not the detection value and / or the stress level data acquired in step S12 corresponds to a state to be notified defined by the determination data 325 (step S15).

  When it determines with falling into the state which should alert | report (step S15; Yes), the control part 31 performs alerting | reporting (step S16). In step S <b> 16, the control unit 31 operates the vibrator 15 and performs display for notification on the display 13.

  When it determines with not being in the state which should be alert | reported (step S15; No), and after performing alerting | reporting of step S16, the control part 31 sets the data containing a determination result by the communication setting data 323. The data is transmitted to the transmission destination (step S17). In step S17, the control unit 31 may transmit data including the determination result in step S14 and the detection value acquired in step S12, or may transmit data including stress level data.

  The control unit 31 determines whether or not an end condition for ending the operation is satisfied (step S18). If the end condition is not satisfied (step S18; No), the control unit 31 returns to step S12. The end condition is that an operation end instruction is input by operating the button 14, the power off operation is performed by a power switch (not shown), the remaining amount of the battery 118 is insufficient, and the like. When the end condition is satisfied (step S18; Yes), the control unit 31 ends the process.

FIG. 10 is a diagram schematically illustrating an example of the determination data 325.
The determination data 325 illustrated in FIG. 10 includes an activity amount level obtained from the detection value of the motion sensor 113, a heart rate level obtained from the detection value of the pulse sensor 114, the stress level of the user U, and the state of the user U. Correlate the judgment result. The data format of the determination data 325 is arbitrary, and may be a table format as shown in FIG. 10 or other formats. The stress level is a value of stress level data calculated by the detection control unit 313 based on the detection value of at least one of the motion sensor 113, the pulse sensor 114, and the pressure sensor 115 as described above. The heart rate level in the example of FIG. 10 is an index obtained by dividing the heart rate per minute of the user U obtained from the detection value of the pulse sensor 114 into a plurality of stages. In this example, the heart rate level is divided into three stages of “Low”, “Normal”, and “High”. The activity amount level in the example of FIG. 10 is an index obtained by calculating the activity amount of the user U from the detection value of the motion sensor 113 and dividing the calculated activity amount into a plurality of stages. In this example, the activity level is divided into three stages of “Low”, “Normal”, and “High”. The stress level is divided into two stages, “Low” and “High”.

In the example of FIG. 10, the activity state of the user U and the determination result of normal or abnormal are associated based on the heart rate level, the activity amount level, and the stress level.
For example, when the heart rate level of the user U is “Normal”, the activity level is “Low”, and the stress level is “Low”, the state of the user U is normal based on the determination data 325 of FIG. It is determined. In this case, the activity state of the user U is determined to be a resting state. This state corresponds to a state where the user U is relaxed, for example.

  Further, for example, when the heart rate level of the user U is “Normal”, the activity level is “Low”, and the stress level is “High”, the state of the user U is determined based on the determination data 325 in FIG. Is determined to be abnormal (a state to be notified). In this case, the activity state of the user U is determined to be a resting state. This state corresponds to a state in which the user U feels uneasy, for example.

  For the heart rate, a different standard according to the age of the user U may be used. For example, the standard value of the resting heart rate is set as follows: school age (7 to 12 years old), adolescence (13 to 25 years old), adulthood (26 to 40 years old), middle age (41 It may be set separately for old age (65 years old or less) and old age (66 years old or older). Specifically, the reference value of the resting heart rate in schoolchildhood may be set to 70 bpm or more and 90 bpm or less. In this case, the reference value of the resting heart rate in adolescence and adulthood may be set to 60 bpm or more and 80 bpm or less, and the reference value of the resting heart rate in middle age or old age may be set to 60 bpm or more and 70 bpm or less. Then, when the heart rate of the user U detected by the terminal device 10 is within the range of the resting heart rate reference value, the heart rate level may be classified as “Normal”. In this case, if the user's U heart rate is lower than the resting heart rate reference value range, it is classified as “Low”, and if the user U heart rate is higher than the resting heart rate reference value range, it is classified as “High”. You may classify.

  The detection value detected by the motion sensor 113 or the activity amount level obtained from this detection value corresponds to the activity amount information. Further, the detection value detected by the pulse sensor 114 or the heart rate level obtained from the detection value of the pulse sensor 114 corresponds to physical information. When the pulse sensor 114 detects blood oxygen saturation, this detected value corresponds to physical information.

FIG. 11 is a flowchart showing the operation of the HMD 20.
The operation illustrated in FIG. 10 is performed by the HMD 20 that receives data from the terminal device 10, and is performed by the HMD 20B worn by the administrator UM in this embodiment. However, in the communication system 1, the HMDs 20A and 20C perform the operation of FIG. 11 by appropriately setting the communication setting data 323 included in the terminal devices 10A, 10B, and 10C and the communication setting data 533 included in the HMDs 20A, 20B, and 20C. It can also be executed. Here, the operation of the HMD 20B will be described.

  When the power of the HMD 20B is turned on, the control unit 51 starts operating (step S31). The control unit 51 determines whether data indicating a determination result has been received from the terminal device 10 (step S32). While data is not received (step S32; No), the control unit 51 waits until data is received. When data is received from the terminal device 10 (step S32; Yes), the control unit 51 specifies the terminal device 10 that is the transmission source (step S33). For example, the control unit 51 specifies whether the transmission source of the data received in step S32 is the terminal device 10A, the terminal device 10B, or the terminal device 10C.

  The control unit 51 refers to the determination result included in the data received in step S32 (step S34), and determines whether or not the determination result is a result indicating a state to be notified (step S35). When the determination result indicates that the state is to be notified (step S35; Yes), the control unit 51 performs notification (step S36). In step S <b> 36, the control unit 51 performs display for notification by the right display unit 22 and the left display unit 24.

  When the determination result indicates that the notification is not in a state (step S35; No), and after executing the notification in step S36, the control unit 51 performs the determination included in the data received in step S32. Data or the like is displayed or the display is updated (step S37).

  The control unit 51 determines whether or not an end condition for ending the operation is satisfied (step S38). If the end condition is not satisfied (step S38; No), the control unit 51 returns to step S32. The termination condition includes accepting an operation termination instruction input by the input unit 213, power-off operation by a power switch (not shown), insufficient battery 249, and the like. When the end condition is satisfied (step S38; Yes), the control unit 51 ends the process.

In step S16 in FIG. 9 and step S36 in FIG. 11, the terminal device 10 and the HMD 20 perform notification when it is determined that the state of the user U is “abnormal” (a state to be notified). In the present embodiment, a configuration in which a plurality of types of notifications are performed may be employed.
In the example of FIG. 10, the determination result of “emergency” is set as the determination result of the state of the user U in addition to “normal” and “abnormal”. “Emergency” is a state in which determination is made regardless of the activity level and the stress level when the heart rate level of the user U is “Low” or less. The detection control unit 313 can determine “emergency” based on the determination data 325 in step S15.
In this case, the terminal device 10 may perform notification corresponding to the determination result of “emergency” in step S16. This notification may be a notification operation common to the case where the determination result is “abnormal”, or may be a notification operation different from the case of “abnormal”. For example, the vibration pattern for notification of the vibrator 15 may be set to a different pattern depending on whether the determination result is “abnormal” or “emergency”. Further, as the notification image displayed by the display control unit 317, different images may be included in the setting data 321 when the determination result is “abnormal” and “emergency”.

  Further, the HMD 20 may perform notification corresponding to the determination result of “emergency” in step S36. This notification may be a notification operation common to the case where the determination result is “abnormal”, or may be a notification operation different from the case of “abnormal”. For example, when the notification image is displayed by the right display unit 22 and the left display unit 24, the notification image may be different in the case of “abnormal” and the case of “emergency”. Further, the display color and display brightness of the notification image may be different between “abnormal” and “emergency”. Specifically, in the case of “emergency”, a notification image with higher luminance may be displayed, or may be displayed in a color that is more likely to alert the user UM.

FIG. 12 is a sequence diagram showing the operation of the communication system 1. Specifically, data transmission / reception and operation of each device between the terminal devices 10A, 10B, and 10C and the HMDs 20A and 20B of the communication system 1 are shown.
FIG. 12 illustrates an example of an operation when the terminal devices 10A, 10B, and 10C determine that the state of the user U is “normal”.

  The terminal device 10A determines the state of the user U1 based on the determination data (step t1), and transmits data including the determination result to the HMD 20A worn by the user U1 (step t2). The HMD 20A receives the data transmitted by the terminal device 10A, and updates the display based on the received data (step t4). The HMD 20A displays the activity level obtained from the detection value of the motion sensor 113, the heart rate level obtained from the detection value of the pulse sensor 114, the stress level, and the like by the right display unit 22 and the left display unit 24. In the example of FIG. 12, since the state of the user U1 is determined to be normal, no notification is performed.

  Also, the terminal device 10A transmits data including the determination result to the HMD 20B (step t3). In the communication system 1, the HMD 20A worn by the administrator UM displays information related to the states of all users U1 and U2 other than the administrator UM, and the administrator UM can manage the state of each user. The HMD 20B also has a function of displaying information related to the state of the administrator UM itself.

  The HMD 20B receives the data transmitted by the terminal device 10A, and updates the display based on the received data (step t5). The HMD 20B displays the activity level obtained from the detection value of the motion sensor 113, the heart rate level obtained from the detection value of the pulse sensor 114, the stress level, and the like by the right display unit 22 and the left display unit 24 (step t5). . Here, the HMD 20B indicates the terminal device 10 (here, the terminal device 10A) that has transmitted data, such as the activity level, heart rate level, and stress level, or the user U (here, the user U1) wearing the terminal device 10. Display in association with. In the example of FIG. 12, since the state of the user U1 is determined to be normal, no notification is performed.

  Similarly, the terminal device 10C worn by the user U2 determines the state of the user U2 based on the determination data (step t9), and transmits data including the determination result to the HMD 20B (step t10). The HMD 20B associates the activity amount level, the heart rate level, the stress level, and the like with the terminal device 10 (here, the terminal device 10C) that transmitted the data or the user U wearing the terminal device 10 (here, the user U2). Are displayed (step t11).

  Further, the terminal device 10B determines the state of the manager UM based on the detection values of the motion sensor 113 and the pulse sensor 114 (step t6). The terminal device 10B transmits data including the determination result to the HMD 20B (step t7). The HMD 20B displays the activity amount level, the heart rate level, the stress level, and the like in association with the terminal device 10B or the user UM, similarly to the case where data is received from the terminal device 10A (step t8).

FIG. 13 is a sequence diagram showing the operation of the communication system 1. Specifically, data transmission / reception and operation of each device between the terminal devices 10A, 10B, and 10C and the HMDs 20A and 20B of the communication system 1 are shown.
FIG. 13 illustrates an example of an operation when the terminal devices 10A, 10B, and 10C determine that the state of the user U is “abnormal”.

The terminal device 10A determines the state of the user U1 based on the determination data (step t21). In FIG. 13, since the state of the user U1 is determined to be abnormal, the terminal device 10A performs notification by using the vibrator 15 and the display 13 (step t22).
The terminal device 10A transmits data including the determination result to the HMD 20A worn by the user U1 (step t23). The HMD 20A receives the data transmitted by the terminal device 10A, and updates the display such as the activity amount level, the heart rate level, and the stress level based on the received data (step t25). Furthermore, since the determination result of the terminal device 10A is a determination result indicating an abnormality, the HMD 20A performs notification.

  Also, the terminal device 10A transmits data including the determination result to the HMD 20B (step t24). The HMD 20B receives the data transmitted by the terminal device 10A, updates the display of activity level, heart rate level, stress level, etc. corresponding to the terminal device 10A or the user U1 based on the received data, and notifies Is executed (step t26).

  Similarly, the terminal device 10C worn by the user U2 determines the state of the user U2 based on the determination data (step t33). In FIG. 13, since the state of the user U2 is determined to be abnormal, the terminal device 10C performs notification using the vibrator 15 and the display 13 (step t34). The terminal device 10C transmits data including the determination result to the HMD 20B (step t35). The HMD 20B receives the data transmitted by the terminal device 10C, updates the display of activity level, heart rate level, stress level, etc. corresponding to the terminal device 10C or the user U2 based on the received data, and notifies Is executed (step t36).

Further, the terminal device 10B determines the state of the manager UM based on the detection values of the motion sensor 113 and the pulse sensor 114 (step t27). In FIG. 13, since the state of the user U2 is determined to be abnormal, the terminal device 10C performs notification using the vibrator 15 and the display 13 (step t28).
The terminal device 10B transmits data including the determination result to the HMD 20B (step t29). The HMD 20B receives the data transmitted by the terminal device 10B, updates the display of activity level, heart rate level, stress level, etc. corresponding to the administrator UM based on the received data, and executes notification. (Step t30).

Furthermore, the terminal device 10B transmits data including the determination result to the HMD 20A (step t31). When it is determined that the state of the administrator UM is abnormal, the administrator UM itself may not be able to cope with this situation. For this reason, when it is determined that the state of the administrator UM is abnormal, the determination result that the state of the administrator UM is abnormal is included for the HMD 20 worn by at least one of the users U1 and U2. Data is sent. This operation may be executed only when it is determined that the state of the administrator UM is abnormal, or may be executed regardless of the determination result of the state of the administrator UM. Further, the transmission destination to which the terminal device 10B transmits data including the determination result is set in advance in the communication setting data 533 that the terminal device 10B has. In this embodiment, the HMD 20A is set as the transmission destination.
The HMD 20A receives the data transmitted by the terminal device 10B, updates the display of the activity level, heart rate level, stress level, etc. of the administrator UM based on the received data, and executes notification (step t32). .

  FIG. 14 is a diagram illustrating a display example in the HMD 20. In particular, a display example in the HMD 20B worn by the administrator UM is shown. In the figure, VR indicates the field of view of the administrator UM. In the field of view VR, the manager UM visually recognizes the outside scene by the outside light OL transmitted through the right light guide plate 26 and the left light guide plate 28. In the drawing, an object OB is shown as a real object existing in the outside scene, that is, in real space. For example, when diving is performed in the sea, the manager UM visually recognizes the state in the sea through the HMD 20B.

  In the example of FIG. 14, the status display unit 270 is arranged in the display area V where the HMD 20B displays an image. The status display unit 270 is an area for displaying various types of information. Here, the status display unit 270 displays information about the statuses of the users U1 and U2 and the administrator UM. The state display unit 270 includes a first state display unit 271, a second state display unit 272, and a third state display unit 273. In the present embodiment, the first state display unit 271 is associated with the user U1, the second state display unit 272 is associated with the user U2, and the third state display unit 273 is associated with the administrator UM. The first state display unit 271 displays the activity level, heart rate level, and stress level of the user U1 based on the determination data transmitted by the terminal device 10A. The second state display unit 272 displays the activity level, heart rate level, and stress level of the user U2 based on the determination data transmitted by the terminal device 10C. The third state display unit 273 displays the activity level, heart rate level, and stress level of the administrator UM based on the determination data transmitted by the terminal device 10B. Further, an emphasis display frame 275 can be additionally displayed on each of the first state display unit 271, the second state display unit 272, and the third state display unit 273. The highlight display frame 275 is a frame that is displayed as a notification operation when the state of the user U is determined to be abnormal, for example, blinking. While the highlight display frame 275 is displayed, the display color of the first state display unit 271, the second state display unit 272, or the third state display unit 273 may be changed. Moreover, the font of the character displayed on the 1st state display part 271, the 2nd state display part 272, or the 3rd state display part 273, and the shape of a symbol may be changed.

As described above, the detection system 100 to which the present invention is applied includes the HMD 20B that is worn on the head of the administrator UM and the terminal device 10 that is worn by the administrator UM or the users U1 and U2 as wearers. The terminal device 10 includes a motion sensor 113 that detects information indicating the amount of activity of the user U, and a pulse sensor 114 that detects information indicating the state of the body. In addition, the terminal device 10 includes a vibrator 15 and / or a display 13 as a notification unit that performs notification. The HMD 20B includes a right display unit 22 and a left display unit 24 that allow an administrator UM to visually recognize information. The detection system 100 performs notification by the display 13 and the vibrator 15 or display of information by the right display unit 22 and the left display unit 24 based on the activity amount level and the heart rate level detected by the terminal device 10. Thereby, based on the information regarding the state of the user U wearing the terminal device 10, the terminal device 10 and the HMD 20 perform notification and display. Thereby, it is possible to promptly provide information to the administrator UM who uses the HMD 20 regarding the activity amount and physical state of the user U. For example, when the activity level or the heart rate level of the users U1 and U2 is in a state that requires attention, this can be promptly notified to the administrator UM.
In the above embodiment, the HMD 20 has been described as causing the user U to visually recognize an image by displaying the image with the right display unit 22 and the left display unit 24. The image displayed by the HMD 20 refers to a screen including at least one of characters, symbols, various graphics, and other images, and the HMD 20 may be anything that allows the user U to visually recognize information in some form. For example, the HMD 20 may display information including only characters so that the user U can visually recognize the information. In addition, the form of information displayed by the HMD 20 is not limited and may be arbitrarily configured.

  The control part 31 of the terminal device 10 determines whether it is a state which alert | reports based on an active mass level and a heart rate level. Based on the determination result, notification by the display 13 and / or 15 is executed. Further, as the notification operation, the HMD 20 can perform a display for notification. Accordingly, it is possible to appropriately determine whether or not the state of the users U1, U2 and / or the administrator UM is a state in which notification is to be performed and perform notification.

  Based on the detection values of the motion sensor 113 and the pulse sensor 114, the control unit 31 may determine whether or not the user U is in a state of performing notification regarding at least one of the physical state and the mental state. In this case, it is possible to appropriately determine the physical state or mental state of the user U wearing the terminal device 10. For example, not only notification and display are performed when a mental and physical disorder of the user U occurs, but notification and display can be performed at a stage where there is a high possibility of malfunction.

  In addition, the terminal device 10 includes a communication unit 111 and a control unit 31 that transmits a determination result to the HMD 20 by the communication unit 111. The HMD 20 includes a communication unit 211 and a control unit 51 that causes the right display unit 22 and the left display unit 24 to display based on the determination result when the communication unit 211 receives the determination result transmitted by the terminal device 10. Thereby, the terminal device 10 determines about the activity amount and body state of the user U, and HMD20 displays based on a determination result. For this reason, the amount of information transmitted from the terminal device 10 to the HMD 20 and the frequency of transmission can be suppressed. In addition, since the processing load of the HMD 20 is small, it is possible to realize a configuration in which the HMD 20 receives and displays a determination result from a plurality of terminal devices 10, for example.

  Since the terminal device 10 includes a heart rate sensor that detects the heart rate of the user U as the body information detection unit, the notification and display reflected on the state of the user U using the body information including the heart rate of the user U. It can be performed. Further, by detecting the heart rate affected by the psychological state of the user U, it is possible to determine or estimate the mental state of the user U, and to perform notification and display more accurately.

  Since the terminal device 10 includes the vibrator 15 that generates vibration as a notification unit, for example, the user device U can perform notification so that the user U can easily notice even in water or in a dark place.

  In addition, since the HMD 20 is configured integrally with goggles worn by the administrator UM, information can be displayed so that the administrator UM can easily recognize the user while wearing the goggles.

  Moreover, the communication system 1 to which the communication system and the control method of the communication system of the present invention are applied includes a plurality of detection systems 100 corresponding to a plurality of users. Each terminal device 10 determines whether or not a motion sensor 113 that detects information indicating the amount of activity of the user U, a pulse sensor 114 that detects information indicating the state of the body of the user U, and a state in which notification is performed. The control part 31 as a determination part is provided. Moreover, the terminal device 10 includes the vibrator 15 and / or the display 13 as a notification unit that performs notification, and the communication unit 111. The control unit 31 causes the notification unit to perform notification based on the determination result, and the determination result. Is transmitted to the HMD 20 designated as the transmission destination. The HMD 20 includes a display unit, a communication unit 211, and a control unit 51 that causes the display unit to display based on the determination result when the determination result transmitted by the terminal device 10 is received. Thereby, it is possible to notify each user himself / herself about the amount of activity and the physical state of the users U1 and U2 and the administrator UM who wear the terminal device 10, and information can be displayed by the HMD 20B. Therefore, the administrator UM can know the states of the users U1, U2 and the administrator UM. In addition, when the states of the users U1 and U2 and the administrator UM are in a state that requires attention, the administrator UM can quickly know this. In this configuration, the processing load of the HMD 20 is small. For this reason, for example, even when the HMD 20B receives the determination results from the plurality of terminal devices 10A, 10B, and 10C, the display can be performed promptly.

[Second Embodiment]
FIG. 15 is a flowchart showing the operation of the terminal device 10 in the second embodiment. FIG. 16 is a flowchart showing the operation of the HMD 20 in the second embodiment.
In the second embodiment, the HMD 20 determines the state of the user U. In this configuration, the terminal device 10 detects the state of the user U by the motion sensor 113 and the pulse sensor 114, and transmits the detection values of the motion sensor 113 and the pulse sensor 114 or determination data obtained from the detection values to the HMD 20. To do. In the second embodiment, the state of the users U1 and U2 of the communication system 1 and the administrator UM is managed by the HMD 20B worn by the administrator UM, so that each of the terminal devices 10A, 10B, and 10C is detected by the HMD 20B. Send the value or judgment data.

  In the second embodiment, although not shown, the control unit 51 has a functional block that performs determination based on detection values of the motion sensor 113 and the pulse sensor 114 or determination data obtained from the detection values. This functional block corresponds to the function of the detection control unit 313 (FIG. 4) described in the first embodiment. For example, the communication control unit 513 or the display control unit 515 of the control unit 51 may also serve as the functional block. The storage unit 53 stores determination data corresponding to the determination data 325.

When the terminal device 10 is powered on, the control unit 31 starts operating (step S51). The control unit 31 acquires detection values of the motion sensor 113, the pulse sensor 114, and the pressure sensor 115 (step S52).
The control unit 31 transmits the acquired detection value of each sensor or data including determination data obtained from the acquired detection value to the HMD 20B by the communication unit 111 (step S53).

  As shown in FIG. 16, when the power of the HMD 20B is turned on, the control unit 51 starts operating (step S71). The control unit 51 determines whether data has been received from the terminal device 10 (step S72). While data is not received (step S72; No), the control unit 51 stands by until data is received. When data is received from the terminal device 10 (step S72; Yes), the control unit 51 specifies the terminal device 10 that is the transmission source (step S73). For example, the control unit 51 specifies whether the transmission source of the data received in step S72 is the terminal device 10A, the terminal device 10B, or the terminal device 10C.

  The control unit 51 refers to the determination data stored in the storage unit 53 and makes a determination based on the detection value or the determination data included in the data received in step S72 (step S74). The control unit 51 determines whether or not the state indicated by the detection value or determination data received in step S72 is a state to be notified (step S75).

  When it determines with falling into the state which should alert | report (step S75; Yes), the control part 51 performs alerting | reporting (step S76). In step S <b> 76, the control unit 51 displays an image for notification by the right display unit 22 and the left display unit 24.

When the determination result indicates that the notification is not in a state (step S75; No), and after executing the notification in step S76, the control unit 51 performs the determination included in the data received in step S72. Data or the like is displayed or the display is updated (step S77).
The control unit 51 transmits data including the determination result to the terminal device 10 that is the transmission source specified in step S73 (step S78).
When data including a detection value is received in step S72, the control unit 51 may generate determination data obtained from the detection value, and transmit data including the determination data and the determination result. In step S72, the control unit 51 may transmit data to the transmission destination device set in the communication setting data 323 in addition to the transmission source terminal device 10 specified in step S73.

  The control unit 51 determines whether or not an end condition for ending the operation is satisfied (step S79). If the end condition is not satisfied (step S79; No), the control unit 51 returns to step S72. The termination condition is the same as the condition described in the first embodiment. When the end condition is satisfied (step S79; Yes), the control unit 51 ends the process.

  Returning to FIG. 15, the terminal device 10 receives data from the HMD 20B (step S54), refers to the determination result included in the received data, and determines whether the determination result is a result indicating a state to be notified. (Step S55). When the determination result indicates that the notification is in a state (step S55; Yes), the control unit 31 performs notification (step S56). In step S <b> 56, the control unit 31 performs notification display by using the vibrator 15 and / or the display 13.

  When the determination result indicates that the notification is not in a state (step S55; No), and after executing the notification in step S56, the control unit 31 determines whether or not an end condition for ending the operation is satisfied. Is determined (step S57). The termination condition is the same as the condition described in the first embodiment. If the end condition is not satisfied (step S57; No), the control unit 31 returns to step S52. When the end condition is satisfied (step S57; Yes), the control unit 31 ends the process.

  FIG. 17 is a sequence diagram showing the operation of the communication system 1. Specifically, data transmission / reception and operation of each device between the terminal devices 10A, 10B, and 10C and the HMDs 20A and 20B of the communication system 1 are shown. FIG. 17 shows an example of an operation when the state of the user U is determined to be “normal”.

The terminal device 10A acquires the detection value of the sensor (step t51), and transmits data including the acquired detection value to the HMD 20B (step t52).
The HMD 20B receives the data transmitted by the terminal device 10A and makes a determination based on the received data (step t53). The HMD 20B updates the display based on the data received from the terminal device 10A and the determination result (step t54).
In addition, the HMD 20B transmits data including the determination result to the terminal device 10A (step t55). Further, the HMD 20B transmits data including the determination result to the HMD 20A belonging to the same detection system 100 as the terminal device 10A (step t56). The HMD 20B specifies the terminal device 10A that transmitted the data in step t52, and sets the HMD 20A associated with the terminal device 10A as the transmission destination. The HMD 20A updates the display based on the determination result included in the data transmitted by the HMD 20B and the determination data (step t57). In the example of FIG. 17, since the state of the user U1 is determined to be normal, no notification is performed.

  Similarly, the terminal device 10C worn by the user U2 acquires the detection value of the sensor (step t58), and transmits data including the acquired detection value to the HMD 20B (step t59). The HMD 20B receives the data transmitted by the terminal device 10C and makes a determination based on the received data (step t60). The HMD 20B updates the display based on the data received from the terminal device 10A and the determination result (step t61).

Further, the terminal device 10B worn by the administrator UM acquires the detection value of the sensor (step t62), and transmits data including the acquired detection value to the HMD 20B (step t63). The HMD 20B receives the data transmitted by the terminal device 10B and makes a determination based on the received data (step t64). The HMD 20B updates the display based on the data received from the terminal device 10B and the determination result (step t65).
In addition, the HMD 20B transmits data including the determination result to the terminal device 10B (step t66).

FIG. 18 is a sequence diagram showing the operation of the communication system 1. Specifically, data transmission / reception and operation of each device between the terminal devices 10A, 10B, and 10C and the HMDs 20A and 20B of the communication system 1 are shown.
FIG. 18 shows an example of an operation when the state of the user U is determined to be “abnormal”.

The terminal device 10A acquires the detection value of the sensor (step t101), and transmits data including the acquired detection value to the HMD 20B (step t102).
The HMD 20B receives the data transmitted by the terminal device 10A and makes a determination based on the received data (step t103). The HMD 20B updates the display based on the data received from the terminal device 10A and the determination result, and executes notification (step t104).
The HMD 20B transmits data including the determination result to the terminal device 10A (step t105). The terminal device 10A receives the data transmitted by the HMD 20B, and performs notification by the vibrator 15 and the display 13 according to the determination result included in the received data (step t106).
Further, the HMD 20B transmits data including the determination result to the HMD 20A belonging to the same detection system 100 as the terminal device 10A (step t107). The HMD 20B specifies the terminal device 10A that transmitted the data in step t52, and sets the HMD 20A associated with the terminal device 10A as the transmission destination. The HMD 20A updates the display based on the determination result included in the data transmitted by the HMD 20B and the determination data, and performs notification (step t108).

Similarly, the terminal device 10C worn by the user U2 acquires the detection value of the sensor (step t109), and transmits data including the acquired detection value to the HMD 20B (step t110).
The HMD 20B receives the data transmitted by the terminal device 10C and makes a determination based on the received data (step t111). The HMD 20B updates the display based on the data received from the terminal device 10C and the determination result, and performs notification (step t112). The HMD 20B transmits data including the determination result to the terminal device 10C (step t113). The terminal device 10A receives the data transmitted by the HMD 20B, and performs notification by the vibrator 15 and the display 13 according to the determination result included in the received data (step t114).

  Further, the terminal device 10B worn by the administrator UM acquires the detection value of the sensor (step t115), and transmits data including the acquired detection value to the HMD 20B (step t116). The HMD 20B receives the data transmitted by the terminal device 10B and makes a determination based on the received data (step t117). The HMD 20B updates the display based on the data received from the terminal device 10A and the determination result, and executes notification (step t118).

  Here, the HMD 20B transmits the data including the determination result to the HMD 20A that is the transmission destination set in the communication setting data 533 in order to notify the other user U of the abnormality of the administrator UM (Step t119). The HMD 20A receives the data transmitted by the HMD 20B, performs notification according to the determination result included in the received data, and updates the display based on the received data (step t120).

  Further, the HMD 20B transmits data including the determination result to the terminal device 10B (step t121). The terminal device 10A receives the data transmitted by the HMD 20B, and performs notification by the vibrator 15 and the display 13 according to the determination result included in the received data (step t122).

In 1st Embodiment mentioned above, terminal device 10A, 10B, 10C demonstrated the example which each obtains the data for a determination from the detection value of the motion sensor 113 and the pulse sensor 114, and determines the state of the user U. On the other hand, as described in the second embodiment, the terminal devices 10A, 10B, and 10C can transmit sensor detection values or determination data to the HMD 20B, and the HMD 20B can perform the determination.
That is, the terminal device 10 according to the second embodiment transmits information detected by the motion sensor 113 and the pulse sensor 114 via the communication unit 111. The HMD 20 makes a determination based on information transmitted by the terminal device 10 and transmits a determination result to the terminal device 10. The terminal device 10 performs notification when the determination result transmitted by the HMD 20 is received. According to this configuration, since the processing load on the terminal device 10 is small, the terminal device 10 can be easily downsized.

[Third Embodiment]
FIG. 19 is a side view of the main part showing the configuration of the HMD 6 of the third embodiment.
The HMD 6 is configured separately from the mask 4 worn by the user U. The mask 4 includes a transparent resin or glass shield 41 that secures a field of view as an underwater mask, and a skirt 42 that is in close contact with the face of the user U and forms a watertight space inside the shield 41. The mask 4 can be said to be a so-called general underwater mask. The mask 4 has a band 43 indicating the shield 41 and the skirt 42. The band 43 is a belt-like member having elasticity, and restrains the head of the user U to hold the shield 41 and the skirt 42 on the face of the user U.

  A hinge 45 is attached to the band 43, and the HMD 6 is connected to the mask 4 via the hinge 45. As shown by an arrow A in the drawing, the HMD 6 can be turned up and down with the hinge 45 as a fulcrum.

FIG. 20 is a perspective view showing the configuration of the HMD 6. In FIG. 20, the illustration of the mask 4 is omitted.
The HMD 6 includes a right holding portion 21A and a left holding portion 23A that are coupled to the hinge 45. The HMD 6 has a glasses-like shape as a whole, and is supported by the hinge 45 in the right holding portion 21A and the left holding portion 23A corresponding to the temples of the glasses.

A right light guide plate 26 </ b> A and a left light guide plate 28 </ b> A are disposed on the front surface of the HMD 6. The right light guide plate 26A and the left light guide plate 28A are supported by the frame 2A, the right light guide plate 26A is located in front of the right eye of the user U, and the left light guide plate 28A is located in front of the left eye of the user U. The HMD 6 includes a right display unit 22A that supplies image light to the right light guide plate 26A and a left display unit 24A that supplies image light to the left light guide plate 28A.
The optical configuration and function of the right light guide plate 26A are the same as those of the right light guide plate 26 (FIG. 7), and the optical configuration and function of the left light guide plate 28A are the same as those of the left light guide plate 28. The right display unit 22A accommodates the right display unit 22, and the left display unit 24A accommodates the left display unit 24. Moreover, each part shown in FIG. 6 can be accommodated in either the right display unit 22A or the left display unit 24A.

  In the third embodiment, an HMD 6 is used instead of the HMD 20 of the first and second embodiments described above. That is, the HMD 6 corresponds to the display device of the present invention. Similar to the HMD 20, the HMD 6 can communicate with the terminal devices 10A, 10B, 10C and the HMD 20A by the communication unit 211. The HMD 6 includes a main processor 201 and can execute notification, display control, and determination of the state of the user U by the function of the control unit 51 realized by the main processor 201 executing a program. . Therefore, in the detection system 100 using the HMD 6 instead of the HMD 20 and the communication system 1 having the detection system 100, the operations shown in FIGS. 11, 12, 13, 16, 17, and 18 can be executed. .

  In this configuration, the HMD 6 is mounted so as to be movable outside the position where the user U (administrator UM) overlaps with the mask 4 (goggles) worn by the user U (administrator UM) and the range where the user U (administrator UM) visually recognizes. The For example, as shown in FIG. 19, if the HMD 6 is rotated upward, the HMD 6 can be retracted outside the field of view where the user U visually recognizes the outside scene through the mask 4. For this reason, in the environment where the mask 4 is mounted, it is possible to realize a state in which the user U can easily view the information displayed by the HMD 6 and a state in which the HMD 6 does not obstruct the user U's visual field.

[Fourth Embodiment]
FIG. 21 is an external view of a main part of the detection device 7 according to the fourth embodiment.
The detection device 7 is a device that also functions as a regulator that is a submersible device. The detection device 7 includes a regulator 71 having a built-in pressure adjustment mechanism, a hose 72 that connects the regulator 71 and an air tank (not shown), and a mouthpiece 73 that the user U holds in the mouth. The occlusal portion 75 that supports the mouthpiece 73 is in a position that abuts on the front teeth of the user U, and incorporates a sensor that measures the pressure with which the user U bites the mouthpiece 73, that is, the occlusal pressure, as will be described later.

FIG. 22 is a block diagram of the detection device 7.
The detection device 7 shown in FIG. 22 includes a microcomputer 701 that executes a program and controls the detection device 7. A memory 703 and a non-volatile storage unit 702 are connected to the microcomputer 701. The nonvolatile storage unit 702 is configured by a semiconductor storage device such as an EEPROM, a flash memory, and eMMC, or other nonvolatile storage devices. The nonvolatile storage unit 702 stores a program executed by the microcomputer 701 and various data processed by the microcomputer 701 executing the program. The memory 703 constitutes a work area that temporarily stores a program to be executed and data to be processed when the microcomputer 701 executes the program. Note that the microcomputer 701 may incorporate the nonvolatile storage unit 702 and the memory 703.

  A communication unit 704 is connected to the microcomputer 701. The communication unit 704 performs wireless communication with an external device. The communication unit 704 according to the present embodiment has a configuration capable of executing wireless communication with a communication method suitable for underwater wireless communication. For example, the communication unit 704 can be configured to execute wireless communication using sound waves or ultrasonic waves. In this case, the communication unit 704 includes a sound output unit, a microphone, an encode / decode unit, and the like (not shown). Alternatively, the communication unit 704 can be configured to execute wireless communication using visible light or infrared light. In this case, the communication unit 704 includes a light emitting unit, a light receiving unit, an encoding / decoding unit, and the like. Further, the communication unit 704 may execute wireless communication using radio waves. In this case, the communication unit 704 includes an RF unit, a baseband unit, an encoding / decoding unit, and the like (not shown). The communication unit 704 transmits and receives a radio signal having a frequency suitable for wireless communication in water, for example. The communication unit 704 has a configuration for performing wireless communication according to standards such as Bluetooth (registered trademark) and wireless LAN (including Wi-Fi (registered trademark)) as a communication method suitable for wireless communication in the atmosphere. Also good. For example, the communication unit 704 may be configured to be able to switch between a communication method suitable for underwater wireless communication and a communication method suitable for wireless communication in the atmosphere.

  In the present embodiment, the detection device 7 communicates with the terminal device 10 worn by the user U. More specifically, the detection device 7 used by the user U1 communicates with the terminal device 10A worn by the user U1, and the detection device 7 used by the user U2 communicates with the terminal device 10C worn by the user U2. . Further, the detection device 7 used by the administrator UM communicates with the terminal device 10B.

A pressure sensor 705 and a respiration rate sensor 706 are connected to the microcomputer 701 as sensors. The pressure sensor 705 is a sensor that detects pressure applied to the occlusal portion 75, and detects occlusal pressure at which the user U bites the occlusal portion 75. The respiration rate sensor 706 detects the respiration of the user U who uses the detection device 7 by detecting the flow rate and / or flow rate of the gas flowing through the regulator 71. The microcomputer 701 can obtain the respiration rate per unit time based on the detection value of the respiration rate sensor 706.
Specifically, the pressure sensor 705 can be configured by an electroactive polymer, a piezoelectric element, a MEMS (Micro Electro Mechanical Systems) pressure sensor, or the like, and other configurations may be used.

  The detection device 7 includes a power supply unit 710 and operates with power supplied from the power supply unit 710. The power supply unit 710 includes a rechargeable battery 711, and a power supply control circuit 712 that detects the remaining capacity of the battery 711 and controls charging of the battery 711.

  In the communication system 1 described in the first, second, and third embodiments, when the user U wearing the terminal device 10 uses the detection device 7, the terminal device 10 is the user U detected by the detection device 7. Occlusal pressure and respiratory rate can be acquired. In the first embodiment, the terminal devices 10A, 10B, and 10C can use the occlusal pressure and / or the respiration rate in the process of calculating the stress intensity of the user U and generating the stress level data. Further, the occlusal pressure and / or the respiration rate are more accurately calculated by combining with the amount of activity detected by the motion sensor 113, the heart rate obtained from the detection value of the pulse sensor 114, or the blood oxygen saturation. The strength of stress can be obtained. In 2nd Embodiment, each of terminal device 10A, 10B, 10C may transmit the detected value of the pressure sensor 705 and the respiration rate sensor 706 to HMD20B. In this case, the detection values of the pressure sensor 705 and the respiration rate sensor 706 may be transmitted together with the detection values of the motion sensor 113 and the pulse sensor 114 or the stress level data obtained from these detection values. Further, data obtained from detection values of the pressure sensor 705 and the respiration rate sensor 706 may be transmitted to the HMD 20B.

In addition, in the detection system 100, other sensors that detect the states of the users U1 and U2 and the administrator UM can be used together with the terminal device 10.
The detection value detected by the pressure sensor 705 or the occlusal pressure data obtained from the detection value corresponds to the activity amount information. The detection value detected by the respiration rate sensor 706 or the respiration rate data obtained from the detection value of the respiration rate sensor 706 can be considered to correspond to physical information, and can also be considered to correspond to activity amount information.

In addition, this invention is not restricted to the structure of the said embodiment, In the range which does not deviate from the summary, it can be implemented in a various aspect.
For example, the function of the terminal device 10 can be realized as a device different from the diving computer. Moreover, the detection system 100 is not limited to a configuration including one terminal device 10 and one HMD 6, 20, and may be configured to include a plurality of terminal devices 10 and one HMD 6, 20. Each of the above embodiments is not limited to the case where one of the plurality of users U is the administrator UM, but can also be applied to a configuration in which all the users U transmit and receive information mutually. For example, the terminal devices 10A, 10B, and 10C may transmit / receive data to / from all of the HMDs 20A, 20B, and 20C.

For example, instead of the HMDs 6 and 20, other types of image display units such as an image display unit worn by the user U, such as a hat or a helmet, may be employed. The display device of the present invention may include a display unit that displays an image corresponding to the left eye of the user U and a display unit that displays an image corresponding to the right eye of the user U.
In addition, the display device of the present invention is not limited to a mask used underwater or a configuration attached with the mask. For example, it may be configured as glasses used on land, goggles, sunglasses, or the like, or may be configured together with these. Further, for example, it may be configured as a head-mounted display built in a body protective device such as a helmet. In this case, the part for positioning the position with respect to the user's body and the part positioned with respect to the part can be used as the mounting portion.

  In addition, as an optical system that guides image light to the user's eyes, a configuration in which virtual images are formed by half mirrors 261 and 281 on part of the right light guide plates 26 and 26A and the left light guide plates 28 and 28A is illustrated. The present invention is not limited to this, and an image may be displayed on a display area having an area that occupies the entire surface or most of the right light guide plates 26 and 26A and the left light guide plates 28 and 28A. In this case, a process of reducing the image may be included in the operation of changing the display position of the image. Furthermore, the optical element of the present invention is not limited to the right light guide plates 26 and 26A and the left light guide plates 28 and 28A having the half mirrors 261 and 281, and may be any optical component that allows image light to enter the user's eyes. Specifically, a diffraction grating, a prism, or a holographic display unit may be used.

  In addition, at least a part of each functional block shown in FIGS. 3, 6, 22, etc. may be realized by hardware, or may be realized by cooperation of hardware and software. The present invention is not limited to a configuration in which independent hardware resources are arranged as shown in the figure. The programs executed by the main processors 101 and 201 and the microcomputer 701 may be stored in a nonvolatile storage unit or another storage device (not shown). Alternatively, the terminal device 10, the HMD 6, 20, the detection device 7 and the like may be configured to acquire and execute a program stored in an external device or a recording medium through a communication line using an external interface or the like.

  DESCRIPTION OF SYMBOLS 1 ... Communication system, 4 ... Mask (goggles), 6 ... HMD (display apparatus), 7 ... Detection apparatus, 10, 10A, 10B, 10C ... Terminal apparatus, 13 ... Display (notification part), 15 ... Vibrator (notification part) ), 20, 20A, 20B, 20C ... HMD (display device), 22, 22A ... right display unit (display unit), 24, 24A ... left display unit (display unit), 26, 26A ... right light guide plate, 28, 28A ... Left light guide plate, 31 ... Control unit (terminal control unit, determination unit), 32 ... Storage unit, 51 ... Control unit (display device control unit, determination unit), 53 ... Storage unit, 75 ... Occlusion unit, 100, 100A, 100B, 100C ... detection system, 101 ... main processor, 111 ... communication unit (first communication unit), 113 ... motion sensor (activity amount detection unit), 114 ... pulse sensor (body information detection unit), DESCRIPTION OF SYMBOLS 15 ... Pressure sensor (activity amount detection part), 131 ... User state display part, 201 ... Main processor, 211 ... Communication part (2nd communication part), 251 ... Right optical system, 252 ... Left optical system, 261 ... Half mirror 270 ... Status display section, 271 ... First status display section, 272 ... Second status display section, 273 ... Third status display section, 275 ... Highlight display frame, 281 ... Half mirror, 311 ... Operating system, 313 ... Detection Control unit, 315 ... communication control unit, 321 ... setting data, 323 ... communication setting data, 325 ... determination data, 511 ... operating system, 513 ... communication control unit, 515 ... communication control unit, 531 ... setting data, 533 ... communication Setting data, 701 ... microcomputer, 704 ... communication unit, 705 ... pressure sensor (activity amount detection unit, pressure sensor), 706 ...吸数 sensor (activity amount detector, body information detection unit).

Claims (15)

A display device worn on the head of the first user, and a terminal device worn by the first user or the second user as a wearer,
The terminal device
An activity amount detection unit for detecting activity amount information indicating the activity amount of the wearer;
A body information detection unit for detecting body information indicating a state of the wearer's body;
An informing unit for informing,
The display device includes a display unit that allows the first user to visually recognize information,
Based on the activity amount information and the physical information, at least one of notification by the notification unit and display by the display unit is performed.
A detection system characterized by that.   The determination part which determines whether it is in the state to perform alerting based on the activity amount information and the physical information, and alerting by the alerting part based on the judgment result of the judgment part, The detection system according to 1.   The said determination part determines whether it is a state which performs alerting | reporting about at least any one of the said wearer's physical condition and a mental state based on the said activity amount information and the said physical information. The described detection system. The terminal device
A first communication unit;
The determination unit;
A terminal control unit that transmits the determination result of the determination unit to the display device by the first communication unit,
The display device
A second communication unit;
The display apparatus control part which displays on the said display part based on the said determination result, when the determination result which the said terminal device transmits is received by the said 2nd communication part, The display apparatus control part is provided. Or the detection system of 3. The terminal device
A first communication unit;
A terminal control unit that transmits the activity amount information detected by the activity amount detection unit and the body information detected by the body information detection unit by a first communication unit;
The display device
A second communication unit;
The determination unit that performs determination based on the activity amount information and the physical information when the activity amount information and the physical information transmitted by the terminal device are received by the second communication unit;
A display device control unit that transmits the determination result of the determination unit to the terminal device by the second communication unit;
The said terminal control part of the said terminal device alert | reports by the said alerting | reporting part, when the determination result which the said display apparatus transmits is received by the said 1st communication part. Detection system.   The detection system according to claim 1, wherein the body information detection unit includes a heart rate sensor that detects a heart rate of the wearer.   The detection system according to claim 1, wherein the physical information detection unit includes a respiration rate sensor that detects a respiration rate of the wearer.   The detection system according to claim 1, wherein the body information detection unit is a pressure sensor that detects an occlusal pressure of the wearer.   The detection system according to claim 1, wherein the notification unit includes a vibrator that generates vibration.   The detection system according to claim 1, wherein the display device is configured integrally with goggles worn by the first user.   The display device is mounted so as to be movable outside a position overlapping the goggles worn by the first user and a range visually recognized by the first user through goggles. The detection system according to any one of 1 to 10. A communication system comprising a plurality of display systems corresponding to a plurality of the users, a display device mounted on a user's head, and a terminal device worn by the user,
Each of the terminal devices
An activity amount detection unit for detecting activity amount information indicating the activity amount of the user;
A body information detecting unit for detecting body information indicating a state of the user's body;
Based on the activity amount information and the physical information, a determination unit that determines whether or not notification is in a state,
An informing unit for informing;
A first communication unit;
Based on the determination result of the determination unit, the notification unit performs notification, and the first communication unit displays the determination result of the determination unit as the transmission destination among the plurality of display devices. A terminal control unit for transmitting,
The display device
A display unit for visually recognizing information to the user;
A second communication unit;
A display device control unit that causes the display unit to display based on the determination result when the determination result transmitted by the terminal device is received by the second communication unit;
A communication system characterized by the above. A communication system comprising a plurality of display systems corresponding to a plurality of the users, a display device mounted on a user's head, and a terminal device worn by the user,
Each of the terminal devices
An activity amount detection unit for detecting activity amount information indicating the activity amount of the user;
A body information detecting unit for detecting body information indicating a state of the user's body;
An informing unit for informing;
A first communication unit;
The display device in which the activity amount information detected by the activity amount detection unit and the physical information detected by the body information detection unit are designated as a transmission destination among the plurality of display devices by the first communication unit. A terminal control unit for transmitting to
At least one of the display devices is
A display unit for visually recognizing information to the user;
A second communication unit;
A determination unit that determines whether or not a notification is made based on the activity amount information and the physical information when the activity amount information and the physical information transmitted by the terminal device are received by the second communication unit; ,
A display device control unit that displays on the display unit based on the determination result of the determination unit, and transmits the determination result of the determination unit to the terminal device by the second communication unit;
The terminal control unit of the terminal device performs notification by the notification unit when the first communication unit receives a determination result transmitted by the display device.
A communication system characterized by the above. A control method of a communication system comprising a plurality of detection systems corresponding to a plurality of the users, a display device worn on a user's head and a terminal device worn by the user,
By each said terminal device,
Detecting activity amount information indicating the activity amount of the user;
Detecting physical information indicating the state of the user's body;
Based on the activity amount information and the physical information, it is determined whether or not it is in a state of performing notification,
Perform notification based on the determination result,
The determination result is transmitted to the display device designated as a transmission destination among the plurality of display devices,
By the display device,
When the determination result transmitted by the terminal device is received, display is performed based on the determination result.
A control method of a communication system characterized by the above. A control method of a communication system comprising a plurality of detection systems corresponding to a plurality of the users, a display device worn on a user's head and a terminal device worn by the user,
By each said terminal device,
Detecting activity amount information indicating the activity amount of the user;
Detecting physical information indicating the state of the user's body;
The detected activity amount information and the physical information are transmitted to the display device designated as a transmission destination among the plurality of display devices,
At least one of the display devices is
When receiving the activity amount information and the physical information transmitted by the terminal device, based on the activity amount information and the physical information, determine whether or not it is in a state of performing notification,
Display based on the judgment result,
Transmitting the determination result to the terminal device;
When the determination result transmitted by the display device is received by the terminal device, notification is performed.
A control method of a communication system characterized by the above.

Images