WO2022059784A1

WO2022059784A1 - Information provision device, information provision method, and program

Info

Publication number: WO2022059784A1
Application number: PCT/JP2021/034398
Authority: WO
Inventors: 隆幸菅原; 早人中尾; 規高田; 秀生鶴; 哲也諏訪; 翔平大段
Original assignee: 株式会社Ｊｖｃケンウッド
Priority date: 2020-09-18
Filing date: 2021-09-17
Publication date: 2022-03-24
Also published as: US20230200711A1

Abstract

The present invention provides information to a user appropriately. An information provision device (10) provides information to a user and comprises: an output unit (26) including a display unit (26A) that outputs visual stimulation, a sound output unit (26B) that outputs auditory stimulation, and a sensory stimulation output unit (26C) that outputs sensory stimulation different from the visual or auditory stimulation; an environment sensor (20) that detects environment information about the environment surrounding the information provision device (10); and an output selection unit (48) that selects any of the display unit (26A), the sound output unit (26B), or the sensory stimulation output unit (26C) on the basis of the environment information.

Description

Information providing equipment, information providing method and program

The present invention relates to an information providing device, an information providing method, and a program.

In recent years, information devices have undergone major evolution with the evolution of high-speed CPUs, high-definition screen display technologies, compact and lightweight battery technologies, the spread of wireless network environments, and widening the bandwidth. As such information devices, not only smartphones, which are typical examples thereof, but also so-called wearable devices worn by users have become widespread. For example, Patent Document 1 describes a device that gives a user the feeling that a virtual object actually exists by presenting a plurality of sensory information to the user. Further, Patent Document 2 describes that the provision form and the provision timing of the information providing device are determined so that the sum of the evaluation functions representing the appropriateness of the information provision timing is maximized.

Japanese Unexamined Patent Publication No. 2011-96171 Japanese Unexamined Patent Publication No. 2011-242219

Here, in the information providing device that provides information to the user, it is required to appropriately provide the information to the user.

In view of the above problems, the present embodiment aims to provide an information providing device, an information providing method, and a program capable of appropriately providing information to a user.

The information providing device according to one aspect of the present embodiment is an information providing device that provides information to the user, and includes a display unit that outputs a visual stimulus, a voice output unit that outputs an auditory stimulus, and visual and auditory senses. Is an output unit including a sensory stimulus output unit that outputs different sensory stimuli, an environment sensor that detects environmental information around the information providing device, the display unit, the voice output unit, and the voice output unit based on the environmental information. An output selection unit for selecting any of the sensory stimulation output units is included.

The information providing method according to one aspect of the present embodiment is an information providing method for providing information to a user, and is a step of detecting surrounding environmental information and a display unit that outputs a visual stimulus based on the environmental information. , A step of selecting one of a voice output unit that outputs an auditory stimulus and a sensory stimulus output unit that outputs a sensory stimulus different from the visual and auditory senses.

The program according to one aspect of the present embodiment includes a step of detecting surrounding environmental information, a display unit that outputs a visual stimulus, a voice output unit that outputs an auditory stimulus, and visual and auditory senses based on the environmental information. Causes the computer to perform an information providing method that provides information to the user, including the step of selecting one of the sensory stimulus output units that output different sensory stimuli.

According to this embodiment, information can be appropriately provided to the user.

FIG. 1 is a schematic diagram of an information providing device according to the present embodiment. FIG. 2 is a diagram showing an example of an image displayed by the information providing device. FIG. 3 is a schematic block diagram of the information providing device according to the present embodiment. FIG. 4 is a flowchart illustrating the processing contents of the information providing device according to the present embodiment. FIG. 5 is a table illustrating an example of an environmental score. FIG. 6 is a table showing an example of an environmental pattern. FIG. 7 is a schematic diagram illustrating an example of the level of the output specification of the content image. FIG. 8 is a table showing the relationship between the environmental pattern, the target device, and the reference output specifications. FIG. 9 is a graph showing an example of a pulse wave. FIG. 10 is a table showing an example of the relationship between the user state and the output specification correction degree. FIG. 11 is a table showing an example of output restriction necessity information.

Hereinafter, the present embodiment will be described in detail based on the drawings. The present embodiment is not limited to the embodiments described below.

(Information providing device)
FIG. 1 is a schematic diagram of an information providing device according to the present embodiment. The information providing device 10 according to the present embodiment is a device that provides information to the user U by outputting a visual stimulus, an auditory stimulus, and a sensory stimulus to the user U. The sensory stimulus here is a stimulus for a sensation different from the visual and auditory senses. In the present embodiment, the sensory stimulus is a tactile stimulus, but is not limited to the tactile stimulus, and may be a stimulus for any sensation different from the visual sense and the auditory sense. For example, the sensory stimulus may be a stimulus for the sense of taste, a stimulus for the sense of smell, or a stimulus for two or more of the sense of touch, taste, and hearing. As shown in FIG. 1, the information providing device 10 is a so-called wearable device worn on the body of the user U. In the example of the present embodiment, the information providing device 10 includes a device 10A worn on the eyes of the user U, a device 10B worn on the ears of the user U, and a device 10C worn on the arm of the user U. The device 10A worn on the eyes of the user U includes a display unit 26A described later that outputs a visual stimulus to the user U (displays an image), and the device 10B worn on the ear of the user U is an auditory stimulus to the user U. The device 10C attached to the arm of the user U includes a later-described voice output unit 26B that outputs (voice), and includes a later-described sensory stimulus output unit 26C that outputs a sensory stimulus to the user U. However, the configuration of FIG. 1 is an example, and the number of devices and the mounting position on the user U may be arbitrary. For example, the information providing device 10 is not limited to a wearable device, and may be a device carried by the user U, for example, a so-called smartphone or tablet terminal.

(Environmental image)
FIG. 2 is a diagram showing an example of an image displayed by the information providing device. As shown in FIG. 2, the information providing device 10 provides the environment image PM to the user U through the display unit 26A. As a result, the user U wearing the information providing device 10 can visually recognize the environment image PM. In the present embodiment, the environment image PM is an image of a landscape that the user U will see when it is assumed that the user U is not equipped with the information providing device 10, and is within the field of view of the user U. It can be said that it is an image of a real object. In the present embodiment, the information providing device 10 provides the environment image PM to the user U, for example, by transmitting external light (peripheral visible light) from the display unit 26A. That is, in the present embodiment, it can be said that the user U directly visually recognizes the image of the actual scenery through the display unit 26A. However, the information providing device 10 is not limited to allowing the user U to directly visually recognize the image of the actual scenery, and by displaying the image of the environment image PM on the display unit 26A, the user U can see the environment image PM through the display unit 26A. May be provided. In this case, the user U will visually recognize the image of the scenery displayed on the display unit 26A as the environment image PM. In this case, the information providing device 10 causes the display unit 26A to display an image captured by the camera 20A, which will be described later, within the visual field range of the user U as an environment image PM. In FIG. 2, roads and buildings are included as the environmental image PM, but this is just an example.

(Content image)
As shown in FIG. 2, the information providing device 10 causes the display unit 26A to display the content image PS. The content image PS is an image other than the actual scenery within the field of view of the user U. The content image PS may be any content (content) as long as it is an image including information to be notified to the user U. For example, the content image PS may be a distribution image such as a movie or a TV program, a navigation image showing directions to the user U, or a communication to the user U such as a telephone or an e-mail. It may be a notification image indicating that the above is received, or it may be an image including all of them. The content image PS may not include an advertisement which is information notifying a product or service.

In the example of FIG. 2, the content image PS is displayed on the display unit 26A so as to be superimposed on the environment image PM provided through the display unit 26A. As a result, the user U can visually recognize the image in which the content image PS is superimposed on the environment image PM. However, the method of displaying the content image PS is not limited to superimposing as shown in FIG. The method of displaying the content image PS, that is, the output specifications described later are set by, for example, environmental information, and will be described in detail later.

(Configuration of information providing device)
FIG. 3 is a schematic block diagram of the information providing device according to the present embodiment. As shown in FIG. 3, the information providing device 10 includes an environment sensor 20, a biological sensor 22, an input unit 24, an output unit 26, a communication unit 28, a storage unit 30, and a control unit 32.

(Environment sensor)
The environment sensor 20 is a sensor that detects environmental information around the information providing device 10. It can be said that the environmental information around the information providing device 10 is information indicating under what kind of environment the information providing device 10 is placed. Further, since the information providing device 10 is attached to the user U, it can be paraphrased that the environment sensor 20 detects the environmental information around the user U.

The environment sensor 20 includes a camera 20A, a microphone 20B, a GNSS receiver 20C, an acceleration sensor 20D, a gyro sensor 20E, an optical sensor 20F, a temperature sensor 20G, and a humidity sensor 20H. However, the environment sensor 20 may include an arbitrary sensor that detects environmental information, for example, a camera 20A, a microphone 20B, a GNSS receiver 20C, an acceleration sensor 20D, a gyro sensor 20E, and an optical sensor. It may include at least one of the sensor 20F, the temperature sensor 20G, and the humidity sensor 20H, or may include another sensor.

The camera 20A is an image pickup device, and captures the surroundings of the information providing device 10 by detecting visible light around the information providing device 10 (user U) as environmental information. The camera 20A may be a video camera that captures images at predetermined frame rates. The position and orientation of the camera 20A in the information providing device 10 are arbitrary, but for example, the camera 20A is provided in the device 10A shown in FIG. 1 and the imaging direction is the direction in which the face of the user U is facing. It may be there. As a result, the camera 20A can image an object in the line of sight of the user U, that is, an object within the field of view of the user U. Further, the number of cameras 20A is arbitrary, and may be singular or plural. If there are a plurality of cameras 20A, the information in the direction in which the cameras 20A are facing is also acquired.

The microphone 20B is a microphone that detects voice (sound wave information) around the information providing device 10 (user U) as environmental information. The position, orientation, number, and the like of the microphone 20B provided in the information providing device 10 are arbitrary. If there are a plurality of microphones 20B, information in the direction in which the microphones 20B are facing is also acquired.

The GNSS receiver 20C is a device that detects the position information of the information providing device 10 (user U) as environmental information. The position information here is the earth coordinates. In the present embodiment, the GNSS receiver 20C is a so-called GNSS (Global Navigation Satellite System) module, which receives radio waves from satellites and detects the position information of the information providing device 10 (user U).

The acceleration sensor 20D is a sensor that detects the acceleration of the information providing device 10 (user U) as environmental information, and detects, for example, gravity, vibration, and impact.

The gyro sensor 20E is a sensor that detects the rotation and orientation of the information providing device 10 (user U) as environmental information, and detects it using the principle of Coriolis force, Euler force, centrifugal force, and the like.

The optical sensor 20F is a sensor that detects the intensity of light around the information providing device 10 (user U) as environmental information. The optical sensor 20F can detect the intensity of visible light, infrared rays, and ultraviolet rays.

The temperature sensor 20G is a sensor that detects the temperature around the information providing device 10 (user U) as environmental information.

The humidity sensor 20H is a sensor that detects the humidity around the information providing device 10 (user U) as environmental information.

(Biological sensor)
The biosensor 22 is a sensor that detects the biometric information of the user U. The biosensor 22 may be provided at any position as long as it can detect the biometric information of the user U. The biometric information here is not immutable such as a fingerprint, but is preferably information whose value changes according to the state of the user U, for example. Furthermore, it is preferable that the biometric information here is information about the autonomic nerve of the user U, that is, information whose value changes regardless of the intention of the user U. Specifically, the biological sensor 22 includes the pulse wave sensor 22A and the brain wave sensor 22B, and detects the pulse wave and the brain wave of the user U as biological information.

The pulse wave sensor 22A is a sensor that detects the pulse wave of the user U. The pulse wave sensor 22A may be, for example, a transmissive photoelectric sensor including a light emitting unit and a light receiving unit. In this case, the pulse wave sensor 22A is configured such that, for example, the light emitting portion and the light receiving portion face each other with the fingertip of the user U interposed therebetween, and the light receiving portion receives the light transmitted through the fingertip, and the pressure of the pulse wave. The pulse waveform may be measured by utilizing the fact that the larger the value, the larger the blood flow. However, the pulse wave sensor 22A is not limited to this, and may be any method capable of detecting a pulse wave.

The brain wave sensor 22B is a sensor that detects the brain wave of the user U. The brain wave sensor 22B may have any configuration as long as it can detect the brain wave of the user U, but in principle, for example, a wave such as an α wave or a β wave or a basic rhythm (background brain wave) that appears in the entire brain. It suffices if the activity can be grasped and the improvement or decrease of the activity of the entire brain can be detected. In the present embodiment, unlike the electroencephalogram measurement for medical purposes, it suffices to be able to roughly measure the change in the state of the user U. Therefore, for example, by attaching only two electrodes to the forehead and the ear, a very simple surface electroencephalogram can be obtained. It is also possible to detect the detection.

The biological sensor 22 is not limited to detecting pulse waves and brain waves as biological information, and may detect at least one of pulse waves and brain waves, for example. Further, the biological sensor 22 may detect other than pulse waves and brain waves as biological information, and may detect, for example, the amount of sweating and the size of the pupil. Further, the biosensor 22 is not an essential configuration and may not be provided in the information processing apparatus 10.

(Input section)
The input unit 24 is a device that accepts user operations, and may be, for example, a touch panel.

(Output section)
The output unit 26 is a device that outputs a stimulus for at least one of the five senses to the user U. Specifically, the output unit 26 includes a display unit 26A, a voice output unit 26B, and a sensory stimulation output unit 26C. The display unit 26A is a display that outputs the visual stimulus of the user U by displaying an image, and can be paraphrased as a visual stimulus output unit. In the present embodiment, the display unit 26A is a so-called HMD (Head Mount Display). The display unit 26A displays the content image PS as described above. The voice output unit 26B is a device (speaker) that outputs the auditory stimulus of the user U by outputting the voice, and can be paraphrased as the auditory stimulus output unit. The sensory stimulus output unit 26C is a device that outputs the sensory stimulus of the user U, and in the present embodiment, the tactile stimulus. For example, the sensory stimulus output unit 26C is a vibration motor such as a vibrator, and outputs a tactile stimulus to the user by physically operating such as vibration. However, the type of the tactile stimulus is not limited to vibration or the like. It may be a thing.

In this way, the output unit 26 stimulates the visual sense, the auditory sense, and the senses different from the visual sense and the auditory sense (tactile sense in the present embodiment) among the five human senses. However, the output unit 26 is not limited to outputting visual stimuli, auditory stimuli, and sensations different from those of visual and auditory senses. For example, the output unit 26 may output at least one of visual stimuli, auditory stimuli, and sensations different from visual and auditory sensations, or at least outputs visual stimuli (displays an image). It may be present, or it may output either an auditory stimulus or a tactile sensation in addition to the visual stimulus, and it may be one of the five senses in addition to at least one of the visual stimulus, the auditory stimulus, and the tactile sensation. It may output other sensory stimuli (that is, at least one of taste stimuli and olfactory stimuli).

(Communication department)
The communication unit 28 is a module that communicates with an external device or the like, and may include, for example, an antenna or the like. The communication method by the communication unit 28 is wireless communication in this embodiment, but the communication method may be arbitrary. The communication unit 28 includes a content image receiving unit 28A. The content image receiving unit 28A is a receiver that receives the content image data, which is the image data of the content image. The content displayed by the content image may include audio and sensory stimuli different from visual and auditory senses. In this case, the content image receiving unit 28A may receive voice data and sensory stimulation data as well as the image data of the content image as the content image data. The content image data is received by the content image receiving unit 28A in this way. For example, the content image receiving unit 28A is stored in the storage unit 30 in advance, and the content image receiving unit 28A reads the content image data from the storage unit 30. May be good.

(Memory)
The storage unit 30 is a memory that stores various information such as calculation contents and programs of the control unit 32. For example, a RAM (Random Access Memory), a main storage device such as a ROM (Read Only Memory), and an HDD ( Includes at least one of external storage devices such as Hard Disk Drive).

The storage unit 30 stores the learning model 30A, the map data 30B, and the specification setting database 30C. The learning model 30A is an AI model used to specify the environment in which the user U is located based on the environment information. The map data 30B is data including position information of actual buildings and natural objects, and can be said to be data in which the earth coordinates and actual buildings and natural objects are associated with each other. The specification setting database 30C is a database that includes information for determining the display specifications of the content image PS as described later. The processing using the learning model 30A, the map data 30B, the specification setting database 30C, and the like will be described later. The learning model 30A, the map data 30B, the specification setting database 30C, and the program for the control unit 32 stored by the storage unit 30 may be stored in a recording medium readable by the information providing device 10. Further, the program for the control unit 32 stored by the storage unit 30, the learning model 30A, the map data 30B, and the specification setting database 30C are not limited to being stored in advance in the storage unit 30, and these data are stored. When used, the information providing device 10 may acquire from an external device by communication.

(Control unit)
The control unit 32 is an arithmetic unit, that is, a CPU (Central Processing Unit). The control unit 32 includes an environment information acquisition unit 40, a biometric information acquisition unit 42, an environment identification unit 44, a user state identification unit 46, an output selection unit 48, an output specification determination unit 50, and a content image acquisition unit 52. And an output control unit 54. The control unit 32 reads out a program (software) from the storage unit 30 and executes it to output the environment information acquisition unit 40, the biometric information acquisition unit 42, the environment identification unit 44, the user state identification unit 46, the output selection unit 48, and the output. The specification determination unit 50, the content image acquisition unit 52, and the output control unit 54 are realized, and their processing is executed. The control unit 32 may execute these processes by one CPU, or may include a plurality of CPUs and execute the processes by the plurality of CPUs. Further, at least one of the environment information acquisition unit 40, the biometric information acquisition unit 42, the environment identification unit 44, the user state identification unit 46, the output selection unit 48, the output specification determination unit 50, the content image acquisition unit 52, and the output control unit 54. The part may be realized by hardware.

The environment information acquisition unit 40 controls the environment sensor 20 to cause the environment sensor 20 to detect the environment information. The environmental information acquisition unit 40 acquires the environmental information detected by the environment sensor 20. The processing of the environment information acquisition unit 40 will be described later. When the environmental information acquisition unit 40 is hardware, it can also be called an environmental information detector.

The biometric information acquisition unit 42 controls the biometric sensor 22 to cause the biometric sensor 22 to detect biometric information. The biological information acquisition unit 42 acquires the environmental information detected by the biological sensor 22. The processing of the biological information acquisition unit 42 will be described later. When the biometric information acquisition unit 42 is hardware, it can also be called a biometric information detector. The biological information acquisition unit 42 is not an essential configuration.

The environment specifying unit 44 identifies the environment in which the user U is placed, based on the environment information acquired by the environment information acquisition unit 40. The environment specifying unit 44 calculates the environment score, which is a score for specifying the environment, and specifies the environment by specifying the environment state pattern indicating the state of the environment based on the environment score. The processing of the environment specifying unit 44 will be described later.

The user state specifying unit 46 specifies the state of the user U based on the biometric information acquired by the biometric information acquisition unit 42. The processing of the user state specifying unit 46 will be described later. The user state specifying unit 46 is not an essential configuration.

The output selection unit 48 selects a target device to be operated in the output unit 26 based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the biometric information acquired by the biometric information acquisition unit 42. The processing of the output selection unit 48 will be described later. When the output selection unit 48 is hardware, it may be called a sensory selector. When the output specification determination unit 50, which will be described later, determines the output specification based on environmental information or the like, the output selection unit 48 may not be provided. In this case, for example, the information providing device 10 operates all of the output units 26, that is, all of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C without selecting the target device. It's okay.

The output specification determination unit 50 outputs a stimulus output by the output unit 26 based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the biological information acquired by the biological information acquisition unit 42 (here, a visual stimulus, Determine the output specifications of auditory and tactile stimuli). For example, the output specification determination unit 50 is a content image PS displayed by the display unit 26A based on at least one of the environmental information acquired by the environmental information acquisition unit 40 and the biometric information acquired by the biometric information acquisition unit 42. It can be said that the display specifications (output specifications) are determined. The output specification is an index showing how the stimulus output by the output unit 26 is output, and the details will be described later. The processing of the output specification determination unit 50 will be described later. When the output selection unit 48 selects the target device based on environmental information or the like, the output specification determination unit 50 may not be provided. In this case, for example, the information providing device 10 may output the stimulus to the selected target device with an arbitrary output specification without determining the output specification from the environmental information or the like.

The content image acquisition unit 52 acquires content image data via the content image receiving unit 28A.

The output control unit 54 controls the output unit 26 to output. The output control unit 54 causes the target device selected by the output selection unit 48 to output with the output specifications determined by the output specification determination unit 50. For example, the output control unit 54 controls the display unit 26A to superimpose the content image PS acquired by the content image acquisition unit 52 on the main image PM, and has a display specification determined by the output specification determination unit 50. To display. When the output control unit 54 is hardware, it may be called a multi-sensory sensory provider.

The information providing device 10 has the configuration as described above.

(Processing content)
Next, the processing content by the information providing device 10 and, more specifically, the processing content to be output to the output unit 26 based on the environmental information and the biological information will be described. FIG. 4 is a flowchart illustrating the processing contents of the information providing device according to the present embodiment.

(Acquisition of environmental information)
As shown in FIG. 4, the information providing device 10 acquires the environmental information detected by the environment sensor 20 by the environment information acquisition unit 40 (step S10). In the present embodiment, the environmental information acquisition unit 40 acquires image data obtained by capturing an image of the periphery of the information providing device 10 (user U) from the camera 20A, and acquires image data around the information providing device 10 (user U) from the microphone 20B. The voice data is acquired, the position information of the information providing device 10 (user U) is acquired from the GNSS receiver 20C, the acceleration information of the information providing device 10 (user U) is acquired from the acceleration sensor 20D, and the gyro sensor 20E is acquired. The direction information of the information providing device 10 (user U), that is, the attitude information is acquired from, and the intensity information of infrared rays and ultraviolet rays around the information providing device 10 (user U) is acquired from the optical sensor 20F, and the temperature sensor 20G is obtained. The temperature information around the information providing device (user U) is acquired from, and the humidity information around the information providing device 10 (user U) is acquired from the humidity sensor 20H. The environmental information acquisition unit 40 sequentially acquires these environmental information at predetermined intervals. The environmental information acquisition unit 40 may acquire each environmental information at the same timing, or may acquire each environmental information at different timings. Further, the predetermined period until the next environmental information is acquired may be arbitrarily set, and the predetermined period may be the same or different for each environmental information.

(Judgment of dangerous condition)
After acquiring the environment information, the information providing device 10 determines whether the environment around the user U is in a dangerous state based on the environment information by the environment specifying unit 44 (step S12). ..

The environment specifying unit 44 determines whether or not it is in a dangerous state based on the image around the information providing device 10 captured by the camera 20A. Hereinafter, the image of the periphery of the information providing device 10 captured by the camera 20A will be appropriately referred to as a peripheral image. The environment specifying unit 44 identifies, for example, an object shown in a peripheral image, and determines whether or not it is in a dangerous state based on the type of the specified object. More specifically, the environment specifying unit 44 determines that the object shown in the peripheral image is in a dangerous state when it is a preset specific object, and determines that it is not in a dangerous state when it is not a specific object. It's okay. The specific object may be set arbitrarily, but it may be an object that may pose a danger to the user U, such as a flame indicating that it is a fire, a vehicle, or a sign indicating that construction is underway. It may be there. Further, the environment specifying unit 44 may determine whether or not it is in a dangerous state based on a plurality of peripheral images continuously captured in time series. For example, the environment specifying unit 44 identifies an object for each of a plurality of peripheral images continuously captured in time series, and whether the object is a specific object and is the same object. To judge. Then, when the same specific object is captured, the environment specifying unit 44 determines whether the specific object reflected in the peripheral image captured later in the time series is relatively larger in the image, that is, the specific object. It is determined whether the specific object is approaching the user U. Then, the environment specifying unit 44 determines that it is in a dangerous state when the specific object is larger than the specific object shown in the peripheral image captured later, that is, when the specific object is approaching the user U. .. On the other hand, the environment specifying unit 44 determines that it is not in a dangerous state when it is not as large as the specific object shown in the peripheral image captured later, that is, when the specific object is not approaching the user U. .. In this way, the environment specifying unit 44 may determine whether it is a dangerous state based on one peripheral image, or determine whether it is a dangerous state based on a plurality of peripheral images continuously captured in time series. You may. For example, the environment specifying unit 44 may switch the determination method according to the type of the object shown in the peripheral image. When a specific object that can be judged as dangerous from one peripheral image such as a flame indicating a fire is shown, the environment specifying unit 44 may determine from one peripheral image that it is in a dangerous state. Further, for example, when a specific object such as a vehicle whose danger cannot be determined is captured from one peripheral image, the environment specifying unit 44 is in a dangerous state based on a plurality of peripheral images continuously captured in time series. You may make a judgment.

The environment specifying unit 44 may specify the object shown in the peripheral image by any method, but for example, the learning model 30A may be used to specify the object. In this case, for example, the learning model 30A is constructed by learning image data and information indicating the type of an object shown in the image as one data set and learning a plurality of data sets as teacher data. It is an AI model. The environment specifying unit 44 inputs the image data of the peripheral image into the learned learning model 30A, acquires the information specifying the type of the object reflected in the peripheral image, and identifies the object. ..

Further, the environment specifying unit 44 may determine whether or not it is in a dangerous state based on the position information acquired by the GNSS receiver 20C in addition to the peripheral image. In this case, the environment specifying unit 44 acquires the location information indicating the location of the user U based on the location information of the information providing device 10 (user U) acquired by the GNSS receiver 20C and the map data 30B. The whereabouts information is information indicating what kind of place the user U (information providing device 10) is in. That is, for example, the whereabouts information is information that the user U is in the shopping center, information that the user U is on the road, and the like. The environment specifying unit 44 reads out the map data 30B, identifies the type of the structure or the natural object within a predetermined distance range with respect to the current position of the user U, and specifies the location information from the structure or the natural object. For example, when the current position of the user U overlaps with the coordinates of the shopping center, it is specified as the location information that the user U is in the shopping center. Then, the environment specifying unit 44 determines that the location information and the type of the object specified from the surrounding image are in a dangerous state when they have a specific relationship, and when they do not have a specific relationship, the dangerous state. Judge that it is not. A specific relationship may be set arbitrarily, but for example, a combination of an object and a whereabouts, which may pose a danger if the object exists in a certain place, is set as a specific relationship. It's okay.

Further, the environment specifying unit 44 determines whether or not it is in a dangerous state based on the voice information acquired by the microphone 20B. Hereinafter, the audio information around the information providing device 10 acquired by the microphone 20B will be appropriately referred to as peripheral audio. The environment specifying unit 44 identifies, for example, the type of voice included in the peripheral voice, and determines whether or not it is in a dangerous state based on the type of the specified voice. More specifically, the environment specifying unit 44 determines that if the type of voice included in the peripheral voice is a preset specific voice, it is in a dangerous state, and if it is not a specific voice, it is determined that it is not in a dangerous state. It's okay. The specific voice may be set arbitrarily, but for example, a voice indicating that it is a fire, a voice indicating that the vehicle is under construction, or a voice indicating that the user U is under construction, which may pose a danger to the user U. It may be there.

The environment specifying unit 44 may specify the type of voice included in the peripheral voice by any method, but may specify the object by using, for example, the learning model 30A. In this case, for example, in the learning model 30A, voice data (for example, data indicating the frequency and intensity of sound) and information indicating the type of the voice are used as one data set, and a plurality of data sets are learned as teacher data. It is a built AI model. The environment specifying unit 44 inputs the voice data of the peripheral voice into the learned learning model 30A, acquires the information specifying the type of the voice included in the peripheral voice, and specifies the voice type. ..

Further, the environment specifying unit 44 may determine whether or not it is in a dangerous state based on the position information acquired by the GNSS receiver 20C in addition to the peripheral voice. In this case, the environment specifying unit 44 acquires the location information indicating the location of the user U based on the location information of the information providing device 10 (user U) acquired by the GNSS receiver 20C and the map data 30B. Then, the environment specifying unit 44 determines that the location information and the type of voice specified from the surrounding voice are in a dangerous state when they have a specific relationship, and when they do not have a specific relationship, they are not in a dangerous state. Judge. The specific relationship may be set arbitrarily, but for example, a combination of sound and whereabouts, which may be dangerous if the sound is generated in a certain place, may be set as a specific relationship. ..

As described above, in the present embodiment, the environment specifying unit 44 determines the dangerous state based on the peripheral image and the peripheral sound. However, the method for determining the dangerous state is not limited to the above and is arbitrary. For example, the environment specifying unit 44 may determine the dangerous state based on either the peripheral image or the peripheral sound. Further, the environment specifying unit 44 has at least an image of the periphery of the information providing device 10 captured by the camera 20A, a sound around the information providing device 10 detected by the microphone 20B, and a position information acquired by the GNSS receiver 20C. You may determine if you are in a dangerous state based on one. Further, in the present embodiment, the determination of the dangerous state is not essential and may not be carried out.

(Setting of danger notification content)
When it is determined that the dangerous state is determined (step S10, Yes), the information providing device 10 sets the danger notification content, which is the notification content for notifying the dangerous state, by the output control unit 54 (step S12). The information providing device 10 sets the danger notification content based on the content of the danger state. The content of the dangerous state is information indicating what kind of danger is imminent, and is specified from the type of the object shown in the peripheral image, the type of sound included in the peripheral sound, and the like. For example, when the object is a vehicle and is approaching, the content of the dangerous state is "the vehicle is approaching". The content of the danger notification is information indicating the content of the dangerous state. For example, when the content of the dangerous state is that the vehicle is approaching, the content of the danger notification is information indicating that the vehicle is approaching.

The content of the danger notification differs depending on the type of the target device selected in step S26 described later. For example, when the display unit 26A is the target device, the danger notification content is the display content (content) of the content image PS. That is, the danger notification content is displayed as the content image PS. In this case, for example, the content of the danger notification is image data indicating the content "Be careful because the car is approaching!". On the other hand, when the voice output unit 26B is the target device, the danger notification content is the content of the voice output from the voice output unit 26B. In this case, for example, the content of the danger notification is voice data for issuing a voice saying "A car is approaching. Please be careful". When the sensory stimulus output unit 26C is the target device, the danger notification content is the content of the sensory stimulus output from the sensory stimulus output unit 26C. In this case, for example, the danger notification content is a tactile stimulus that attracts the attention of the user U.

The setting of the danger notification content in step S14 may be executed at an arbitrary timing after the danger notification content is determined in step S12 and before the danger notification content is output in the subsequent step S38. For example, it may be executed after selecting the target device in the subsequent step S32.

(Calculation of environmental score)
When it is determined that the state is not dangerous (step S12; No), the information providing device 10 calculates various environmental scores based on the environmental information by the environment specifying unit 44 as shown in steps S16 to S22. The environment score is a score for specifying the environment in which the user U (information providing device 10) is placed. Specifically, the environment specifying unit 44 calculates the posture score (step S16), the whereabouts score (step S18), the movement score (step S20), and the safety score as the environment score. (Step S22). The order from step S16 to step S22 is not limited to this, and is arbitrary. Even when the danger notification content is set in step S14, various environmental scores are calculated as shown in steps S16 to S22. Hereinafter, the environmental score will be described more specifically.

FIG. 5 is a table illustrating an example of an environmental score. As shown in FIG. 5, the environment specifying unit 44 calculates an environment score for each environment category. The environment category indicates the type of environment of user U. In the example of FIG. 5, the posture of user U, the location of user U, the movement of user U, and the safety of the environment around user U are shown. And, including. Further, the environment specifying unit 44 divides the environment category into more specific subcategories, and calculates the environment score for each subcategory.

(Posture score)
The environment specifying unit 44 calculates the posture score as the environment score for the posture category of the user U. That is, the posture score is information indicating the posture of the user U, and can be said to be information indicating what kind of posture the user U is in as a numerical value. The environment specifying unit 44 calculates the posture score based on the environment information related to the posture of the user U among the plurality of types of environment information. Environmental information related to the posture of the user U includes a peripheral image acquired by the camera 20A and the orientation of the information providing device 10 detected by the gyro sensor 20E.

More specifically, in the example of FIG. 5, the posture category of the user U includes a subcategory of standing and a subcategory of the face facing horizontally. The environment specifying unit 44 calculates the posture score for the sub-category of standing state based on the peripheral image acquired by the camera 20A. The posture score for the subcategory of the standing state can be said to be a numerical value indicating the degree of matching of the posture of the user U with the standing state. The method of calculating the posture score for the sub-category of standing may be arbitrary, but for example, it may be calculated using the learning model 30A. In this case, for example, in the learning model 30A, the image data of the scenery reflected in the field of view of a person and the information indicating whether the person is standing are used as one data set, and a plurality of data sets are learned as teacher data. It is a constructed AI model. By inputting the image data of the peripheral image into the learned learning model 30A, the environment specifying unit 44 acquires a numerical value indicating the degree of coincidence with the standing state and uses it as a posture score. Although the degree of agreement with respect to the standing state is used here, the degree of agreement is not limited to the standing state, and may be, for example, the degree of agreement with a sitting state or a sleeping state.

Further, the environment specifying unit 44 calculates the posture score for the sub-category that the face orientation is horizontal based on the orientation of the information providing device 10 detected by the gyro sensor 20E. The posture score for the subcategory in which the face orientation is horizontal can be said to be a numerical value indicating the degree of coincidence of the posture (face orientation) of the user U with respect to the horizontal direction. The method of calculating the posture score for the subcategory in which the face orientation is horizontal may be arbitrary. Although the degree of coincidence with respect to the horizontal direction of the face is used here, the degree of coincidence with respect to the horizontal direction may be used.

In this way, it can be said that the environment specifying unit 44 sets information (here, the posture score) indicating the posture of the user U based on the peripheral image and the orientation of the information providing device 10. However, the environment specifying unit 44 is not limited to using the peripheral image and the orientation of the information providing device 10 in order to set the information indicating the posture of the user U, and may use arbitrary environmental information, for example, the peripheral image. And at least one of the orientation of the information providing device 10 may be used.

(Whereabouts score)
The environment specifying unit 44 calculates the whereabouts score as the environment score for the category of the whereabouts of the user U. That is, the whereabouts score is information indicating the whereabouts of the user U, and can be said to be information indicating what kind of place the user U is located in as a numerical value. The environment specifying unit 44 calculates the location score based on the environment information related to the location of the user U among the plurality of types of environment information. Environmental information related to the location of the user U includes peripheral images acquired by the camera 20A, position information of the information providing device 10 acquired by the GNSS receiver 20C, and peripheral audio acquired by the microphone 20B. Be done.

More specifically, in the example of FIG. 5, the category of the whereabouts of the user U includes a subcategory of being on the train, a subcategory of being on the railroad track, and a subcategory of being the sound in the train. ing. The environment specifying unit 44 calculates the whereabouts score for the subcategory of being in the train based on the peripheral image acquired by the camera 20A. The whereabouts score for the subcategory of being on the train can be said to be a numerical value indicating the degree of matching of the whereabouts of the user U with respect to the place of being on the train. The method of calculating the whereabouts score for the subcategory of being in the train may be arbitrary, but for example, it may be calculated using the learning model 30A. In this case, for example, in the learning model 30A, the image data of the scenery reflected in the field of view of a person and the information indicating whether the person is in the train are used as one data set, and a plurality of data sets are learned as teacher data. It is an AI model constructed by. By inputting the image data of the peripheral image into the learned learning model 30A, the environment specifying unit 44 acquires a numerical value indicating the degree of coincidence with the location in the train and uses it as the location score. Although the degree of coincidence with respect to the location in the train is calculated here, the degree of coincidence with respect to being in any type of vehicle may be calculated without limitation.

The environment specifying unit 44 calculates the whereabouts score for the subcategory of being on the railroad track based on the position information of the information providing device 10 acquired by the GNSS receiver 20C. The whereabouts score for the subcategory of being on the railroad track can be said to be a numerical value indicating the degree of matching of the whereabouts of the user U with the whereabouts of being on the railroad track. The method of calculating the whereabouts score for the sub-category of being on the railroad track may be arbitrary, but for example, map data 30B may be used. For example, the environment specifying unit 44 reads out the map data 30B, and when the current position of the user U overlaps with the coordinates of the railroad track, the location score is such that the degree of matching of the user U's location with the location on the track is high. Is calculated. Although the degree of coincidence on the track is calculated here, the degree of coincidence with the position of any kind of structure or natural object may be calculated without limitation.

The environment specifying unit 44 calculates the whereabouts score for the subcategory that it is the sound in the train based on the peripheral voice acquired by the microphone 20B. The whereabouts score for the subcategory of sounds in the train can be said to be a numerical value indicating the degree of matching of the surrounding sounds with the sounds in the train. The method of calculating the whereabouts score for the subcategory of sound in the train may be arbitrary, but for example, in the same manner as the method of determining whether or not a dangerous state is based on the surrounding voice as described above, that is, for example, for example. Judgment may be made by determining whether the peripheral sound is a specific type of sound. Although the degree of matching with the sound in the train is calculated here, the degree of matching with the sound in any place may be calculated without limitation.

In this way, it can be said that the environment specifying unit 44 sets information indicating the whereabouts of the user U (here, the whereabouts score) based on the peripheral image, the peripheral voice, and the position information of the information providing device 10. However, the environment specifying unit 44 is not limited to using the peripheral image, the peripheral voice, and the position information of the information providing device 10 in order to set the information indicating the location of the user U, and may use any environmental information. For example, at least one of a peripheral image, a peripheral sound, and a position information of the information providing device 10 may be used.

(Movement score)
The environment specifying unit 44 calculates the movement score as the environment score for the movement category of the user U. That is, the movement score is information indicating the movement of the user U, and can be said to be information indicating how the user U is moving as a numerical value. The environment specifying unit 44 calculates the motion score based on the environmental information related to the motion of the user U among the plurality of types of environmental information. Examples of the environmental information related to the movement of the user U include the acceleration information acquired by the acceleration sensor 20D.

More specifically, in the example of FIG. 5, a subcategory that the user U is moving is included with respect to the movement category of the user U. The environment specifying unit 44 calculates the whereabouts score for the subcategory of moving based on the acceleration information of the information providing device 10 acquired by the acceleration sensor 20D. The movement score for the subcategory of moving can be said to be a numerical value indicating the degree of agreement between the current situation of the user U and the movement of the user U. The method of calculating the movement score for the subcategory of moving may be arbitrary, but for example, the movement score may be calculated from the change in acceleration in a predetermined period. For example, when there is a change in acceleration in a predetermined period, the movement score is calculated so that the degree of agreement with the movement of the user U is high. Further, for example, the position information of the information providing device 10 may be acquired and the movement score may be calculated based on the degree of change in the position in a predetermined period. In this case, the speed can be predicted from the amount of change in position during a predetermined period, and the means of transportation such as a vehicle or walking can be specified. Although the degree of coincidence for moving is calculated here, the degree of coincidence for moving at a predetermined speed may be calculated, for example.

As described above, it can be said that the environment specifying unit 44 sets the information indicating the movement of the user U (here, the movement score) based on the acceleration information of the information providing device 10 and the position information of the information providing device 10. However, the environment specifying unit 44 is not limited to using the acceleration information and the position information in order to set the information indicating the movement of the user U, and may use any environment information, for example, the acceleration information and the position information. At least one may be used.

(Safety score)
The environment specifying unit 44 calculates the safety score as the environment score for the safety category of the user U. That is, the safety score is information indicating the safety of the user U, and can be said to be information indicating whether the user U is in a safe environment as a numerical value. The environment specifying unit 44 calculates the safety score based on the environmental information related to the safety of the user U among the plurality of types of environmental information. Environmental information related to the safety of the user U includes the peripheral image acquired by the camera 20A, the peripheral sound acquired by the microphone 20B, the light intensity information detected by the optical sensor 20F, and the temperature sensor 20G. Examples include the detected ambient temperature information and the ambient humidity information detected by the humidity sensor 20H.

More specifically, in the example of FIG. 5, for the safety category of the user U, the subcategory of being bright, the subcategory of having an appropriate amount of infrared rays and ultraviolet rays, and the subcategory of having an appropriate temperature are suitable. It includes a sub-category of high humidity and a sub-category of dangerous goods. The environment specifying unit 44 calculates a safety score for the subcategory of brightness based on the intensity of visible light in the surroundings acquired by the optical sensor 20F. The safety score for the bright subcategory can be said to be a numerical value indicating the degree of matching of the surrounding brightness with sufficient brightness. The method of calculating the safety score for the subcategory of bright may be arbitrary, but for example, it may be calculated based on the intensity of visible light detected by the optical sensor 20F. Further, for example, a safety score for the subcategory of brightness may be calculated based on the brightness of the image captured by the camera 20A. Although the degree of coincidence with respect to sufficient brightness is calculated here, the degree of coincidence with respect to any degree of brightness may be calculated without limitation.

The environment specifying unit 44 calculates the safety score for the subcategory that the amount of infrared rays and ultraviolet rays is appropriate based on the intensity of infrared rays and ultraviolet rays in the vicinity acquired by the optical sensor 20F. The safety score for the subcategory that the amount of infrared rays and ultraviolet rays is appropriate can be said to be a numerical value indicating the degree of matching of the intensities of surrounding infrared rays and ultraviolet rays with the appropriate intensities of infrared rays and ultraviolet rays. The method of calculating the safety score for the subcategory that the amount of infrared rays or ultraviolet rays is appropriate may be arbitrary, but for example, it may be calculated based on the intensity of infrared rays or ultraviolet rays detected by the optical sensor 20F. Although the degree of coincidence with respect to the appropriate intensity of infrared rays and ultraviolet rays is calculated here, the degree of coincidence with respect to any intensity of infrared rays and ultraviolet rays may be calculated without limitation.

The environment specifying unit 44 calculates a safety score for the subcategory that the temperature is suitable based on the ambient temperature acquired by the temperature sensor 20G. The safety score for the subcategory of suitable temperature can be said to be a numerical value indicating the degree of agreement between the ambient temperature and the suitable temperature. The method of calculating the safety score for the subcategory of suitable temperature may be arbitrary, but may be calculated based on, for example, the ambient temperature detected by the temperature sensor 20G. Although the degree of coincidence with respect to a suitable temperature is calculated here, the degree of coincidence with respect to any temperature may be calculated without limitation.

The environment specifying unit 44 calculates a safety score for the subcategory that the humidity is suitable based on the surrounding humidity acquired by the humidity sensor 20H. The safety score for the subcategory of suitable humidity can be said to be a numerical value indicating the degree of agreement between the surrounding humidity and the suitable humidity. The method of calculating the safety score for the subcategory of suitable humidity may be arbitrary, but may be calculated based on, for example, the ambient humidity detected by the humidity sensor 20H. Although the degree of coincidence with respect to suitable humidity is calculated here, the degree of coincidence with respect to any humidity may be calculated without limitation.

The environment specifying unit 44 calculates the safety score for the subcategory that there is a dangerous substance based on the peripheral image acquired by the camera 20A. The safety score for the subcategory of dangerous goods can be said to be a numerical value indicating the degree of agreement with the presence of dangerous goods. The method of calculating the safety score for the subcategory that there is a dangerous substance may be arbitrary, but for example, it is the same method as the method of determining whether or not it is in a dangerous state based on the peripheral image as described above, that is, for example. , The judgment may be made by judging whether the object included in the peripheral image is a specific object. Further, the environment specifying unit 44 calculates a safety score for the subcategory that there is a dangerous substance based on the peripheral voice acquired by the microphone 20B. The method of calculating the safety score for the subcategory of dangerous goods may be arbitrary, but for example, in the same manner as the method of determining whether or not a dangerous state is based on the surrounding voice as described above, that is, for example. , The judgment may be made by judging whether the peripheral voice is a specific type of voice.

(Example of environmental score)
FIG. 5 illustrates the environmental scores calculated for the environment D1 to the environment D4. Environments D1 to D4 indicate cases where the user U is in a different environment, and an environment score for each category (sub-category) in each environment is calculated.

The types of environment categories and subcategories shown in FIG. 5 are examples, and the values of the environment scores in environments D1 to D4 are also examples. Further, the information providing device 10 can take an error or the like into consideration by expressing the information indicating the environment of the user U as a numerical value such as an environment score, and estimate the environment of the user U more accurately. can do. In other words, it can be said that the information providing device 10 can accurately estimate the environment of the user U by classifying the environmental information into any of three or more degrees (here, the environmental score). However, the information indicating the environment of the user U set by the information providing device 10 based on the environment information is not limited to a value such as an environment score, and may be data of any method, for example, Yes or No. Information indicating either of the two options may be used.

(Determination of environmental pattern)
The information providing device 10 calculates various environmental scores by the method described above in steps S16 to S22 shown in FIG. As shown in FIG. 4, after the information providing device 10 calculates the environment score, the environment specifying unit 44 determines an environment pattern indicating the environment in which the user U is placed based on each environment score (step). S24). That is, the environment specifying unit 44 determines how the user U is in the environment based on the environment score. While the environmental information and the environmental score are the information indicating some elements of the environment of the user U detected by the environment sensor 20, the environmental pattern is set based on the information indicating some elements. , It can be said that it is an index that comprehensively shows the environment.

FIG. 6 is a table showing an example of an environmental pattern. In the present embodiment, the environment specifying unit 44 selects an environment pattern that matches the environment in which the user U is placed from among the environment patterns corresponding to various environments, based on the environment score. In the present embodiment, for example, correspondence information (table) in which the value of the environmental score and the environmental pattern are associated with each other is recorded in the specification setting database 30C. The environment specifying unit 44 determines the environment pattern based on the environment information and the corresponding information. Specifically, the environment specifying unit 44 selects an environment pattern associated with the calculated environment score value from the corresponding information, and selects it as the environment pattern to be adopted. In the example of FIG. 6, the environment pattern PT1 indicates that the user U is sitting in the train, the environment pattern PT2 indicates that the user U is walking on the sidewalk, and the environment pattern PT3 indicates that the user U is walking on the sidewalk. It indicates that the user U is walking on a dark sidewalk, and the environmental pattern PT4 indicates that the user U is shopping.

In the examples of FIGS. 5 and 6, in the environment D1, the environment score of "standing" is 10, and the environment score of "face orientation is horizontal" is 100. Therefore, the user U sits down. It can be predicted that the face is turned almost horizontally. Further, since the environmental score of "inside the train" is 90, the environmental score of "on the railroad track" is 100, and the environmental score of "sound in the train" is 90, it can be seen that the user U is in the train. Further, since the environment score of "moving" is 100, it can be seen that the user U is moving with a constant velocity or acceleration. In addition, the environmental score of "bright" is 50, which means that it is darker than the outside because it is inside the train. In addition, the environmental scores of "infrared rays and ultraviolet rays are appropriate", "suitable temperature", and "suitable humidity" are 100, which can be said to be safe. In addition, the environmental score of "there is a dangerous substance" is 10 in terms of images and 20 in terms of sound, so this is also considered safe. That is, in the environment D1, it is possible to estimate that the user U is in a safe and comfortable situation while moving in the train from each environment score, and the environment pattern of the environment D1 is It is said to be the environmental pattern PT1 indicating that the person is sitting on the train.

Further, in the examples of FIGS. 5 and 6, in the environment D2, the environment score of the “standing state” is 10, and the environment score of the “face orientation in the horizontal direction” is 90. It can be predicted that he will sit and turn his face almost horizontally. Further, since the environmental score of "inside the train" is 0, the environmental score of "on the railroad track" is 0, and the environmental score of "sound in the train" is 10, it can be seen that the user U is not on the train. Although not shown here, in the environment D2, it can be confirmed that the user U is on the road based on the environment score of the place of residence. Further, since the environment score of "moving" is 100, it can be seen that the user U is moving with a constant velocity or acceleration. In addition, the environmental score of "bright" is 100, which indicates that it is a bright outdoor environment. Further, the "appropriate amount of infrared rays and ultraviolet rays" is 80, and it can be seen that there is a slight influence of ultraviolet rays and the like. Further, the environmental scores of "suitable temperature" and "suitable humidity" are 100, which can be said to be safe. In addition, the environmental score of "there is a dangerous substance" is 10 in terms of images and 20 in terms of sound, so this is also considered safe. That is, in the environment D2, it is possible to estimate from each environment score that the user U is moving on the sidewalk on foot, is bright outdoors, and no dangerous substance is recognized, and the environment pattern of the environment D2 is. , It is said to be the environmental pattern PT2 indicating that the person is walking on the sidewalk.

Further, in the examples of FIGS. 5 and 6, in the environment D3, the environment score of the “standing state” is 0, and the environment score of the “face orientation in the horizontal direction” is 90. It can be predicted that he will sit and turn his face almost horizontally. Further, since the environmental score of "inside the train" is 5, the environmental score of "on the railroad track" is 0, and the environmental score of "sound in the train" is 5, it can be seen that the user U is not on the train. Although not shown here, in the environment D3, it can be confirmed that the user U is on the road based on the environment score of the place of residence. Further, since the environment score of "moving" is 100, it can be seen that the user U is moving with a constant velocity or acceleration. Further, the environment score of "bright" is 10, which indicates that the environment is dark. Further, the "appropriate amount of infrared rays and ultraviolet rays" is 100, which shows that it is safe. In addition, the environmental score of "suitable temperature" is 75, which can be said to be hotter or colder than the standard. Further, since the environmental score of "there is a dangerous substance" is 90 in the image and 80 in the sound, it can be seen that something is making a sound and approaching. Further, although not shown, the object can be determined from the sound and the image, and here it can be determined that the car is approaching from the front and the sound is the engine sound of the car. That is, in the environment D3, it is possible to estimate from each environment score that the user U is moving on the sidewalk on foot, is dark outdoors, and the vehicle is approaching as a dangerous object, and the environment of the environment D3. The pattern is the environmental pattern PT3, which indicates walking on a dark sidewalk.

Further, in the examples of FIGS. 5 and 6, in the environment D4, the environment score of the "standing state" is 0, and the environment score of the "face orientation in the horizontal direction" is 90. It can be predicted that he will sit and turn his face almost horizontally. Further, since the environmental score of "inside the train" is 20, the environmental score of "on the railroad track" is 0, and the environmental score of "sound in the train" is 5, it can be seen that the user U is not on the train. Although not shown here, in the environment D3, it can be confirmed that the user U is in the shopping center based on the environment score of the place of residence. Further, since the environment score of "moving" is 80, it can be seen that the user U is moving slowly. In addition, the environmental score of "bright" is 70, and it can be expected that the environment score is relatively bright but as bright as indoor lighting. Further, the "appropriate amount of infrared rays and ultraviolet rays" is 100, which shows that it is safe. Further, the environmental score of "suitable temperature" is 100, which is comfortable, but the environmental score of "suitable humidity" is 90, so it cannot be said that it is comfortable. In addition, the environmental score of "there is a dangerous substance" is 10 in terms of images and 20 in terms of sound, so this is also considered safe. That is, in the environment D4, it is possible to estimate from each environment score that the user U is moving in the shopping center on foot, the surrounding area is relatively bright, and there are no dangerous substances, and the environment pattern of the environment D4 is. , The environmental pattern PT4 indicating that the person is shopping.

(Target device and standard output specification settings)
After selecting the environment pattern, the information providing device 10 selects the target device to be operated from the output units 26 based on the environment pattern by the output selection unit 48 and the output specification determination unit 50, as shown in FIG. The reference output specification is set (step S26).

(Setting of target device)
As described above, the target device is a device that is operated in the output unit 26, and in the present embodiment, the output selection unit 48 and the display unit 26A are based on environmental information, more preferably based on an environmental pattern. The target device is selected from the voice output unit 26B and the sensory stimulation output unit 26C. Since the environment pattern is information indicating the environment of the current user U, by selecting the target device based on the environment pattern, it is possible to select an appropriate sensory stimulus according to the environment of the current user U.

For example, the output selection unit 48 determines whether it is highly necessary for the user U to visually recognize the surrounding environment based on the environment information, and determines whether the display unit 26A is the target device based on the determination result. good. In this case, for example, the output specification determination unit 50 selects the display unit 26A as the target device when the necessity of visually recognizing the surrounding environment is lower than the predetermined standard, and the display unit 50 exceeds the predetermined standard. 26A is not the target device. It may be arbitrarily determined whether or not the user U needs to visually recognize the surrounding environment, but for example, when the user U is moving or there is a dangerous object, it is set to be equal to or higher than a predetermined standard. You may judge that it will be.

Further, for example, the output selection unit 48 determines whether the user U has a high need to hear surrounding sounds based on the environmental information, and determines whether the voice output unit 26B is the target device based on the determination result. It's okay. In this case, for example, the output specification determination unit 50 selects the audio output unit 26B as the target device when the necessity of listening to the surrounding sound is lower than the predetermined standard, and outputs the audio when the necessity exceeds the predetermined standard. Part 26B is not the target device. It may be arbitrarily determined whether the user U has a high need to hear the surrounding sounds, but for example, when the user U is moving or there is a dangerous object, the standard is exceeded. You may judge.

Further, for example, the output selection unit 48 may determine whether the user U may receive the tactile stimulus based on the environmental information, and may determine whether the sensory stimulus output unit 26C is the target device based on the determination result. .. In this case, for example, when the output specification determination unit 50 determines that the tactile stimulus may be received, the sensory stimulus output unit 26C is selected as the target device, and when it is determined that the tactile stimulus is not acceptable, the sensory stimulus is stimulated. The output unit 26C is not the target device. It may be arbitrarily determined whether the user U can receive the tactile stimulus, but for example, when the user U is moving or there is a dangerous object, it is determined that the user U is not allowed to receive the tactile stimulus. You can do it.

The example of the method of selecting the target device by the output selection unit 48 has been described above. More specifically, the output selection unit 48 has a relationship between the environment pattern and the target device, for example, as shown in FIG. 8 described later. It is preferable to select the target device based on the table showing.

(Setting of standard output specifications)
Further, the output specification determination unit 50 determines the reference output specification, which is the reference output specification, based on the environmental information, more preferably based on the environmental pattern. The output specification is an index showing how the stimulus output by the output unit 26 is output. For example, the output specification of the display unit 26A indicates how to display the content image PS to be output, and can be rephrased as the display specification. Examples of the output specifications of the display unit 26A include the size (area) of the content image PS, the transparency of the content image PS, and the display content (content) of the content image PS. The size of the content image PS refers to the area occupied by the content image PS in the screen of the display unit 26A. The transparency of the content image PS refers to the degree of transparency of the content image PS. The higher the transparency of the content image PS, the more light incident on the user U's eyes as the background image PA is transmitted through the content image PS, and the background image PA superimposed on the content image PS is visually recognized more clearly. Will be done. In this way, it can be said that the output specification determination unit 50 determines the size, transparency, and display content of the content image PS as the output specifications of the display unit 26A based on the environment pattern. However, the output specifications of the display unit 26A are not limited to all of the size, transparency, and display content of the content image PS. For example, the output specification of the display unit 26A may be at least one of the size, transparency, and display content of the content image PS, or may be another.

For example, the output specification determination unit 50 determines whether it is highly necessary for the user U to visually recognize the surrounding environment based on the environment information, and based on the determination result, determines the output specification (reference output specification) of the display unit 26A. You may decide. In this case, the output specification determination unit 50 determines the output specification (reference output specification) of the display unit 26A so that the higher the necessity of visually recognizing the surrounding environment, the higher the visibility of the environment image PM. The visibility here refers to the ease of viewing the environment image PM. For example, the output specification determination unit 50 may reduce the size of the content image PS or increase the transparency of the content image PS as the necessity of visually recognizing the surrounding environment increases. The restrictions on the display contents may be increased, or these may be combined. In order to increase the limitation on the display content of the content image PS, for example, the distribution image may be excluded from the display content and the display content may be at least one of the navigation image and the notification image. Further, it may be arbitrarily determined whether or not the user U needs to visually recognize the surrounding environment, and examples thereof include the case where the user U is moving or there is a dangerous object.

FIG. 7 is a schematic diagram illustrating an example of the level of the output specification of the content image. In the present embodiment, the output specification determination unit 50 may classify the output specifications of the content image PS into levels and select the level of the output specifications based on the environmental information. In this case, the output specifications of the content image PS are set so that the visibility of the environment image PM is different for each level. In the present embodiment, each level of the output specification is set so that the higher the level, the stronger the output stimulus and the lower the visibility of the environmental image PM. Therefore, the output specification determination unit 50 sets the level of the output specification higher as the necessity of visually recognizing the surrounding environment is lower. In the example of FIG. 7, at level 0, the content image PS is not displayed and only the environment image PM is visually recognized, so that the visibility of the environment image PM is the highest.

As shown in FIG. 7, at level 1, the content image PS is displayed, but the display content of the content image PS is limited. Here, the distribution image is excluded from the display content, and the display content is at least one of the navigation image and the notification image. Further, at level 1, the size of the content image PS is set small. At level 1, the content image PS is superimposed and displayed on the environment image PM only when it is necessary to display the navigation image and the notification image. Therefore, the visibility of the environment image PM at level 1 is lower than level 0 because the content image PS is displayed, but it is higher because the display content of the content image PS is limited.

As shown in FIG. 7, at level 2, the display content of the content image PS is not limited, but the size of the content image PS is limited and the size of the content image PS is set small. The visibility of the environment image PM at level 2 is lower than that at level 1 because the display content is not limited.

As shown in FIG. 7, at level 3, the display content and size of the content image PS are not limited, and for example, the content image PS is displayed on the entire screen of the display unit 26A. However, at level 3, the transparency of the content image PS is limited, and the transparency is set high. Therefore, at level 3, the translucent content image PS and the environment image PM superimposed on the content image PS are visually recognized. The visibility of the environment image PM at level 3 is lower than that of level 2 because the size of the content image PS is not limited.

As shown in FIG. 7, at level 4, the display content, size, and transparency of the content image PS are not limited, and for example, the content image PS has zero transparency on the entire screen of the display unit 26A. It is displayed. At level 4, since the transparency of the content image PS is zero (opaque), the environment image PM is not visually recognized, and only the content image PS is visually recognized. Therefore, the visibility of the environmental image PM at level 4 is the lowest. At level 4, for example, an image that falls within the visual field range of the user U may be displayed as an environment image PM in a part of the screen of the display unit 26A.

Although the output specifications of the display unit 26A have been described above, the output specification determination unit 50 also determines the output specifications of the voice output unit 26B and the sensory stimulation output unit 26C. Examples of the output specifications (voice specifications) of the voice output unit 26B include volume, presence / absence and degree of sound. Acoustic refers to special effects such as surround and three-dimensional sound fields. The louder the volume and the louder the degree of sound, the stronger the degree of auditory stimulation to the user U can be. For example, the output specification determination unit 50 determines whether it is highly necessary for the user U to hear surrounding sounds based on the environmental information, and based on the determination result, determines the output specification (reference output specification) of the audio output unit 26B. You may decide. In this case, the output specification determination unit 50 has an output specification (reference output specification) of the audio output unit 26B so that the lower the need to hear the surrounding sound, the louder the volume and the louder the degree of sound. To decide. It may be arbitrarily determined whether or not the user U has a high need to hear the surrounding sounds, and examples thereof include the case where the user U is moving or there is a dangerous object. The output specification determination unit 50 may set the level of the output specification of the audio output unit 26B in the same manner as the output specification of the display unit 26A.

The output specifications of the sensory stimulus output unit 26C include the strength of the tactile stimulus and the frequency of outputting the tactile stimulus. The higher the intensity and frequency of the tactile stimulus, the stronger the degree of the tactile stimulus to the user U can be. For example, the output specification determination unit 50 determines whether the user U is in a state suitable for receiving a tactile stimulus based on the environmental information, and based on the determination result, the output specification (reference output specification) of the sensory stimulus output unit 26C. ) May be determined. In this case, the output specification determination unit 50 outputs the sensory stimulus output unit 26C so that the strength of the tactile stimulus increases and the frequency of the tactile stimulus increases as the output specification determination unit 50 is suitable for receiving the tactile stimulus. Determine the specifications (reference output specifications). It may be arbitrarily determined whether the user U is in a state suitable for receiving the tactile stimulus, and examples thereof include the case where the user U is moving or there is a dangerous object. The output specification determination unit 50 may set the level of the output specification of the sensory stimulation output unit 26C in the same manner as the output specification of the display unit 26A.

(Specific example of setting the target device and standard output specifications)
It is more preferable that the output selection unit 48 and the output specification determination unit 50 determine the target device and the reference output specification based on the relationship between the environment pattern and the target device and the reference output specification. FIG. 8 is a table showing the relationship between the environmental pattern, the target device, and the reference output specifications. The output selection unit 48 and the output specification determination unit 50 determine the target device and the reference output specification based on the relational information indicating the relationship between the environment pattern and the target device and the reference output specification. The relational information is information (table) in which the environment pattern, the target device, and the reference output specification are stored in association with each other, and is stored in, for example, the specification setting database 30C. In the related information, reference output specifications are set for each type of the output unit 26, that is, here, for each of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C. The output selection unit 48 and the output specification determination unit 50 determine the target device and the reference output specification based on this related information and the environment pattern set by the environment identification unit 44. Specifically, the output selection unit 48 and the output specification determination unit 50 read out the relational information, and from the relational information, select the target device and the reference output specification associated with the environment pattern set by the environment identification unit 44. Select to determine the target device and reference output specifications.

In the example of FIG. 8, for the environmental pattern PT1 that is supposed to be sitting in the train, the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C are all targeted devices, and their reference outputs are used. The specification level is assigned to 4. The higher the level, the higher the output stimulus. In addition, the environment pattern PT2, which is said to be walking on the sidewalk, is almost safe and comfortable, but since it is considered that forward attention is required because the person is walking, the display unit 26A, the audio output unit 26B, and the like. And all of the sensory stimulus output unit 26C are targeted devices, and the level of their reference output specifications is assigned to 3. In addition, for the environmental pattern PT3, which is said to be walking on a dark sidewalk, it cannot be said that the situation is safe, and it is necessary to look ahead and hear the outside sound well. And the sensory stimulus output unit 26C is the target device, and the levels of the reference output specifications of the display unit 26A, the voice output unit 26B, and the sensory stimulus output unit 26C are assigned to 0, 2, and 2, respectively. In addition, although it is almost safe for the environmental pattern PT4 that is said to be shopping, it is assumed that it is not necessary to provide distracting information only to the shopping center, so the display unit 26A and the audio output unit All of 26B and the sensory stimulus output unit 26C are targeted devices, and the level of their reference output specifications is assigned to 2. However, the allocation of the target device and the reference output specification for each environment pattern in FIG. 8 is an example and may be set as appropriate.

As described above, in the present embodiment, the information providing device 10 sets the target device and the reference output specification based on the relationship between the environment pattern and the target device and the reference output specification set in advance. However, the setting method of the target device and the reference output specification is not limited to this, and the information providing device 10 sets the target device and the reference output specification by an arbitrary method based on the environmental information detected by the environment sensor 20. good. Further, the information providing device 10 is not limited to selecting both the target device and the reference output specification based on the environmental information, and may select at least one of the target device and the reference output specification.

(Acquisition of biometric information)
Further, as shown in FIG. 4, the information providing device 10 acquires the biometric information of the user U detected by the biometric sensor 22 by the biometric information acquisition unit 42 (step S28). The biological information acquisition unit 42 acquires the pulse wave information of the user U from the pulse wave sensor 22A, and acquires the brain wave information of the user U from the brain wave sensor 22B. FIG. 9 is a graph showing an example of a pulse wave. As shown in FIG. 9, the pulse wave is a waveform in which a peak called R wave WR appears at predetermined time intervals. The heart is dominated by the autonomic nervous system, and the pulse rate moves by generating electrical signals at the cellular level that trigger the movement of the heart. Normally, pulse rate increases when sympathetic excitement secretes adrenaline and decreases when parasympathetic excitement secretes acetylcholine. According to Nobuyuki Ueda, "Evaluation of Diabetic Independent Neuropathy Using Power Spectrum Analysis of ECG RR Spacing" (Diabetes 35 (1): 17-23, 1992), the function of the autonomic nerve is shown in FIG. It is said that it can be found by examining the fluctuation of the RR interval in the time waveform of the pulse wave as shown in the example. The RR sensation is the interval between R wave WRs that are continuous in time series. At the cellular level, electrocardiography is a repetition of depolarization / action potential and repolarization / resting potential, and by detecting this electrical activity from the body surface, electrocardiogram can be detected. It should be noted that the pulse wave travels at a very high speed and is transmitted throughout the body almost at the same time as the heart strikes, so it can be said that the heartbeat is also synchronized with the pulse wave. Since the pulse wave hit by the heart and the R wave of the electrocardiogram are synchronized, the RR interval of the pulse wave can be considered to be equivalent to the RR interval of the electrocardiogram. The fluctuation of the pulse wave RR interval can be said to be a time differential value, so by calculating the differential value and detecting the magnitude of the fluctuation, the activity of the autonomic nerves of the living body is almost irrelevant to the wearer's intention. It is possible to predict to some extent the degree of calming and the degree of calming, that is, frustration due to mental disorder, unpleasant feelings due to a crowded train, and stress that occurs in a relatively short time.

On the other hand, EEG is a wave such as α wave and β wave, and the activity of the whole brain is increased or decreased by detecting the basic rhythm (background EEG) activity that appears in the whole brain and detecting its amplitude. You can predict to some extent that you are doing it. For example, from the degree of activity of the brain in the prefrontal cortex, the degree of attention such as how much interest you have in the visually stimulated object can be known.

(Specification of user status and calculation of output specification correction degree)
As shown in FIG. 4, after acquiring the biometric information, the information providing device 10 identifies the user state indicating the mental state of the user U based on the biometric information of the user U by the user state specifying unit 46, and the user state. The output specification correction degree is calculated based on (step S30). The output specification correction degree is a value for correcting the reference output specification set by the output specification determination unit 50, and the final output specification is determined based on the reference output specification and the output specification correction degree.

FIG. 10 is a table showing an example of the relationship between the user state and the output specification correction degree. In the present embodiment, the user state specifying unit 46 specifies the brain activity of the user U as the user state based on the brain wave information of the user U. The user state specifying unit 46 may specify the brain activity by any method based on the brain wave information of the user U, and for example, the brain activity is from a specific region of the frequency with respect to the waveforms of the α wave and the β wave. You may specify the degree. In this case, for example, the user state specifying unit 46 performs a fast Fourier transform on the time waveform of the brain wave to calculate the power spectrum amount of the high frequency portion (for example, 10 Hz to 11.75 Hz) of the α wave. When the power spectrum amount of the high frequency part of the α wave is large, it can be expected that the user state identification unit 46 is very concentrated while relaxing. Therefore, the larger the power spectrum amount of the high frequency part of the α wave, the more the brain activity of the user state specifying unit 46. Judge that the degree is high. The user state specifying unit 46 sets the brain activity when the power spectrum amount of the high frequency part of the α wave is within a predetermined numerical range as VA3, and sets the power spectrum amount of the high frequency part of the α wave as the brain activity degree VA3. The brain activity in the case of a predetermined numerical range lower than the numerical range of the case is VA2, and the power spectral amount of the high frequency portion of the α wave is in the predetermined numerical range lower than the numerical range of the brain activity VA2. The brain activity in a certain case is defined as VA1. Here, it is assumed that the brain activity is higher in the order of VA1, VA2, and VA3. The larger the power spectrum amount of the high frequency component of the β wave (for example, 18 Hz to 29.75 Hz), the higher the possibility of psychological "warning" and "upset". Therefore, the power spectrum of the high frequency component of the β wave The amount may also be used to specify brain activity.

The user state specifying unit 46 determines the output specification correction degree based on the brain activity of the user U. In the present embodiment, the output specification correction degree is determined based on the output specification correction degree relation information indicating the relationship between the user state (brain activity in this example) and the output specification correction degree. The output specification correction degree-related information is information (table) in which the user state and the output specification correction degree are stored in association with each other, and is stored in, for example, the specification setting database 30C. In the output specification correction degree related information, the output specification correction degree is set for each type of the output unit 26, that is, here, for each of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C. The user state specifying unit 46 determines the output specification correction degree based on the output specification correction degree related information and the specified user state. Specifically, the user state specifying unit 46 reads out the output specification correction degree related information, and from the output specification correction degree related information, outputs the output specification correction degree associated with the set brain activity of the user U. Select to determine the output specification correction degree. In the example of FIG. 10, the output specification correction degree of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C is set to -1 with respect to the brain activity degree VA3, respectively, with respect to the brain activity degree VA2. , The output specification correction degree of the display unit 26A, the voice output unit 26B, and the sensory stimulus output unit 26C is set to 0, respectively, and the display unit 26A, the voice output unit 26B, and the sensory stimulus output are set with respect to the brain activity VA1. The output specification correction degree of the unit 26C is set to 1, respectively. The output specification correction degree here is set to a value that increases the output specification as the value increases. That is, the user state specifying unit 46 sets the output specification correction degree so that the lower the brain activity, the higher the output specification. It should be noted that increasing the output specifications here means strengthening the sensory stimulus, and the same applies thereafter. The value of the output specification correction degree in FIG. 10 is an example and may be set as appropriate.

Further, the user state specifying unit 46 specifies the mental stability of the user U as the user state based on the pulse wave information of the user U. In the present embodiment, the user state specifying unit 46 calculates the fluctuation value of the interval length between continuous R waves WH in time series from the brain wave information of the user U, that is, the differential value of the RR interval, and R -The brain activity of the user U is specified based on the differential value of the R interval. The user state specifying unit 46 specifies that the smaller the differential value of the RR interval, that is, the more the interval length between the R waves WH does not fluctuate, the higher the mental stability of the user U is. In the example of FIG. 10, the user state specifying unit 46 classifies the mental stability into one of VB3, VB2, and VB1 from the pulse wave information of the user U. The user state specifying unit 46 sets the stability of the mind when the differential value of the RR interval is within a predetermined numerical range as VB3, and sets the differential value of the RR interval as the stability of the mind VB3. When the stability of the heart is VB2 in the case of a predetermined numerical range higher than the numerical range, and the differential value of the RR interval is a predetermined numerical range lower than the numerical range when the stability of the heart is VB2. Let VB1 be the stability of the mind. It is assumed that the stability of the mind is higher in the order of VB1, VB2, and VB3.

The user state specifying unit 46 determines the output specification correction degree based on the output specification correction degree related information and the specified mental stability. Specifically, the user state specifying unit 46 reads out the output specification correction degree related information, and from the output specification correction degree related information, the output specification correction degree associated with the set mental stability of the user U. Select to determine the output specification correction degree. In the example of FIG. 10, the output specification correction degree of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C is set to 1 with respect to the mental stability VB3, respectively, with respect to the mental stability VB2. The output specification correction degree of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C is set to 0, respectively, and the display unit 26A, the voice output unit 26B, and the sensation are set with respect to the mental stability VB1. The output specification correction degree of the stimulus output unit 26C is set to -1, respectively. That is, the user state specifying unit 46 sets the output specification correction degree so that the higher the stability of the mind, the higher the output specification (sensory stimulation). The value of the output specification correction degree in FIG. 10 is an example and may be set as appropriate.

In this way, the user state specifying unit 46 sets the output specification correction degree based on the preset relationship between the user state and the output specification correction degree. However, the method of setting the output specification correction degree is not limited to this, and the information providing device 10 may set the output specification correction degree by any method based on the biological information detected by the biological sensor 22. Further, the information providing device 10 calculates the output specification correction degree using both the brain activity specified from the electroencephalogram and the mental stability specified from the pulse wave, but is not limited thereto. For example, the information providing device 10 may calculate the output specification correction degree by using either the brain activity specified from the electroencephalogram or the mental stability specified from the pulse wave. Further, the information providing device 10 handles the biometric information as a numerical value, and by estimating the user state based on the biometric information, it is possible to take into account the error of the biometric information and the like, and the psychology of the user U can be more accurately performed. The state can be estimated. In other words, it can be said that the information providing device 10 can accurately estimate the psychological state of the user U by classifying the biometric information and the user state based on the biometric information into any of three or more degrees. However, the information providing device 10 is not limited to classifying the biometric information and the user state based on the biometric information into three or more degrees, and treats the information as, for example, information indicating either Yes or No. You may.

(Generation of output restriction necessity information)
Further, as shown in FIG. 4, the information providing device 10 generates output restriction necessity information based on the biometric information of the user U by the user state specifying unit 46 (step S32). FIG. 11 is a table showing an example of output restriction necessity information. The output restriction necessity information is information indicating whether or not the output restriction of the output unit 26 is necessary, and can be said to be information indicating whether or not the operation of the output unit 26 is permitted. The output restriction necessity information is generated for each output unit 26, that is, for each of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C. In other words, the user state specifying unit 46 provides output restriction necessity information indicating whether or not to permit the operation of each of the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C based on the biological information. Generate. More specifically, the user state specifying unit 46 generates output restriction necessity information based on both biometric information and environmental information. The user state specifying unit 46 generates output restriction necessity information based on the user state set based on the biological information and the environmental score calculated based on the environmental information. In the example of FIG. 11, the user state specifying unit 46 generates output restriction necessity information based on the brain activity as the user state and the location score for the subcategory of being on the railroad track as the environmental score. In the example of FIG. 11, the user state specifying unit 46 has a location score of 100 for the subcategory of being on the railroad track, and the display unit 26A satisfies the first condition that the brain activity is VA3 and VA2. Generates output restriction necessity information that disallows the use of. The first condition is not limited to the case where the location score for the subcategory of being on the railroad track is 100 and the brain activity is VA3 and VA2. For example, the position of the information providing device 10 is a predetermined area. It may be in the case where the brain activity is equal to or less than a predetermined brain activity threshold. The predetermined area here may be, for example, on a railroad track or a roadway.

Further, in the example of FIG. 11, the user state specifying unit 46 generates output restriction necessity information based on the brain activity as the user state and the motion score for the subcategory of moving as the environmental score. .. In the example of FIG. 11, the user state specifying unit 46 has a motion score of 0 for the subcategory of being moving, and the display unit 26A satisfies the first condition that the brain activity is VA3 and VA2. Generates output restriction necessity information that disallows use. The second condition is not limited to the case where the movement score for the subcategory of moving is 0 and the brain activity is VA3 or VA2, for example, per unit time of the position of the information providing device 10. The change amount may be equal to or less than a predetermined change amount threshold value, and the brain activity may be equal to or less than a predetermined brain activity threshold value.

As described above, the user state specifying unit 46 satisfies the case where the biometric information and the environmental information satisfy a specific relationship, and here, when the user state and the environmental score satisfy at least one of the first condition and the second condition. Generates output restriction necessity information that disallows the use of the display unit 26A. On the other hand, when the user state and the environment score do not satisfy the first condition and the second condition, the user state specifying unit 46 does not generate the output restriction necessity information disallowing the use of the display unit 26A. , Generates output restriction necessity information that permits the use of the display unit 26A. However, the generation of output restriction necessity information is not an essential process.

(Acquisition of content image)
Further, as shown in FIG. 4, the information providing device 10 acquires the image data of the content image PS by the content image acquisition unit 52 (step S34). The image data of the content image PS is image data for displaying the content (display content) of the content image. The content image acquisition unit 52 acquires image data of the content image from an external device via the content image reception unit 28A.

The content image acquisition unit 52 may acquire image data of the content image of the content (display content) according to the position (earth coordinates) of the information providing device 10 (user U). The position of the information providing device 10 is specified by the GNSS receiver 20C. For example, when the user U is located within a predetermined range with respect to a certain position, the content image acquisition unit 52 receives the content related to the position. In principle, the content image PS can be displayed and controlled at the will of the user U, but if the display is set to be possible, it is convenient because it is not known when, where, and at what timing, but it can be annoying. obtain. Therefore, in the specification setting database 30C, information indicating whether or not the content image PS set by the user U can be displayed, display specifications, and the like may be recorded. The content image acquisition unit 52 reads this information from the specification setting database 30C, and controls the acquisition of the content image PS based on this information. Further, the location information and the specification setting database 30C may describe the same information on a site on the Internet, and the content image acquisition unit 52 may control the acquisition of the content image PS while checking the contents. .. The step S34 for acquiring the image data of the content image PS is not limited to being executed before the step S36 described later, and may be executed at any timing before the step S38 described later.

The content image acquisition unit 52 may acquire audio data and tactile stimulus data related to the content image PS as well as the image data of the content image PS. The audio output unit 26B outputs audio data related to the content image PS as audio content (audio content), and the sensory stimulus output unit 26C outputs tactile stimulus data related to the content image PS to tactile stimulus content (tactile sensation). Output as the content of the stimulus).

(Setting of output specifications)
Next, as shown in FIG. 4, the information providing device 10 determines the output specifications by the output specification determining unit 50 based on the reference output specifications and the output specification correction degree (step S36). The output specification determination unit 50 determines the reference output specification set based on the environmental information as the final output specification for the output unit 26 by correcting the reference output specification set based on the biological information with the output specification correction degree. The formula for correcting the reference output specification with the output specification correction degree may be arbitrary.

As described above, the information providing device 10 corrects the reference output specification set based on the environmental information with the output specification correction degree set based on the biological information, and determines the final output specification. .. However, the information providing device 10 is not limited to determining the output specifications by correcting the reference output specifications with the output specification correction degree, and uses at least one of the environmental information and the biometric information to adjust the output specifications by an arbitrary method. It may be something to decide. That is, the information providing device 10 may determine the output specifications by an arbitrary method based on the environmental information and the biometric information, or determine the output specifications by an arbitrary method based on either the environmental information or the biometric information. You may decide. For example, the information providing device 10 may determine the output specifications by using the method for determining the above-mentioned reference output specifications based on the environmental information among the environmental information and the biological information. Further, for example, the information providing device 10 may determine the output specifications by using the above-mentioned method of determining the output specification correction degree based on the biological information among the environmental information and the biological information.

When the output restriction necessity information disallowing the use of the output unit 26 is generated in step S32, the output selection unit 48 is based not only on the environment score but also on the output restriction necessity information. Select the target device. That is, even the output unit 26 selected as the target device based on the environmental score in step S26 is excluded from the target device if the specification is disapproved in the output restriction necessity information. In other words, the output selection unit 48 selects the target device based on the output restriction necessity information and the environmental information. Furthermore, since the output restriction necessity information is set based on the biological information, it can be said that the target device is set based on the biological information and the environmental information. However, the output selection unit 48 is not limited to setting the target device based on both the biological information and the environmental information, and may select the target device based on at least one of the biological information and the environmental information.

(Output control)
After setting the target device and the output specifications and acquiring the image data of the content image PS and the like, the information providing device 10 uses the output control unit 54 for the target device based on the output specifications as shown in FIG. Output is performed (step S38). The output control unit 54 does not operate the output unit 26 that is not the target device.

For example, when the display unit 26A is the target device, the output control unit 54 uses the content image data acquired by the content image acquisition unit 52 so as to comply with the output specifications of the display unit 26A for the display unit 26A. Display the based content image PS. As described above, the output specifications are set based on the environmental information and biological information. Therefore, by displaying the content image PS according to the output specifications, the environment in which the user U is placed and the psychological state of the user U are displayed. The content image PS can be displayed in an appropriate manner according to the above.

When the audio output unit 26B is the target device, the output control unit 54 outputs the audio acquired by the content image acquisition unit 52 so as to follow the output specifications of the audio output unit 26B for the audio output unit 26B. Output audio based on data. Even in this case, for example, the higher the brain activity of the user U or the lower the stability of the mind of the user U, the weaker the auditory stimulus, so that the user U can concentrate on other things or have a margin in the mind. If there is no such thing, it is possible to reduce the risk of being bothered by voice. On the other hand, the lower the brain activity of the user U and the higher the stability of the mind of the user U, the stronger the auditory stimulus, so that information can be appropriately obtained by voice.

When the sensory stimulus output unit 26C is the target device, the output control unit 54 causes the content image acquisition unit 52 to follow the output specifications of the sensory stimulus output unit 26C with respect to the sensory stimulus output unit 26C. The tactile stimulus based on the acquired tactile stimulus data is output. Even in this case, for example, the higher the brain activity of the user U or the lower the stability of the mind of the user U, the weaker the tactile stimulus, so that the user U can concentrate on other things or have a margin in the mind. In the absence of, the risk of being bothered by tactile stimuli can be reduced. On the other hand, the lower the brain activity of the user U and the higher the stability of the mind of the user U, the stronger the tactile stimulus, so that information can be appropriately obtained by the tactile stimulus.

Further, when it is determined in step S12 that the danger state is determined and the danger notification content is set, the output control unit 54 causes the target device to notify the danger notification content so as to comply with the set output specifications. ..

As described above, the information providing device 10 according to the present embodiment is appropriate according to the environment in which the user U is placed and the psychological state of the user U by setting the output specifications based on the environmental information and the biological information. It is possible to output sensory stimuli to a certain degree. Further, the information providing device 10 selects an appropriate sensory stimulus according to the environment in which the user U is placed and the psychological state of the user U by selecting the target device to be operated based on the environmental information and the biological information. can. However, the information providing device 10 is not limited to using both environmental information and biological information, and for example, only one of them may be used. Therefore, the information providing device 10 may, for example, select a target device and set an output specification based on environmental information, or select a target device and set an output specification based on biometric information. May be good.

(effect)
As described above, the information providing device 10 according to one aspect of the present embodiment is a device that provides information to the user U, and includes an output unit 26, an environment sensor 20, an output specification determination unit 50, and an output. It includes a control unit 54. The output unit 26 includes a display unit 26A that outputs a visual stimulus, a voice output unit 26B that outputs an auditory stimulus, and a sensory stimulus output unit 26C that outputs a sensory stimulus different from the visual and auditory stimuli. The environment sensor 20 detects environmental information around the information providing device 10. The output specification determination unit 50 determines the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus, that is, the output specifications of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C, based on the environmental information. .. The output control unit 54 causes the output unit 26 to output visual stimuli, auditory stimuli, and sensory stimuli based on the output specifications. The information providing device 10 sets the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus based on the environmental information, so that the visual stimulus, the auditory stimulus, and the sensory stimulus are set according to the environment in which the user U is placed. The stimulus can be balanced and output. Therefore, according to the information providing device 10, information can be appropriately provided to the user U.

Further, the information providing device 10 according to one aspect of the present embodiment includes a plurality of environment sensors that detect different types of environmental information from each other, and an environment specifying unit 44. The environment specifying unit 44 identifies an environment pattern that comprehensively indicates the current environment of the user U based on different types of environment information. The output specification determination unit 50 determines the output specifications based on the environment pattern. The information providing device 10 sets the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus based on the environmental pattern specified from a plurality of types of environmental information, so that the information providing device 10 can be set according to the environment in which the user U is placed. Information can be provided more appropriately.

Further, in the information providing device 10 according to one embodiment of the present embodiment, the output specification determination unit 50 determines the size of the image displayed by the display unit 26A and the transparency of the image displayed by the display unit 26A as the output specifications of the visual stimulus. And the content (display content) of the image displayed by the display unit 26A, at least one is determined. The information providing device 10 can more appropriately provide visual information by determining these as the output specifications of the visual stimulus.

Further, in the information providing device 10 according to one aspect of the present embodiment, the output specification determination unit 50 determines at least one of the volume of the voice output by the voice output unit 26B and the sound as the output specifications of the auditory stimulus. decide. The information providing device 10 can more appropriately provide auditory information by determining these as the output specifications of the auditory stimulus.

Further, in the information providing device 10 according to one embodiment of the present embodiment, the sensory stimulus output unit 26C outputs a tactile stimulus as a sensory stimulus, and the output specification determination unit 50 outputs a auditory stimulus as an output specification. At least one of the strength of the tactile stimulus output by the sensory stimulus output unit 26C and the frequency of outputting the tactile stimulus is determined. The information providing device 10 can more appropriately provide tactile information by determining these as output specifications of the tactile stimulus.

Further, the information providing device 10 according to one aspect of the present embodiment is a device that provides information to the user U, and includes an output unit 26, a biosensor 22, an output specification determination unit 50, and an output control unit 54. including. The output unit 26 includes a display unit 26A that outputs a visual stimulus, a voice output unit 26B that outputs an auditory stimulus, and a sensory stimulus output unit 26C that outputs a sensory stimulus different from the visual and auditory stimuli. The biosensor 22 detects the biometric information of the user U. The output specification determination unit 50 determines the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus, that is, the output specifications of the display unit 26A, the audio output unit 26B, and the sensory stimulus output unit 26C, based on the biological information. .. The output control unit 54 causes the output unit 26 to output visual stimuli, auditory stimuli, and sensory stimuli based on the output specifications. The information providing device 10 sets the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus based on the environmental information, so that the visual stimulus, the auditory stimulus, and the sensory stimulus are balanced according to the psychological state of the user U. Can be output appropriately. Therefore, according to the information providing device 10, information can be appropriately provided to the user U.

Further, in one embodiment of the present embodiment, the biological information includes information on the autonomic nerve of the user U, and the output specification determination unit 50 determines the output specification based on the information on the autonomic nerve of the user U. The information providing device 10 provides information more appropriately according to the psychological state of the user U by setting the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus based on the information about the autonomic nerve of the user U. can.

Further, the information providing device 10 according to one aspect of the present embodiment is a device that provides information to the user U, and includes an output unit 26, an environment sensor 20, an output selection unit 48, and an output control unit 54. include. The output unit 26 includes a display unit 26A that outputs a visual stimulus, a voice output unit 26B that outputs an auditory stimulus, and a sensory stimulus output unit 26C that outputs a sensory stimulus different from the visual and auditory stimuli. The environment sensor 20 detects environmental information around the information providing device 10. The output selection unit 48 selects the target device to be used from the display unit 26A, the voice output unit 26B, and the sensory stimulation output unit 26C based on the environmental information. The output control unit 54 controls the target device. By selecting the target device based on the environmental information, the information providing device 10 appropriately determines which stimulus, the visual stimulus, the auditory stimulus, or the sensory stimulus, is output according to the environment in which the user U is placed. Can be selected. Therefore, according to the information providing device 10, information can be appropriately provided to the user U according to the environment in which the user U is placed.

Further, the information providing device 10 according to one embodiment of the present embodiment further includes a biological sensor 22 for detecting the biological information of the user, and the output selection unit 48 is a target based on the environmental information and the biological information of the user U. Select a device. The information providing device 10 can select an appropriate sensory stimulus according to the environment in which the user U is placed and the psychological state of the user U by selecting the target device to be operated based on the environmental information and the biological information.

Further, in one embodiment of the present embodiment, the environment sensor 20 detects the position information of the information providing device 10 as the environmental information, and the biosensor 22 detects the brain activity of the user U as the biometric information. The output selection unit 48 has the first condition that the position of the information providing device 10 is within a predetermined area and the brain activity is equal to or less than the brain activity threshold value, and the amount of change in the position of the information providing device 10 per unit time. Is less than or equal to the predetermined change amount threshold value, and the display unit 26A is selected as the target device when at least one of the second condition that the brain activity is equal to or less than the brain activity threshold value is satisfied. On the other hand, if the output selection unit 48 does not satisfy the first condition and the second condition, the output selection unit 48 does not select the display unit 26A as the target device. Since the information providing device 10 determines whether to operate the display unit 26A in this way, for example, when the user U is not moving and is relaxed, or when the user is in a vehicle and is relaxed. For example, the visual stimulus can be appropriately output to the user U.

Although the present embodiments have been described above, the embodiments are not limited by the contents of these embodiments. Further, the above-mentioned components include those that can be easily assumed by those skilled in the art, those that are substantially the same, that is, those in a so-called equal range. Further, the above-mentioned components can be appropriately combined, and the configurations of the respective embodiments can be combined. Further, various omissions, replacements or changes of the components can be made without departing from the gist of the above-described embodiment.

The information providing device, the information providing method, and the program of the present embodiment can be used, for example, for displaying an image.

10 Information provider 20 Environmental sensor 22 Biological sensor 26 Output unit 26A Display unit 26B Audio output unit 26C Sensory stimulus output unit 40 Environmental information acquisition unit 42 Biological information acquisition unit 44 Environmental identification unit 46 User status identification unit 48 Output selection unit 50 Output Specification determination unit 52 Content image acquisition unit 54 Output control unit PM environment image PS content image

Claims

An information providing device that provides information to users.
A display unit that outputs a visual stimulus, an audio output unit that outputs an auditory stimulus, and an output unit that includes a sensory stimulus output unit that outputs a sensory stimulus different from the visual and auditory senses.
An environment sensor that detects environmental information around the information providing device, and
An output selection unit that selects any of the display unit, the voice output unit, and the sensory stimulus output unit based on the environmental information.
including,
Information providing device.
Further including a biosensor for detecting the biometric information of the user.
The information provision according to claim 1, wherein the output selection unit selects one of the display unit, the voice output unit, and the sensory stimulus output unit based on the environmental information and the biometric information of the user. Device.
The environment sensor detects the position information of the information providing device as the environment information, and receives the position information.
The biosensor detects the brain activity of the user as the biometric information, and determines the brain activity of the user.
The output selection unit is
The first condition that the position of the information providing device is within a predetermined area and the brain activity is equal to or less than the predetermined brain activity threshold value, and the amount of change in the position of the information providing device per unit time are predetermined. The display unit is selected when at least one of the second condition that is equal to or less than the change amount threshold and the brain activity is equal to or less than the brain activity threshold is satisfied.
The information providing device according to claim 2, wherein the display unit is not selected when the first condition and the second condition are not satisfied.
The information providing device according to claim 3, wherein the predetermined area is on a railroad track or a roadway.
The information providing device according to any one of claims 1 to 4, wherein the sensory stimulus output unit outputs a tactile stimulus as the sensory stimulus.
An output specification determination unit that determines the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus based on the environmental information.
The aspect according to any one of claims 1 to 5, wherein the output unit further includes an output control unit that outputs the visual stimulus, the auditory stimulus, and the sensory stimulus based on the output specifications. Information providing device.
A biosensor that detects the biometric information of the user and
An output specification determination unit that determines the output specifications of the visual stimulus, the auditory stimulus, and the sensory stimulus based on the biological information.
The invention according to any one of claims 1 to 6, wherein the output unit further includes an output control unit that outputs the visual stimulus, the auditory stimulus, and the sensory stimulus based on the output specifications. Information providing device.
It is an information provision method that provides information to users.
Steps to detect surrounding environmental information and
A step of selecting one of a display unit that outputs a visual stimulus, a voice output unit that outputs an auditory stimulus, and a sensory stimulus output unit that outputs a sensory stimulus different from the visual and auditory senses based on the environmental information.
including,
Information provision method.
Steps to detect surrounding environmental information and
A step of selecting one of a display unit that outputs a visual stimulus, a voice output unit that outputs an auditory stimulus, and a sensory stimulus output unit that outputs a sensory stimulus different from the visual and auditory senses based on the environmental information.
including,
Have the computer perform an information provision method that provides information to the user,
program.