JP2024043867A - Measurement device, program and model generation device - Google Patents

Abstract

[Problem] To provide a measurement device, a program, and a model generation device for accurately measuring the reactivity of a subject. [Solution] The measurement device has an acquisition unit that acquires a reference signal output to a measurement space and a reaction signal that indicates the reaction of the subject to measurement information output to the measurement space in conjunction with the output of the reference signal, and a measurement unit that measures the reactivity of the subject based on the reaction signal and the reference signal. [Selected Figure] Figure 3

Description

This disclosure relates to a measurement device, a program, and a model generation device.

2. Description of the Related Art Diagnostic devices that detect the movement of a subject's line of sight in order to diagnose, for example, cognitive function have been proposed. For example, Patent Document 1 discloses a cognitive dysfunction diagnostic device that easily diagnoses cognitive dysfunction. This cognitive dysfunction diagnosing device detects the subject's viewpoints on a display screen that displays a diagnostic image in time series, and diagnoses the subject's cognitive dysfunction based on the distribution of the viewpoints.

International Publication No. 2019/098173

However, the device in Patent Document 1 had difficulty accurately measuring and recording the movement of the subject's gaze relative to the diagnostic image.

The present disclosure aims to provide a measurement device, a program, and a model generation device that accurately measure the reactivity of a subject.

The measurement device according to the present disclosure includes an acquisition unit that acquires a reference signal output to a measurement space and a reaction signal indicating a reaction of a subject to measurement information output to the measurement space in conjunction with the output of the reference signal. and a measurement unit that measures the reactivity of the subject based on the reaction signal and the reference signal.

A program according to the present disclosure causes a computer to acquire a reference signal output to a measurement space and a reaction signal indicating a reaction of a subject to measurement information output to the measurement space in conjunction with the output of the reference signal. and measuring the reactivity of the subject based on the reaction signal and the reference signal.

A model generation device according to the present disclosure includes a reference signal outputted to a measurement space, a reaction signal indicating a reaction of a plurality of subjects to measurement information outputted to the measurement space in conjunction with the output of the reference signal, and a plurality of an acquisition unit that acquires the judgment result of determining the condition of the subject; and data that associates the reaction signals of the plurality of subjects based on the reference signal, and combines the correlated reaction signals and the judgment result. and a training unit that generates a machine learning model based on the set.

A program according to the present disclosure provides a computer with a reference signal outputted to a measurement space, and a reaction signal indicating the reaction of a plurality of subjects to measurement information outputted to the measurement space in conjunction with the output of the reference signal. a step of acquiring judgment results that have determined the states of multiple subjects; and a step of associating the reaction signals of the plurality of subjects based on a reference signal, and combining data of the correlated reaction signals and the judgment results. and generating a machine learning model based on the set.

According to the present disclosure, it is possible to accurately measure the reactivity of a subject to measurement information.

1 is a diagram showing the configuration of a measurement system including a measurement device according to Embodiment 1 of the present disclosure. FIG. 2 is a diagram illustrating a configuration of a terminal device. FIG. 2 is a diagram showing the configuration of a measuring device. FIG. 13 shows changes in eye response signals and reference signals. 1 is a diagram showing an overview of Embodiment 1. FIG. 4 is a flowchart showing the operation of the first embodiment. FIG. 3 is a diagram illustrating how a subject views a measurement video. It is a figure which shows the screen which aligns a line of sight in a measurement video. FIG. 13 is a diagram showing how scenes change in a measurement video. FIG. 1 illustrates how a machine learning model is used to determine the reactivity of a subject's eye. FIG. 3 is a diagram showing an overview of Embodiment 2; FIG. 3 is a diagram showing the configuration of a detection section according to Embodiment 2. FIG. FIG. 7 is a diagram showing how scenes change in the measurement video of Embodiment 2; FIG. 13 shows changes in hand response signals and reference signals. FIG. 7 is a diagram showing an outline of a modification of the second embodiment. FIG. 1 shows changes in eye and hand response signals and baseline signals. FIG. 13 is a diagram showing an overview of a fourth embodiment. FIG. 7 is a diagram showing the configuration of a model generation device according to a fifth embodiment. 12 is a flowchart showing the operation of Embodiment 5. 7 is a diagram showing how optical signals are output in modified examples of Embodiments 1 to 5. FIG.

The following describes an embodiment of the present disclosure with reference to the attached drawings.

(Embodiment 1)
1 shows a configuration of a measurement system including a measurement device according to a first embodiment of the present disclosure. The measurement system 1 includes a plurality of terminal devices 2, a communication system 3, and a measurement device 4. The terminal devices 2, the communication system 3, and the measurement devices 4 are connected to each other.

The terminal device 2 is placed in the measurement space and detects the subject's reactivity. Here, the measurement space refers to the space in which the subject is positioned and the subject's reaction is measured.

The communication system 3 has multiple BSs (Base Stations) 5, a core network 8, and the Internet 9. The BSs 5 are connected to the core network 8. The core network 8 is connected to the measurement device 4 via the Internet 9. The communication system 3 can be configured, for example, from a fifth generation mobile communication system (5G). However, the communication system according to the present disclosure is not limited to this, and may be other cellular communication systems such as 4G and 6G, wireless communication systems such as WiFi (registered trademark) and wireless LAN (Local Area Network), or fixed telephone lines.

BS5 wirelessly communicates with terminal devices 2 present within its wireless communication area, and transmits data including the detection data transmitted from the terminal devices 2 to the core network 8. Here, the data may include information such as the subject's age and sex, for example.

The core network 8 is a network line that connects the terminal device 2 and the measurement device 4 via the BS 5 and the Internet 9. The core network 8 may, for example, include a program that controls the communication function in the communication system 3, and upon receiving data transmitted from the BS 5, transmit the data to the measurement device 4 via the Internet 9 based on the program.

The measurement device 4 acquires data including the detection data from the terminal device 2 via the communication system 3, and measures the reactivity of the subject, for example the reactivity of the subject's eyes, based on the detection data.
However, the measurement system 1 according to the present disclosure is not limited to the above-mentioned system configuration, and may have a system configuration other than the above.

Figure 2 shows the detailed configuration of terminal device 2.

The terminal device 2 has, as its hardware configuration, a storage unit 10, a processor 11, a user interface (UI) 12, and a communication unit 13, which are interconnected via a bus B.
The terminal device 2 may be realized by a computing device such as a smartphone, a tablet, a personal computer, etc. Programs or instructions for realizing various functions and processes described later in the terminal device 2 may be downloaded from any external device via a network or the like, or may be provided from a removable storage medium such as a flash memory.

The storage unit 10 is realized by one or more non-transitory storage mediums such as random access memory, flash memory, hard disk drive, etc., and stores the programs or instructions together with the installed programs or instructions. Stores files, data, etc. used for execution. The storage unit 10 according to this embodiment may store measurement information, reference signals, and the like.

Here, the measurement information is information used in measurements, tests, etc. provided to the subject, and is output to a measurement space in which the reaction of the subject is measured. The measurement information is, for example, information for making the subject react to a specific region, and may be composed of a measurement video in which the image changes so that the subject's eyes react.
Further, the reference signal may be a signal that is output to the measurement space in conjunction with the measurement information, and indicates the temporal position of the measurement information that is sequentially output, for example. For example, the reference signal may be configured to vary in intensity (amplitude) sequentially, or may be configured to vary in intensity at regular time intervals (eg, every few milliseconds), e.g., to count time. good. Further, the reference signal may be configured such that the frequency changes sequentially. Further, the reference signal may be configured to sequentially switch between ON and OFF.
In one embodiment, the reference signal may consist of a high frequency sound wave signal that is higher than the predetermined frequency. Specifically, the sound wave signal can be composed of a sound with a frequency exceeding the approximately 14 kHz frequency band, which is difficult for the average person to perceive, or a sound with an inaudible frequency band.

The user interface (UI) 12 may include input devices such as a keyboard, mouse, camera, microphone, and wireless controller, output devices such as a display, speaker, headset, and printer, and input/output devices such as a touch panel. and the terminal device 2. For example, the user may operate the terminal device 2 by operating a GUI (Graphical User Interface) displayed on a display or touch panel using a keyboard, a mouse, or the like.

Here, under the control of the processor 11, the display displays measurement information such as a measurement video in the measurement space.
The speaker, under the control of the processor 11, outputs a sound wave signal of a reference signal into the measurement space.

The camera acquires a photographed image obtained by photographing the reaction of the subject to the measurement video, for example, a photographed video containing continuous reaction signals of the subject to the measurement video. In this way, the camera also functions as a detection unit that detects the reaction of the subject. For example, the camera photographs a range including the subject's eyes, and detects the reaction of the subject's eyes to the measurement video, that is, the movement of the viewpoint. Note that the captured image is not limited to a captured video, and may include, for example, a plurality of images captured at predetermined time intervals.
The microphone acquires received sound including the sound wave signal outputted from the speaker into the measurement space as a reference signal.

The processor 11 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuits, etc. that may be configured from one or more processor cores. The processor 11 executes various functions and processes of the terminal device 2, which will be described later, in accordance with programs, instructions, and data such as parameters necessary to execute the programs or instructions stored in the storage unit 10. The processor 11 according to this embodiment implements an output control section 11a and a data processing section 11b.

The output control unit 11a controls the display, speaker, etc. of the UI 12 so as to output the measurement video and the sound wave signal stored in the storage unit 10. At this time, the output control unit 11a controls to output the measurement moving image to the measurement space in conjunction with the output of the sound wave signal. That is, the output control unit 11a causes the UI 12 to output the sound wave signal and the measurement video to the measurement space so that the intensity of the sound wave signal changes sequentially as the measurement video progresses.

The output control unit 11a is not limited to outputting the measurement video and sound wave signal stored in the storage unit 10. For example, the output control unit 11a may output the measurement video and sound wave signal distributed from the measurement device 4 via the communication system 3.

The data processing unit 11b receives the captured video captured by the camera of the UI 12 and the received sound acquired by the microphone of the UI 12, and sequentially associates the captured video and the received sound and temporarily stores them. That is, the data processing unit 11b may sequentially associate the photographed video and the received sound based on the input time and save them as a video file. Thereby, the reaction of the subject included in the captured video is saved together with the sound wave signal included in the received sound. In this way, detection data is obtained in which the reaction of the subject to the measurement video is associated with the sound wave signal.

The communication unit 13 is realized by various communication circuits that perform communication processing with the BS 5, and outputs the detection data obtained by the data processing unit 11b to the BS 5.

Note that the above-described hardware configuration is merely an example, and the terminal device 2 according to the present disclosure may be realized with any other appropriate hardware configuration.

FIG. 3 shows the configuration of the measuring device 4 in detail.

The measuring device 4 has a communication section 14, a processor 15, a storage section 16, and a user interface (UI) 17, which are interconnected via a bus B, as a hardware configuration.

The measuring device 4 may be realized by a computing device such as a server, a smartphone, a tablet, or a personal computer. Programs or instructions for realizing the various functions and processes of the measuring device 4 described below may be downloaded from an external device via a network or the like, or may be provided from a removable storage medium such as a flash memory.

The communication unit 14 is realized by various communication circuits that perform communication processing with the Internet 9, and receives detection data output from the plurality of terminal devices 2 via the communication system 3.

The processor 15 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuits, etc. that may be configured from one or more processor cores. The processor 15 executes various functions and processes of the measuring device 4, which will be described later, in accordance with programs, instructions, and data such as parameters necessary to execute the programs or instructions stored in the storage unit 16. The processor 15 according to this embodiment implements an acquisition section 15a and a measurement section 15b.

Based on the detection data received by the communication unit 14, the acquisition unit 15a acquires a reference signal (e.g., a sound wave signal) output to the measurement space and a response signal indicating the subject's response to measurement information (e.g., a measurement video) output to the measurement space in conjunction with the output of the reference signal.

For example, as shown in FIG. 4, the acquisition unit 15a acquires continuous reaction signals Sa1 and Sa2 indicating the eye reaction of the subject from the captured video of the detection data, and associates them with the reaction signals Sa1 and Sa2. The detected sound wave signal Sb is obtained from the received sound of the detection data. Here, the reaction signals Sa1 and Sa2 indicate time on the horizontal axis and the moving direction of the subject S's eyes on the vertical axis (Sa1 is the horizontal direction, Sa2 is the vertical direction). Moreover, the sound wave signal Sb shows time on the horizontal axis and amplitude on the vertical axis. Note that the times of the reaction signals Sa1 and Sa2 and the sound wave signal Sb indicate the playback time (time from start to end) of the measurement video.

The measurement unit 15b measures the reactivity of the subject based on the response signal and the sound wave signal acquired by the acquisition unit 15a. For example, in diagnosing dementia, it is said that signs of dementia can be found based on the reactivity of the eyes. For this reason, in diagnosing dementia, the measurement unit 15b may measure the reactivity of the subject's eyes based on the sound wave signal Sb by comparing the response signals Sa1 and Sa2 with a model generated from the response signals of healthy individuals and dementia patients. The measurement unit 15b may also determine a specific disease by comparing the response signals Sa1 and Sa2 with the above-mentioned model.

The storage unit 16 stores the subject's reaction signals Sa1 and Sa2 and the sound wave signal Sb obtained from the processor 15, the eye reactivity of the subject, and the determination result of a specific disease. The storage unit 16 is implemented by one or more non-transitory storage media, such as random access memory, flash memory, hard disk drive, etc., and stores the programs or instructions as well as the installed programs or instructions. Stores files, data, etc. used for execution. The storage unit 16 according to this embodiment stores reaction signals of a plurality of subjects in association with sound wave signals.

The user interface (UI) 17 may be composed of input devices such as a keyboard, mouse, camera, microphone, etc., output devices such as a display, speaker, headset, printer, etc., and input/output devices such as a touch panel, and provides an interface between the user and the measurement device 4.

Note that the hardware configuration described above is merely an example, and the measurement device 4 according to the present disclosure may be realized by other appropriate hardware configurations.

Next, the operation of this embodiment will be described.
An overview of this embodiment is shown in Fig. 5. In this embodiment, the terminal device 2 outputs a reference signal and measurement information in conjunction with each other to the measurement space R, and acquires a video of the subject S's reaction to the measurement information together with a received sound including the reference signal. Next, the measurement device 4 acquires a reaction signal indicating the subject S's reaction to the measurement information from the video, and acquires a reference signal from the received sound. Then, the measurement device 4 measures the reactivity of the subject S based on the reaction signal and the reference signal, and outputs the measurement result to the terminal device 2.

Next, the operation of this embodiment will be described in detail with reference to the flow chart shown in FIG.
2 controls the display and speaker of the UI 12 to output the measurement video (measurement information) and the sound wave signal (reference signal) Sb stored in the storage unit 10. As a result, in step S1, the measurement video is displayed on the display, and the sound wave signal Sb is output from the speaker to the measurement space R in which the measurement video is displayed.

At this time, the output control unit 11a of the processor 11 displays the sound wave signal Sb and the measurement video in conjunction with each other on the display. For example, the output control unit 11a may output the sound wave signal Sb and the measurement video based on saved data in which the sound wave signal Sb and the measurement video are stored in association with each other. That is, the saved data is data in which the sound wave signal Sb and the measurement video are stored in association with each other so that when either the sound wave signal Sb or the measurement video is output, the other is also output in conjunction. Examples of the saved data include data in which the sound wave signal Sb and the measurement video are included in one file, such as a video file in which the sound wave signal Sb and the measurement video are stored in association with each other. The saved video files include, for example, MP4 (Moving Picture Experts Group Audio Layer-4), AVI (Audio Video Interleave), MOV (QuickTime File Format), WebM, or FLV ( Flash Video), etc.
Note that the output control unit 11a may store the sound wave signal Sb and the measurement video separately, and start displaying the measurement video simultaneously with the start of output of the sound wave signal Sb. Further, the output control unit 11a may start displaying the measurement video at a predetermined timing while outputting the sound wave signal Sb.

As shown in FIG. 7, in the measurement space R in which the subject S is measured, the subject S watches the measurement video displayed on the display 12b while a sound wave signal Sb is output from the speaker 12a of the UI 12. At this time, the eyes of the subject S react to changes in the scene in the measurement video.

Here, an example of a measurement video will be explained.
For example, as shown in FIG. 8, when the measurement video M is started, a screen for aligning the line of sight of the subject S with respect to the display 12b is displayed as pre-processing. On this alignment screen, for example, four reference points P are sequentially displayed at predetermined positions, and voice and text instructions for the subject S to sequentially visually recognize the four reference points P are output together with the sound wave signal Sb. It's okay. Thereby, the line of sight of the subject S is aligned with the display 12b, and the position of the line of sight of the subject S can be accurately measured. At the same time, it can be confirmed that the sound wave signal Sb is normally reproduced in the measurement space R in conjunction with the measurement video M.

Next, as shown in FIG. 9A, a maze is displayed in the measurement video M, part of which is hidden by a blindfold Ma. Then, an instruction to "follow the entrance and exit of the maze with your line of sight" is output to the subject S, for example, by voice.

After a predetermined time elapses in the scene of the blindfold Ma, the measurement video M switches to a scene in which the entire maze is hidden by the blindfold Mb, as shown in FIG. 9B. Then, after a predetermined period of time has elapsed in the scene of the blindfold Mb, the measurement video M switches to a scene of a maze in which a portion other than the blindfold Ma is hidden by the blindfold Mc, as shown in FIG. 9C. At this time, if the subject S has memorized the maze displayed in fragments on the display 12b, the subject S's line of sight will move along the path connecting the entrance and exit of the maze.

At this time, the sound wave signal Sb shown in FIG. 7 is composed of a high frequency higher than a predetermined frequency, for example, a frequency exceeding a frequency band that is difficult for the average person to notice. Therefore, the subject S can watch the measurement video M without being distracted by the sound wave signal Sb.

In this way, while the subject S is watching the measurement video M, the camera 12c of the UI 12 captures the eye response of the subject S to the measurement video M, and the microphone 12d of the UI 12 collects the sound wave signal Sb output to the measurement space R.

Conventionally, when displaying a video on a terminal device and diagnosing dementia etc. based on the movement of the subject S's line of sight, it is necessary to confirm that the video is displayed correctly on the screen of the terminal device during measurement. A doctor or assistant was required to be present during the measurement. Furthermore, conventionally, there has been no method for accurately linking the temporal position of the moving image displayed at the time of measurement and the movement of the subject's S line of sight.

Therefore, in the present disclosure, the sound wave signal Sb is output to the measurement space R in conjunction with the measurement video M, and the eye reaction of the subject S to the measurement video M is photographed by the camera 12c, and the sound wave signal Sb output to the measurement space R is received by the microphone 12d. As a result, the photographed video of the eye reaction of the subject S and the sound wave signal Sb output to the measurement space R are acquired synchronously. For this reason, for example, based on the photographed video and the sound wave signal Sb acquired from multiple terminal devices 2, the eye reaction of each subject S and the temporal position of the measurement video M being played at that time can be accurately grasped. In addition, as shown in FIG. 8, when aligning the gaze, if the gaze of the subject S does not move in the acquired photographed video or if the sound wave signal Sb does not exist, it can be determined that the measurement is invalid because the reference signal and measurement information are not correctly output from the terminal device or the subject S cannot visually observe the measurement information. This makes it possible to ensure that the measurement information is correctly output even if a doctor or assistant is not present during the measurement.

Then, the captured video captured by the camera 12c and the received sound acquired by the microphone 12d are input to the data processing unit 11b of the processor 11. The data processing unit 11b sequentially associates the captured video and the received sound input from the camera 12c and the microphone 12d and temporarily stores them. As a storage format, for example, a file format in which the captured video and the received sound are associated and included in one video file can be mentioned. Here, the captured video is a video of the subject S's eye reaction to the measurement video M, and the received sound includes a sound wave signal Sa linked to the measurement video M. Therefore, by synchronously acquiring the captured video and the received sound, the eye reaction (movement) of the subject S is associated with the measurement video M via the sound wave signal Sb.

Now consider a case where the eye response of the subject S to the measurement video M is recorded in association with the time of a clock built into the terminal device 2, rather than with the sound wave signal Sb. In this case, the start time between pressing the play button for the measurement video M and the start of playback of the measurement video M on multiple different terminal devices 2 may differ due to factors such as the processing speed of the terminal device 2 and the available cache capacity. Furthermore, there is a risk that playback of the measurement video M may freeze temporarily during playback. As a result, the eye response of the subject S and the measurement video M will be associated with different times. In this case, even if the logs of each terminal device 2 are examined, it is not possible to determine the exact time at which the measurement video M was actually displayed on the display 12b.

Therefore, in the present disclosure, the eye reaction of the subject S is associated with the measurement video M via the sound wave signal Sb acquired from the measurement space R. As a result, even if, for example, the start of playback of the measurement video M from the terminal device 2 is delayed, the progress of the sound wave signal Sb will also be delayed along with the measurement video M. The eye reaction of the subject S and the measurement video M can be recorded in association with each other at the same temporal position.

Then, the data processing unit 11b determines that the reaction measurement has ended, for example, when the measurement video M has ended. The terminal device 2 then outputs the detection data, including the captured video and the received sound, to the measurement device 4 via the communication system 3.

For example, as shown in FIG. 1, the acquired detection data is first output wirelessly to the BS 5, and then output from the BS 5 to the measurement device 4 via the core network 8 and the Internet 9. For example, when the communication system 3 is configured with a 5G communication system or the like, the detection data can be streamed (output) from the terminal device 2 to the measurement device 4 and recorded in the measurement device 4.

The detection data output to the measurement device 4 shown in FIG. 3 is received by the acquisition unit 15a of the processor 15 via the communication unit 14. Then, in step S2, the acquisition unit 15a extracts reaction signals Sa1 and Sa2 indicating eye movements of the subject S and a sound wave signal Sb from the detection data.

Specifically, the acquisition unit 15a detects the eye part of the subject from the photographed video included in the detection data, and tracks the movement of the eyes (movement of the line of sight) according to the progress of the photographed video. The eye movements of the subject S indicate the reaction to the measurement video M. Therefore, the acquisition unit 15a can acquire continuous reaction signals Sa1 and Sa2 indicating the eye reaction of the subject S to the measurement video M by tracking the eye movements. Furthermore, the acquisition unit 15a performs a process of extracting the sound wave signal Sb from the received sound included in the detection data.
At this time, the acquisition unit 15a detects the orientation of the examinee's face in addition to the eye region of the examinee from the photographed image, and tracks the eye movement of the examinee based on the facial orientation. It's okay. For example, the acquisition unit 15a may limit the detection range of the subject's eye region based on the direction of the subject's face, and may track the movement of the subject's eyes within the detection range. Thereby, the eye reaction signals Sa1 and Sa2 can be measured more accurately.

Here, since the captured video and the received sound are associated with each other in the terminal device 2, the acquisition unit 15a can extract data from the acquired or recorded detection data in a form in which the sound wave signal Sb and the response signals Sa1 and Sa2 are associated with each other, as shown in FIG. 4.

The acquisition unit 15a may also record the detection data, or the sound wave signal Sb and the response signals Sa1 and Sa2 extracted from the detection data, in the storage unit 16 for each subject.

Next, in step S3, the measurement unit 15b of the processor 15 measures the reactivity of the subject S's eyes based on the response signals Sa1 and Sa2 and the sound wave signal Sb.

For example, when the response signals Sa1 and Sa2 of multiple subjects S are acquired, the response signals Sa1 and Sa2 are acquired and recorded in a manner that is similarly associated with the sound wave signal Sb between subjects S. That is, the measurement unit 15b can accurately measure the eye reactivity of the subject S by comparing the response signals Sa1 and Sa2 of the subject S with the response signals of other subjects at the same time position based on the sound wave signal Sb.

For example, the measurement unit 15b may compare the response signals Sa1 and Sa2 of the subject S with the response signals of a healthy person, a dementia patient, etc., based on the sound wave signal Sb. Generally, in diseases such as dementia, symptoms such as delayed reactions due to hesitation appear in the eye reactivity, unlike healthy people. Therefore, the measurement unit 15b may determine whether the eye reactivity of the subject S is normal or not by comparing the response signals Sa1 and Sa2 of the subject S with the response signals of a healthy person, a dementia patient, etc.

The measurement unit 15b may also use AI (Artificial Intelligence) to measure the reactivity of the subject S's eyes. For example, as shown in FIG. 10, the measurement unit 15b generates a trained machine learning model MD that learns the characteristics of both subjects based on case data of reaction signals in which the reaction signals Sa1 and Sa2 of multiple subjects are classified into healthy subjects and dementia patients. At this time, the trained machine learning model MD may be trained by assigning labels of classifications such as Alzheimer's dementia and Lewy body dementia and progression levels to each case data. Then, the measurement unit 15b may input the sonic signal Sb and the reaction signals Sa1 and Sa2 detected in a specific subject S to the trained machine learning model Ma to determine a sign of dementia from the reactivity of the subject S's eyes. The measurement unit 15b may also evaluate the reactivity of the subject S's eyes in stages.

In this way, the measurement unit 15b measures a reaction indicating the reaction of the subject S to the sound wave signal Sb output to the measurement space R and the measurement video M output to the measurement space R in conjunction with the output of the sound wave signal Sb. The reactivity of the subject S is measured based on the signals Sa1 and Sa2. This reaction signal Sa is accurately associated with the temporal position of the measurement video M via the sound wave signal Sb acquired from the measurement space R synchronously with the measurement video M. Therefore, the eye reactivity of the subject S to the display content of the measurement video M can be accurately measured.

The measurement unit 15b also measures the subject's reactivity by associating the response signals Sa1 and Sa2 acquired in different measurement spaces R and at different times based on the sound wave signal Sb. This allows the measurement unit 15b to easily measure the subject's reactivity.

The measurement unit 15b also measures the responsiveness of the subject S's eyes based on the continuous response signal Sa, which indicates the subject S's response to the measurement video M. On the other hand, when measuring the subject's response to a still image, the responsiveness of the subject S to an unchanging scene is measured based on the distribution of the response signal, so there is a large difference in the amount of information that can be obtained within the same amount of time. In this way, the measurement unit 15b can more accurately measure the responsiveness of the subject S's eyes by acquiring a large amount of information by linking the time-series changes in the response signal Sa with the measurement video M.

The sound wave signal Sb may also be configured to change in amplitude or frequency at intervals of several milliseconds, for example. This allows the time position of the measurement video M to be more precisely associated with the response signals Sa1 and Sa2 of the subject S, and therefore the movement of the subject S's line of sight can also be more precisely tracked. This allows the reactivity of the subject S's eyes to be measured more accurately.

The measurement unit 15b also compares the response signals Sa1 and Sa2 of the subject S with the response signals of other subjects based on the sound wave signal Sb. At this time, the response signals of multiple subjects S can be compared in correspondence with the same time position, so that the reactivity of the subject S's eyes can be measured more accurately.

Note that the measurement unit 15b may measure the eye reactivity of the subject S based on the reaction signals Sa1 and Sa2 when the scene of the measurement video M changes. For example, the measurement unit 15b may, based on the reaction signals Sa1 and Sa2 when the scene is changed from the scene where the entire maze is blindfolded shown in FIG. 9B to the scene where a part of the maze is blindfolded as shown in FIG. 9C, The eye reactivity of the subject S may also be measured.
Since the subject S's line of sight moves according to the switching of the screen and his/her own memory, the eye reactivity of the subject S can be measured more accurately based on the reaction signal Sa at that time.

In this way, the measurement unit 15b measures the reactivity of the eyes of the subject S, and then determines the state of the cognitive function of the subject S, etc., based on the measured reactivity. Then, in step S4, the measurement unit 15b outputs the measurement results, such as the reactivity measurement results and the possibility of disease, to the terminal device 2 of the subject S via the communication unit 14.

By regularly performing such measurements, we can scientifically evaluate the decline in cognitive function by comparing the recorded past and present reaction signals Sa1 and Sa2 of the subject S based on the sound wave signal Sb. can be recorded and evaluated. Furthermore, if the subject S develops dementia, it becomes possible to accurately compare the recorded data of the subject S with data of other healthy subjects measured using the same method.

(Embodiment 2)
Hereinafter, a second embodiment of the present disclosure will be described. Here, the differences from the first embodiment will be mainly described, and common reference numerals will be used to denote common features with the first embodiment, and detailed descriptions thereof will be omitted.

In the above-mentioned first embodiment, the measurement unit 15b measures the reactivity of the eye of the subject S, but it is not limited to this as long as it is possible to measure the reactivity of the subject S.
For example, the measurement unit 15b may measure the reactivity of the hand of the subject S. Furthermore, the measurement unit 15b may determine whether or not a disease such as Parkinson's disease is present based on the reactivity of the hand.

FIG. 11 shows an overview of this embodiment. In this embodiment, the terminal device 2 outputs a reference signal and measurement information in conjunction with each other to the measurement space R, and receives a reaction signal including the reference signal indicating the reaction of the hand of the subject S to the measurement information. Acquire with sound. Subsequently, the measuring device 4 extracts the hand reaction signal and the reference signal included in the received sound. Then, the measuring device 4 measures the hand reactivity of the subject S based on the hand reaction signal and the reference signal, and outputs the measurement result to the terminal device 2.

For example, as shown in FIG. 12, the UI 12 of the terminal device 2 may include a detection unit 21. This detection section 21 has a grip section 22, five detection buttons 23, and a speaker 24.

The grip portion 22 is a portion that the subject S grips with his/her hand, and is formed so as to extend in a rod shape.
The detection buttons 23 are to be pressed by the subject S with the fingers, and are arranged corresponding to the positions of the fingers. When the five detection buttons 23 are pressed by the subject S with the fingers, the five detection buttons 23 each control the speaker 24 to output a reaction sound into the measurement space R. For example, the reaction sound may be configured so that a sound of a different frequency band is output for each detection button 23. Furthermore, the reaction sound of the detection buttons 23 may be output into the measurement space R as a reaction signal Sc of a different frequency band from the sound wave signal Sb.

The detection unit 21 further includes a parallel sensor and a vibration sensor (not shown). The parallel sensor is a sensor that detects the parallelism and inclination of the hand. The vibration sensor is a sensor that detects the trembling of the hand. The detection signals detected by the parallel sensor and the vibration sensor are output to the data processing unit 11b of the processor 11 via an electric signal cable. The detection unit 21 may be physically connected to the terminal device 2 via an electric signal cable, or may be connected via wireless communication.

Next, we will explain the operation of this embodiment.

First, as in the first embodiment, under the control of the output control unit 11a, a sound wave signal Sb is output from the speaker 12a of the UI 12 to the measurement space R, and a measurement video is displayed on the display 12b in conjunction with the output of this sound wave signal Sb.

Here, an example of the measurement video will be described.
For example, as shown in Fig. 13, a plurality of rectangular change parts M1a are displayed in the measurement video M1. An instruction such as "When the change parts M1a are displayed in red, press the detection button 23" is output to the subject S, for example, by voice and text. The plurality of change parts M1a then change to red in sequence at irregular intervals, thereby inducing a reaction from the hand of the subject S. Note that the measurement video M1 and the instruction may be configured so that the measurement is not performed simultaneously on both the left and right hands, but is performed separately on the left and right hands.

12, the subject S holds the gripping portion 22 of the detection unit 21, and presses the detection button 23 with his/her finger in response to a change in the display of the change portion M1a. Then, a response signal Sc is output from the speaker 24 in response to the pressing of the detection button 23. In addition, a vibration sensor or the like built into the detection unit 21 constantly detects the trembling of the subject S's hand, and outputs the detection signal to the data processing unit 11b. Note that the response signal Sc may be composed of, for example, a sound wave signal having a frequency lower than that of the sound wave signal Sb, for example, a frequency included in the human audible range.
Then, similarly to the first embodiment, the microphone 12d receives the received sound including the sound wave signal Sb and the response signal Sc outputted to the measurement space R, and outputs the received sound to the data processing unit 11b.

The data processing unit 11b generates detection data by associating the detection signal detected by the detection unit 21 with the received sound input from the microphone 12d. Before the measurement, for example, four reference points P as shown in FIG. 8 may be sequentially flashed in red to induce the subject S to press the detection button 23, and the measurement may be performed after checking whether there is an error in the detection signal and the reaction signal Sc output from the detection unit 21. In this way, by associating the detection signal with the received sound, the reaction signal Sc and the detection signal of the hand of the subject S can be associated with the measurement video M1 via the sound wave signal Sb.
The detection data is then output via the data processing unit 11b to the communication unit 13 and the communication system 3 to the measuring device 4. The detection data is then input to the acquisition unit 15a of the processor 15 via the communication unit .

The acquisition unit 15a extracts the sound wave signal Sb and the reaction signal Sc from the received sound included in the detection data. As a result, data is acquired in which the reaction signal Sc of the subject S's hand is associated with the sound wave signal Sb, as shown in FIG. 14. Here, only the reaction signal Sc of one finger is shown. The measurement unit 15b then measures the reactivity of the subject S's hand based on the reaction signal Sc and the sound wave signal Sb acquired by the acquisition unit 15a. For example, the measurement unit 15b measures the reactivity of the subject S's hand by comparing the reaction signal Sc of the subject S with the reaction signals of other subjects based on the sound wave signal Sb.

At this time, the measurement unit 15b measures the reactivity of the subject S based on the sound wave signal Sb output to the measurement space R and the reaction signal Sc indicating the reaction of the subject S to the measurement video M output to the measurement space R in conjunction with the output of the sound wave signal Sb. This reaction signal Sc is accurately associated with the temporal position of the measurement video M1 via the sound wave signal Sb acquired from the measurement space R in synchronization with the measurement video M1. Therefore, by comparing with the reaction signals of other subjects, the reactivity of the hand of the subject S can be measured more accurately.

The measurement unit 15b may also determine whether the subject S has a disease such as Parkinson's disease based on the reactivity of the subject S's hand. Generally, compared to healthy individuals, a disease such as Parkinson's disease exhibits symptoms such as delayed reaction to opening and closing the hand. Therefore, the measurement unit 15b may determine the possibility of a disease such as Parkinson's disease based on the reactivity of the subject S's hand.

Additionally, diseases such as Parkinson's disease generally cause symptoms such as tremors and muscle rigidity. Therefore, the measuring unit 15b measures the hand tremor of the subject S based on the detection signal obtained from the vibration sensor built into the detecting unit 21 in addition to the hand reactivity, thereby detecting Parkinson's disease, etc. The possibility of disease may be determined.

As shown in FIG. 15, the terminal device 2 receives a reaction signal indicating the reaction of the hand of the subject S to the measurement information, a photographed video of the eye reaction, and a detection signal detected by the detection unit 21. may be acquired together with the received sound including the reference signal. Thereby, the measuring device 4 acquires the reference signal, the hand reaction signal, the eye reaction signal, and the detection signal. Then, the measuring device 4 measures the reactivity of the subject S based on the hand and eye reaction signals, the reference signal, and the detection signal.
At this time, the acquisition unit 15a acquires data in which the eye reaction signals Sa1 and Sa2 and the hand reaction signal Sc of the subject S are associated with the sound wave signal Sb, as shown in FIG. For example, the measuring unit 15b may measure the reaction signals Sa1 and Sa2 and the reaction signal Sc of the subject S at the same temporal position based on the sound wave signal Sb, and the reaction signals of the eyes and hands of other subjects. The reactivity of the eyes and hands of the subject S may be measured by comparing the signals with each other. Furthermore, the measurement unit 15b may determine a specific disease of the subject S, such as Parkinson's disease, or the degree of progression of the disease, based on the reactivity of the eyes and hands and the vibration of the hand.

Note that in the above, the reactivity of the subject S was measured using the detection unit 21, but the detection unit 21 is integrally arranged on the display 12b of the terminal device 2 (for example, a touch panel), and the display 12b is manually operated. The reactivity of the subject S may be directly measured by touching the test subject S without using the detection unit 21. For example, when the changing portion M1a of the measurement video M1 is displayed in red, the terminal device 2 instructs the subject S to touch the portion displayed in red with a finger. Thereby, the terminal device 2 may acquire the reaction signal Sc of the subject's S hand from the display 12b. Further, during this measurement, the terminal device 2 may acquire detection signals from a parallel sensor and a vibration sensor provided in the terminal device 2. Then, the data processing unit 11b generates detection data by associating the detection signal and reaction signal Sc detected by the terminal device 2, the captured video captured by the camera 12c, and the received sound input from the microphone 12d. You may.

According to the present embodiment, the measurement unit 15b measures the sound wave signal Sb output to the measurement space R and the measurement video M1 output to the measurement space R in conjunction with the output of the sound wave signal Sb. The reactivity of the hand of the subject S is measured based on the reaction signal Sc indicating the reaction of the hand. This reaction signal Sc can be correlated with the measurement video M1 with high precision via the sound wave signal Sb acquired from the measurement space R synchronously with the measurement video M1, so the reactivity of the hand of the subject S can be accurately determined. can be measured.

(Embodiment 3)
Embodiment 3 of the present disclosure will be described below. Here, we will mainly explain the differences from the above embodiments 1 and 2, and common points with the above embodiments 1 and 2 will be described in detail using common reference numerals. Omitted.

In the above-mentioned first and second embodiments, the measurement unit 15b judges a specific disease of the subject S, such as dementia, but it is not limited to this as long as the judgment can be made based on the reactivity of the subject S.

For example, the measurement unit 15b may determine whether the reflexes have improved based on the reactivity of the subject S. For example, reflexes (eg, body reflexes, dynamic visual acuity, etc.) are considered important for baseball players when hitting a ball at bat. However, although various methods have been proposed to improve reflexes (e.g., intake of good quality sleep, training methods, etc.), the conditions of athletes change daily, and their effects and changes can be easily evaluated during matches. It was difficult to accurately measure and evaluate in terms of time.

Therefore, as in the first and second embodiments, the terminal device 2 detects the reactions of the athlete's eyes and hands before and after trying the method for improving reflexes, for example, before and after ingesting the medicine. Subsequently, the acquisition unit 15a of the measuring device 4 acquires and records the athlete's reaction signals Sa1, Sa2, and Sc associated with the sound wave signal Sb, as shown in FIG. 16, for example. Then, the measurement unit 15b compares the recorded reaction signals Sa1, Sa2, and Sc based on the sound wave signal Sb, thereby determining the improvement in reflexes before and after the training. Based on this determination result, it is possible to evaluate the effectiveness of training for the athlete.

Note that in the present embodiment, the measurement unit 15b determines whether the reflexes have improved based on the reaction signals before and after trying the method for improving reflexes, but the method is not limited thereto. For example, the measurement unit 15b may determine the reflex state of each player on the team by comparing the reaction signals Sa1, Sa2, and Sc of a plurality of baseball players based on the sound wave signal Sb. Based on the results of this judgment, it is possible to identify players to be used in games or as pinch hitters.

According to this embodiment, the measurement unit 15b can accurately indicate the state of the reflexes of the subject S (athlete) in numerical form based on the reactivity of the subject S.

(Embodiment 4)
Hereinafter, a fourth embodiment of the present disclosure will be described. Here, the differences from the first to third embodiments will be mainly described, and common reference numerals will be used for common points with the first to third embodiments, and detailed descriptions thereof will be omitted.

The measurement unit 15b may determine the driving aptitude of the subject S based on the responsiveness of the subject S. For example, for a car driver, the response of the eyes to the driving environment, the response of the feet to operate the brakes, etc., the response of the hands to operate the steering wheel, etc., and the hearing response to external sounds, etc. are considered to be important.

Therefore, as shown in FIG. 17, the terminal device 2 takes pictures of the foot reaction signals, hand reaction signals, ear (auditory) reaction signals, and eye reactions of the subject S in response to the measurement information. A moving image is acquired together with received sound including a reference signal. Thereby, the measuring device 4 acquires the reference signal, the eye reaction signal, the foot reaction signal, the hand reaction signal, and the ear reaction signal. Then, the measuring device 4 measures the driving aptitude of the subject S based on the reaction signals of the eyes, feet, hands, and ears and the reference signal.

Specifically, as in the first embodiment, the terminal device 2 may acquire a photographed video of the reaction of the driver's (subject S) eyes to the measurement video. Here, the terminal device 2 may use a driving simulator, for example. Further, the measurement video may be displayed on the display 12b, for example, a video that changes depending on the operation of the driving simulator. Subsequently, the acquisition unit 15a of the measuring device 4 acquires the driver's eye reaction signal Sa and the sound wave signal Sb from the photographed video and the received sound. The measurement unit 15b then converts the driver's eye reaction signal Sa to the eye reaction signal Sa of other drivers (for example, a driver with driving ability and a driver lacking driving ability) based on the sonic signal Sb. The reactivity of the driver's eyes to the measured video is determined by comparing with the measured video. The measurement unit 15b can evaluate the line of sight during driving based on this determination result.

Furthermore, the terminal device 2 may acquire reaction signals of the driver's feet in response to the measurement video. For example, the detection unit 21 may be placed in a foot brake of a driving simulator, and a foot reaction signal (reaction sound) in response to a brake operation may be output from the speaker 24 in response to a press of the detection button 23 (for example, brake). Alternatively, the detection unit 21 may be placed on the accelerator of the driving simulator, and a foot reaction signal (reaction sound) corresponding to the accelerator operation may be output from the speaker 24 in response to the depression of the detection button 23 (for example, the accelerator). Thereby, the acquisition unit 15a of the measuring device 4 acquires the foot reaction signal and the sound wave signal Sb. Then, the measurement unit 15b determines the reactivity of the driver's feet to the measurement video by comparing the reaction signals of the driver's feet with the reaction signals of other drivers' feet based on the sound wave signal Sb. You may. Thereby, the measurement unit 15b can evaluate the reaction time of the brake operation or the reaction time of the accelerator operation.

Further, the terminal device 2 may acquire the reaction signal Sc of the driver's hand to the measurement video similarly to the second embodiment. For example, the detection section 21 may be placed on a steering wheel of a driving simulator, and a hand reaction signal corresponding to the steering wheel operation may be outputted from a parallel sensor of the detection section 21 to the data processing section 11b via an electric signal cable. Thereby, the acquisition unit 15a of the measuring device 4 acquires the hand reaction signal Sc and the sound wave signal Sb. Then, the measurement unit 15b determines the reactivity of the driver's hand with respect to the measurement video by comparing the reaction signal Sc of the driver's hand with the reaction signal of other drivers based on the sonic signal Sb. It's okay. Note that each reaction sound such as a brake or an accelerator may be outputted to the measurement space R as a reaction signal in a frequency band different from that of the sound wave signal Sb. Thereby, the measurement unit 15b can accurately measure and evaluate multiple reactivity of the subject S during driving.

The terminal device 2 may also output a test sound within the audible range of the subject S to the measurement space R as measurement information, and measure the driver's hearing in response to this test sound. Here, the test sound may be, for example, a horn sound. When the driver hears the test sound, it is necessary for him or her to check the direction from which the test sound is coming, so the driver's eye response signal Sa at this time is obtained as an ear response signal from the captured video. Then, the measurement unit 15b may determine the responsiveness of the driver's hearing to the test sound in the measurement video by measuring the driver's eye response signal based on the sound wave signal Sb. This allows the measurement unit 15b to evaluate whether the driver has the hearing required for driving.

In this way, the measurement unit 15b can accurately determine the driving ability of the driver based on the driver's responsiveness. For example, by determining the driving ability of an elderly person when renewing their driver's license, the license can be appropriately renewed.

According to this embodiment, the measurement unit 15b can accurately determine the driving ability of the driver based on the driver's reactivity.

(Embodiment 5)
Hereinafter, a sixth embodiment of the present disclosure will be described. Here, the differences from the first to fourth embodiments will be mainly described, and common reference numerals will be used for common points with the first to fourth embodiments, and detailed descriptions thereof will be omitted.

In the above first to fourth embodiments, a machine learning model was used to measure the response of the subject S from the sound wave signal Sb and the response signal, but such a machine learning model can be generated as follows.

For example, a machine learning model may be generated by a model generation device 61 shown in FIG. The model generation device 61 has a communication section 14, a processor 62, a storage section 16, and a user interface (UI) 17 that are interconnected via a bus B as a hardware configuration. Note that the communication unit 14, storage unit 16, and UI 17 have the same configuration as the measuring device 4, so a description thereof will be omitted.

The processor 62 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuits, etc. that may be configured from one or more processor cores. The processor 62 executes various functions and processes of the model generation device 61, which will be described later, in accordance with programs, instructions, and data such as parameters necessary to execute the programs or instructions stored in the storage unit 16. The processor 62 according to this embodiment implements an acquisition section 62a and a training section 62b.

The acquisition unit 62a acquires training data for training a machine learning model. This training data includes multiple subjects for a reference signal (for example, a sound wave signal Sb) output to the measurement space R and measurement information (for example, a measurement video) that is output to the measurement space R in conjunction with the output of the reference signal. It includes a reaction signal indicating the reaction of the examiner S, and determination results of the experts determining whether the conditions of the plurality of subjects are "normal" or "abnormal."

Here, when determining a disease such as dementia, the expert is a doctor or the like. For example, if a doctor determines that a subject has a tendency toward dementia, a doctor may perform a brain MRI (Magnetic Resonance Imaging) test or a brain CT (Computed Tomography) test, etc., and examine each subject. It may also be determined whether the person has dementia or not.

The training unit 62b associates the reaction signals of the multiple subjects S based on the reference signal acquired by the acquisition unit 62a, for example by aligning their temporal positions. Then, the training unit 62b generates a trained machine learning model by machine learning the machine learning model based on a data set that combines the associated reaction signals and the judgment results.

Next, the operation of this embodiment will be described with reference to the flowchart shown in FIG.

First, as in the first embodiment, in step S1, the terminal device 2 displays a measurement video M and outputs a sound wave signal Sb to the measurement space R in conjunction with the display of the measurement video M. The terminal device 2 synchronously acquires a captured video of the eye reaction of the subject S and the sound wave signal Sb, and generates detection data that associates these.

In this way, the generated detection data of the plurality of subjects S are input to the acquisition unit 62a of the model generation device 61 via the communication unit 14 or the UI 17, respectively. In step S21, the acquisition unit 62a extracts a reaction signal Sa indicating the eye movement of the subject S and a sound wave signal Sb from the detection data to acquire training data. At this time, the determination result of whether the subject S is "normal" or "abnormal" diagnosed by a doctor is input to the acquisition unit 62a as training data, for example. Furthermore, the determination result may include other information such as the type of dementia and the degree of progression.

Next, the training unit 62b associates the multiple reaction signals Sa with the same time position based on the sound wave signal Sb. Then, in step S22, the training unit 62b generates a machine learning model based on a data set combining the associated reaction signals Sa and the judgment results of each subject S.
At this time, the reaction signals Sa of the multiple subjects S can be compared with high accuracy based on the sound wave signal Sb acquired synchronously with the measurement video M from the measurement space R. That is, the reaction signals S of the multiple subjects S can be compared with very high accuracy and features can be extracted because the time positions of the reaction signals S of the multiple subjects S are all consistent with respect to the measurement video M based on the sound wave signal Sb. Therefore, the training unit 62b can generate a machine learning model that outputs an accurate judgment result (normal or abnormal, etc.).

Note that the model generation device 61 may be built into the measurement device 4. Thereby, the model generation device 61 can generate a machine learning model based on the reaction signal Sa and the sound wave signal Sb transmitted from the terminal device 2 to the measurement device 4. Further, the model generation device 61 and the measurement device 4 may be built into the terminal device 2. Thereby, the model generation device 61 can generate a machine learning model based on the reaction signal Sa and the sound wave signal Sb acquired by the terminal device 2.

According to the present embodiment, the training unit 62b is configured to train the subject S with respect to the sound wave signal Sb output to the measurement space R and the measurement video M output to the measurement space R in conjunction with the output of the sound wave signal Sb. A machine learning model is trained by inputting a reaction signal Sa indicating a reaction and a determination result of whether the subject S is "normal" or "abnormal" by an expert. At this time, since the reaction signal Sa can compare multiple training data with high precision based on the sound wave signal Sb acquired from the measurement space R synchronously with the measurement video M, accurate judgment results can be output. Machine learning models can be generated.

In the first to fifth embodiments described above, the reference signal is composed of the sound wave signal Sb, but it may be any signal that is output to the measurement space R, and is not limited to this. For example, the reference signal may be composed of an optical signal displayed on the display 12b that can be photographed by the camera 12c. For example, as shown in FIG. 20, a mirror 12e may be arranged on the display 12b of the first embodiment, and the optical signal Sd displayed together with the measurement video M may be projected onto the camera 12c by the mirror 12e.

Specifically, the light signal Sd may be configured so that a part of the measurement video M blinks in a specific pattern. The blinking pattern of the light signal Sd is configured to indicate a temporal position of the measurement video M, and is output to the measurement space R in conjunction with the measurement video M. The light signal Sd is then projected by the mirror 12e and photographed by the camera 12c together with the subject S's reaction. That is, the light signal Sd is projected by the mirror 12e into the shooting range of the camera 12c that photographs the subject S's reaction. Note that the light signal Sd does not need to blink the entire measurement video M, but only a part of the measurement video M may be blinked. This allows the data processing unit 11b to acquire a reference signal (light signal Sd) from the photographed video in which the subject S's reaction is photographed.

Further, the terminal device 2 may output a plurality of reference signals, for example, a sound wave signal Sb and an optical signal Sd to the measurement space R. By extracting the reference signals of the optical signal Sd and the sound wave signal Sb from the detection data acquired from the measurement space R, the terminal device 2 can accurately associate the captured moving image of the detection data with the received sound. For example, when the temporal positions of the photographed video and the received sound are shifted, the terminal device 2 aligns the photographed video and the received sound based on the sound wave signal Sb and the optical signal Sd, and responds to the reaction of the subject S. The signal can be more accurately correlated with the measurement video M. Further, if the temporal positions of the optical signal Sd and the sound wave signal Sb acquired during measurement are shifted, the terminal device 2 can display a warning message on the display 12b and cancel the measurement.

Furthermore, in the first to fifth embodiments described above, the measurement device 4 may distribute the reference signal and the measurement video M to the terminal device 2 via the communication unit 14. For example, the measuring device 4 may stream the reference signal and the measurement video M, that is, while delivering the reference signal and the measurement video M, the terminal device 2 may sequentially output the received measurement information from the speaker 12a and the display 12b.
Furthermore, in the first to fifth embodiments described above, the terminal device 2 may distribute detection data in which the captured video and the received sound are associated with each other to the measuring device 4 via the communication unit 13. For example, the terminal device 2 may stream the detection data, that is, may distribute the detection data to the measurement device 4 while acquiring the captured video and received sound from the camera 12c and the microphone 12d.

Generally, when streaming measurement information or detection data, phenomena such as the distribution and playback of measurement information or detection data may be temporarily interrupted or stopped depending on the internet environment, device processing speed, etc. . Here, in the present disclosure, the reference signal is output to the measurement space R in conjunction with the measurement information. As a result, even if there is a temporary interruption or stoppage in the distribution or playback of measurement information or detection data, the temporal position where the problem occurred in the measurement information or detection data can be determined based on the reference signal. can be accurately grasped.

In addition, in the above-mentioned embodiments 1 to 5, the terminal device 2 and the measurement device 4 are connected via the communication system 3, but this is not limited as long as the reaction signal and the reference signal can be acquired by the acquisition unit 15a of the measurement device 4. For example, the terminal device 2 and the measurement device 4 may be directly connected.

In addition, in the above first to fifth embodiments, the measuring device 4 is configured as a device separate from the terminal device 2, but it may be built into the terminal device 2.
For example, the processor 11 of the terminal device 2 can be configured to include an acquisition unit 15a and a measurement unit 15b. The acquisition unit 15a and the measurement unit 15b configure the measurement device 4 of the present disclosure.

For example, as in the first embodiment, the sound wave signal Sb is output from the UI 12 to the measurement space R, and the measurement moving image M is output from the UI 12 to the measurement space R in conjunction with the output of the sound wave signal Sb. Subsequently, the UI 12 photographs the reaction of the subject S to the measurement video M and receives the received sound including the sound wave signal Sb. Then, the data processing unit 11b generates detection data by associating the captured video with the received sound, and outputs the detection data to the acquisition unit 15a.
When the detection data is input, the acquisition unit 15a acquires reaction signals Sa1 and Sa2 indicating the reaction of the subject from the captured video, and acquires a sound wave signal Sb associated with the reaction signals Sa1 and Sa2 from the received sound. . Then, the measurement unit 15b measures the reactivity of the subject S based on the reaction signals Sa1 and Sa2 and the sound wave signal Sb.

In the above-mentioned first to fifth embodiments, the measurement video M and the sound wave signal Sb are output from the terminal device 2, but they may be output to the measurement space R in conjunction with each other, and are not limited to this. For example, the measurement video M and the sound wave signal Sb may be transmitted from a broadcasting station by television broadcasting radio waves and output to the measurement space R by a television receiving the radio waves. In addition, the measurement video M and the sound wave signal Sb may be stored in a recording medium such as a DVD (Digital Versatile Disc) and output from an output device arranged separately from the terminal device 2. Then, the subject S's reaction to the output measurement video is acquired by the terminal device 2 such as a smartphone and transmitted to the measurement device 4 via the communication system 3.
In this way, the reaction signal may be acquired by a device different from the device that outputs the sonic signal Sb and the measurement video M to the measurement space R. Even if the reaction signal is acquired by a different device, the sonic signal Sb and the measurement video M are output to the measurement space R in conjunction with each other, so that the reactivity of the subject S can be accurately measured.

Further, in the first to fifth embodiments described above, the measuring device 4 is connected to the terminal device 2 via the communication system 3, but the measuring device 4 may be placed within the core network 8 of the communication system 3. . For example, the measurement device 4 may be placed in a cloud server within the core network 8. Furthermore, when an edge server is provided between the BS 5 and the core network 8, the measurement device 4 may be placed on the edge server, or some functions of the measurement device 4 may be provided between the core network 8 and the edge server. They may be placed separately.

The above describes the embodiments of the present disclosure in detail with reference to the drawings, but the functions of each of the above-mentioned devices can be realized by a computer program.

A computer that realizes the functions of each device described above through programs includes input devices such as a keyboard, mouse, and touch pad, output devices such as a display and speakers, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random). storage devices such as hard disk drives and SSDs (Solid State Drives), DVD-ROMs (Digital Versatile Disk Read Only Memory), and USB (Universal Serial Bus) memories. Reading devices and networks that read information from recording media Each part is connected by a bus, including a network card for communication.

Then, the reading device reads a program for realizing the functions of each of the above devices from a recording medium in which the program is recorded, and stores the program in the storage device. Alternatively, the network card communicates with a server device connected to the network, and causes the storage device to store programs downloaded from the server device to implement the functions of each device.

Then, the CPU copies the program stored in the storage device to the RAM, and sequentially reads out and executes the instructions included in the program from the RAM, thereby realizing the functions of each of the devices described above.

Although specific examples of the present disclosure have been described in detail above, these are merely examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples illustrated above.

The measurement device, program, and model generation device disclosed herein can be used in devices and programs that utilize reaction signals that indicate the subject's reaction.

2 Terminal device 3 Communication system 4 Measuring device 5 BS
7 Main server 8 Core network 9 Internet 10 Storage unit 11, 62 Processor 11a Output control unit 11b Data processing unit 12 UI
12a Speaker 12b Display 12c Camera 12d Microphone 12e Mirror 13 Communication unit 14 Communication unit 15 Processor 15a, 62a Acquisition unit 15b Measurement unit 16 Storage unit 17 UI
21 Detection unit 22 Holding unit 23 Detection button 61 Model generation device 62b Training unit M, M1 Measurement video M1a Change unit Ma to Mc Blindfold MD Machine learning model P Reference point R Measurement space S Subject Sa, Sa1, Sa2 Eye response signal Sb Sound wave signal Sc Hand response signal Sd Optical signal

Claims (18)

an acquisition unit that acquires a reference signal output to the measurement space and a reaction signal indicating a reaction of the subject to the measurement information output to the measurement space in conjunction with the output of the reference signal;
a measurement unit that measures the reactivity of the subject based on the reference signal and the reaction signal;
A measuring device with The measuring device according to claim 1, wherein the reference signal changes so as to indicate the temporal position of the measurement information. The measurement device according to claim 1, wherein the reference signal includes a sound wave signal composed of a high frequency higher than a predetermined frequency. The measuring device according to claim 1, wherein the reference signal includes an optical signal configured such that a part of the measurement information blinks in a specific pattern. The acquisition unit acquires a reaction signal indicating an eye reaction of the subject to the measurement information,
The measuring device according to claim 1, wherein the measuring unit measures the eye reactivity of the subject based on the reaction signal and the reference signal. The acquisition unit acquires a response signal indicating a response of the subject's hand to the measurement information,
The measurement device according to claim 1 , wherein the measurement unit measures the reactivity of the subject's hand based on the reaction signal and the reference signal. The measurement information consists of a measurement video,
The acquisition unit acquires the reference signal output in conjunction with the measurement video and a continuous reaction signal indicating the reaction of the subject to the measurement video,
The measuring device according to claim 1, wherein the measuring section measures the reactivity of the subject based on the reference signal and the continuous reaction signal. The measurement device according to claim 1, wherein the measurement unit measures the reactivity of the subject by comparing the response signal of the subject with response signals indicating the responses of other subjects to the measurement information based on the reference signal. The measuring device according to claim 1, wherein the measurement information acquired in different measurement spaces and at different times is correlated based on the reference signal to measure the reactivity of the subject. The measuring unit inputs the reaction signal and the reference signal to a machine learning model trained based on the reaction signal indicating the reaction of other subjects to the measurement information, and measures the reactivity of the subject. The measuring device according to claim 1, which measures. The measuring device according to claim 1, wherein the measuring unit determines a specific disease of the subject based on the reactivity of the subject. The measurement device according to claim 1, wherein the measurement unit determines the state of the subject's reflexes based on the subject's reactivity. The measurement device according to claim 1, wherein the measurement unit determines the driving aptitude of the subject based on the subject's responsiveness. The measurement device according to claim 1, wherein the measurement unit is connected to a terminal device that outputs the reference signal and the measurement information to the measurement space via a communication system, and distributes the reference signal and the measurement information to the terminal device. The measurement device according to claim 1, wherein the reaction signal is acquired by a device different from the device that outputs the reference signal and the measurement information to the measurement space. to the computer,
acquiring a reference signal output to the measurement space and a reaction signal indicating a reaction of the subject to the measurement information output to the measurement space in conjunction with the output of the reference signal;
measuring the reactivity of the subject based on the reaction signal and the reference signal;
A program to run. an acquisition unit that acquires a reference signal output to a measurement space, a response signal indicating a response of a plurality of subjects to measurement information output to the measurement space in conjunction with the output of the reference signal, and a determination result of determining a state of the plurality of subjects;
a training unit that associates the response signals of the plurality of subjects based on the reference signal, and generates a machine learning model based on a data set that combines the associated response signals with the determination results;
A model generating device having: to the computer,
a reference signal outputted to a measurement space; a reaction signal indicating a reaction of a plurality of subjects to measurement information outputted to the measurement space in conjunction with the output of the reference signal; and a state of the plurality of subjects. a step of obtaining a determination result of determining the
a step of associating the reaction signals of the plurality of subjects based on the reference signal and generating a machine learning model based on a data set combining the correlated reaction signals and the determination results;
A program to run.

Images