JP7334958B2 - Determination device, determination method and determination program - Google Patents

Description

The present invention relates to a determination device, a determination method, and a determination program.

In recent years, users can virtually perform various actions by perceiving three-dimensional (3D) images using a predetermined display device (for example, a head mounted display (HMD)). Experienced xR (Virtual Reality (VR), Augmented Reality (AR)), Mixed Reality (collective term for Mixed Reality (MR), etc.) is utilized in various fields. For example, VR is used not only for entertainment such as games, but also for sports, training such as piloting an airplane and driving a car, rehabilitation, and surgical assistance robots (eg, da Vinci (registered trademark)).

On the other hand, a user perceiving a 3D image using a predetermined display device may experience a predetermined biological reaction (eg, Nausea, Oculomotor, Disorientation, etc.). is known (for example, Patent Document 1). The body reaction is also called motion sickness, 3D motion sickness, VR motion sickness, or the like. For convenience of explanation, this biological reaction is hereinafter referred to as “visual motion sickness”.

JP 2019-184883 A

There are individual differences in human cognitive ability (for example, receptivity to 3D images), and performing various actions related to the cognitive ability does not necessarily work well for all users. For example, when a user performs training or rehabilitation using 3D images, the user may not be able to obtain a sufficient effect of the training or rehabilitation due to motion sickness. Therefore, it is determined whether the user has a specific cognitive ability (for example, receptivity to 3D images) in order to appropriately obtain the effect of performing a specific action by the user (for example, an action using 3D images). It is desired to make it possible.

Accordingly, one object of the present invention is to provide a determination device, a determination method, and a determination program capable of determining whether or not a person has a specific cognitive ability (for example, receptivity to 3D images).

A determination device according to an aspect of the present invention includes an acquisition unit that acquires a brain image of a user, a computation unit that computes a gray matter volume of the user's brain based on the brain image, and the computed gray matter volume. a determination unit that determines whether the user has a specific cognitive ability based on the volume.

According to this aspect, it is possible to determine whether the user has a specific cognitive ability based on the gray matter volume. Therefore, by performing a specific action in consideration of the determination result, it is possible to appropriately obtain the effect of performing the action.

In the above aspect, the predictor generated based on machine learning using the gray matter volume of the predetermined region calculated based on the brain image of each user and the performance of the predetermined task related to the specific cognitive ability of each user wherein the determination unit determines whether the user has the specific cognitive ability based on the predicted performance value generated by inputting the calculated gray matter volume into the predictor You can judge.

According to this aspect, since a predictor generated based on machine learning using the gray matter volume and the performance is used, it is possible to improve the accuracy of determining whether or not the person has the specific cognitive ability. can.

In the above aspect, the machine learning uses a subjective index value related to the specific cognitive ability of each user in addition to the gray matter volume and the performance, and the acquisition unit acquires the brain image. A subjective index value of the user may be obtained, and the predicted value may be generated by inputting the calculated gray matter volume and the obtained subjective index value into the predictor.

According to this aspect, in addition to the gray matter volume and the performance, a predictor that has been trained by machine learning using the subjective index value is used, so the accuracy of determining whether or not the person has the specific cognitive ability. can be improved.

In the above aspect, the predetermined region may be a region corresponding to at least part of the superior parietal lobule and the caudate nucleus. According to this configuration, it is determined whether or not the user has the specific cognitive ability based on the gray matter volume of the superior parietal lobule and the caudate nucleus, so it is possible to improve the accuracy of determination of the acceptability. .

In the above aspect, the specific cognitive ability may include receptivity to 3D images. According to this aspect, by performing an action using the 3D video in consideration of the determination result as to whether or not the user has receptivity to the 3D video, the effect of performing the action can be appropriately obtained. can be done.

In the above aspect, the three-dimensional image is perceived by the user using a predetermined display device, and the predetermined display device includes a first display unit that displays a two-dimensional image for the user's right eye and a display unit that displays a two-dimensional image for the user's left eye. and a second display for displaying a two-dimensional image.

In the above aspect, a head-mounted display mounted on the user's head, or a non-mounted display in which the user looks into the first display section and the second display section with the right and left eyes, respectively. It may be a device.

A determination method according to another aspect of the present invention includes the steps of acquiring a brain image of a user, calculating a gray matter volume in a predetermined region of the user's brain based on the brain image, and determining whether the user has a particular cognitive ability based on the gray matter volume.

A determination program according to another aspect of the present invention comprises an acquisition unit that acquires a user's brain image, and a gray matter volume of a predetermined region of the user's brain based on the brain image. and a determination unit that determines whether the user has a specific cognitive ability based on the calculated gray matter volume.

ADVANTAGE OF THE INVENTION According to this invention, the determination apparatus, determination method, and determination program which can determine whether it has a specific cognitive ability (for example, receptivity to a 3D image) can be provided.

It is a figure which shows an example of the action using the 3D image|video which concerns on embodiment of this invention. FIG. 4 is a diagram showing an example of the correlation between acceptability for 3D images and gray matter volume calculated using a predetermined region of a brain image as a region of interest according to the embodiment of the present invention; It is a figure showing an example of functional composition of judgment system 1 concerning an embodiment of the present invention. It is a figure which shows an example of the hardware constitutions of each apparatus which comprises the determination system 1 which concerns on embodiment of this invention. 4 is a flowchart showing an example of learning operation according to the embodiment of the present invention; 4 is a flow chart showing an example of determination operation according to the embodiment of the present invention; FIG. 4 is a diagram showing an example of the relationship between the performance of a perceptual task and the acceptability for 3D images according to the embodiment of the present invention; FIG. 5 is a diagram showing an example of the effect of determining the acceptability of a 3D image according to the embodiment of the present invention;

Embodiments of the present invention will be described with reference to the accompanying drawings. It should be noted that, in each figure, the same reference numerals have the same or similar configurations.

In this embodiment, receptivity to three-dimensional (3D) images is exemplified as an example of a specific cognitive ability determined based on the volume of gray matter in a predetermined region of the brain (gray matter volume). Note that the specific cognitive ability is not limited to receptivity to 3D images, and may be any ability that correlates with the volume of gray matter in a predetermined region of the brain.

In this embodiment, a user perceives (also referred to as viewing, recognizing, etc.) a three-dimensional (3D) image using a predetermined display device. Specifically, the predetermined display device includes a first display unit that displays a two-dimensional (2D) image for the user's right eye (right-eye image) and a second display unit that displays a 2D image for the left eye (left-eye image). 2 display. Here, the 3D image is an image viewed stereoscopically by the user, and is also called stereoscopic image or 3D stereoscopic vision. The 3D image can also be said to be an image that allows the user to perceive depth. For example, displaying an image with parallax (i.e., a right-eye image and a left-eye image) to the user (also called binocular parallax); A 3D image may be perceived by the user by changing the appearance of the image according to movement (also referred to as motion parallax). Note that the 3D image is not limited to one that uses binocular parallax, and may be a hologram, a volume scanning type, or the like. A 2D image is an image that can be viewed two-dimensionally by a user, and is also called a two-dimensional image.

Here, the right-eye image and left-eye image for perceiving a 3D image using binocular parallax are images obtained by capturing the same object at different angles. The user synthesizes the image for the right eye displayed on the first display unit for the right eye and the image for the left eye displayed on the second display unit for the left eye into one in his or her brain, thereby stereoscopically displaying the object. (i.e., perceive as a 3D image).

The predetermined display device including the first display section and the second display section may be a wearable display device that is worn on a predetermined site of the user, or may be a non-wearable display device. The wearable display device may be, for example, a head mounted display (HMD) worn on the user's head, a VR headset, goggles, or the like. Non-wearable display devices include, for example, immersive displays such as Cave Automatic Virtual Environment (CAVE), stereoscopic televisions, and consoles with a first display for the right eye and a second display for the left eye. It may be a surgical assistance robot or the like.

FIG. 1 is a diagram showing an example of an action using 3D video according to this embodiment. In FIG. 1, the HMD 100 worn on the user's head is exemplified as the predetermined display device, but as described above, the predetermined display device is not limited to this. In addition, in FIG. 1, sports (here, tennis) are shown as an example of an action using 3D images, but the action is an action in a field other than sports (for example, piloting an aircraft, driving a vehicle, rehabilitation, patient's surgery) and the like. In addition, in FIG. 1, the 3D video realizes virtual reality (VR), but may also realize augmented reality (AR) or mixed reality (MR).

In FIG. 1, the user perceives the 3D image V by the parallax between the right-eye image and the left-eye image displayed on the first display unit 101 and the second display unit 102 of the HMD 100, respectively. For example, in FIG. 1, the user is training for tennis based on a 3D image V perceived using the HMD 100 worn on the head.

In addition, the HMD 100 includes a sensor 103 that detects motion of a user's predetermined parts (eg, head, eyes, hands, feet, etc.), and displays right-eye and left-eye images based on the analysis result of the motion. It may have a tracking function to control. The tracking function is also called head tracking, eye tracking, motion tracking, and the like. The sensor that detects the movement may be, for example, a magnetometer that detects direction, a gyroscope that detects angle and angular velocity, an accelerometer that detects acceleration, an infrared sensor, or the like.

The tracking function changes the 3D image perceived by the user according to the user's predetermined movement. For example, in FIG. 1, the movement, position, etc. of the racket and ball perceived in the 3D image V change according to the movement of the hand of the user operating the racket, the direction of the eyes, and the movement of the head. Therefore, it is possible to improve the sense of realism and immersion of the user.

By the way, there are users who experience a predetermined biological reaction called "visual motion sickness" (for example, discomfort, eye fatigue, lightheadedness, etc.) triggered by visual movement in 3D images as described above. It is known.

Therefore, perceptual tasks of 2D images and 3D images (hereinafter abbreviated as 2D/3D images) for a predetermined number of subjects (for example, Multiple Object Tracking (MOT) using 2D/3D images) A group of subjects with no significant difference in 2D/3D image performance (first subject group) and a group of subjects with significantly lower 3D image performance than 2D image performance (second 2 subject groups). Note that the MOT task is a predetermined task of tracking multiple objects in an image, and is used to evaluate a subject's perceptual ability.

The first subject group can achieve the same results in the perceptual task using the 3D image as in the case of using the 2D image, so it can be said that the receptivity to the 3D image is higher than that of the second subject group. On the other hand, the second subject group has a significantly lower performance in the perceptual task using the 3D image than the case of using the 2D image, so it can be said that the second subject group has lower receptivity to the 3D image than the first subject group. Here, the receptivity to 3D images means that even if a person perceives 3D images, motion sickness is unlikely to occur, and the effect of performing actions using 3D images can be appropriately obtained.

FIG. 2 is a diagram showing an example of the correlation between the receptivity to 3D images and the gray matter volume calculated using a predetermined region of a brain image as a region of interest (ROI) according to the present embodiment. FIG. 2 shows a brain image I1 viewed from the side of the human brain, and a brain image I2 showing a cross section obtained by cutting the human brain along the cutting plane X of the brain image I1. The superior parietal lobule shown in brain image I1 is one of the gyri on the lateral surface of the cerebrum. The superior parietal lobule plays a role in recognizing information related to space, such as the position, movement, and depth of an object. On the other hand, the caudate nucleus shown in the brain image I2 exists near the center of the brain and on both sides of the thalamus. The caudate nucleus is involved in motor control, learning, cognition, and the like.

As shown in FIG. 2, for the first group of subjects with relatively high receptivity to 3D images (i.e., relatively small performance differences in the 2D/3D image perception task), the gray matter in the superior parietal lobule It tends to be larger in volume and smaller in caudate gray matter volume. On the other hand, for the second group of subjects, who had relatively high receptivity to 3D images (i.e., 3D image performance was significantly lower than 2D image performance), the gray matter volume in the superior parietal lobule was smaller and the caudate There tends to be a large nuclear gray matter volume.

Thus, there is a positive correlation between receptivity to 3D images and superior parietal gray matter volume. On the other hand, there is a negative correlation between receptivity to 3D images and caudate gray matter volume. That is, it is assumed that specific cognitive abilities, such as receptivity to 3D images, are correlated with gray matter volume in a given region of the brain. Therefore, the present inventors focused on the correlation between a specific cognitive ability (for example, receptivity to 3D images) and the gray matter volume in a predetermined region of the brain, and based on the gray matter volume in the predetermined region, a specific The idea was to determine cognitive performance (eg, receptivity to 3D images).

In the following, it is assumed that the predetermined region of the brain used to determine a specific cognitive ability (for example, receptivity to 3D images) is at least one of the superior parietal lobule and the caudate nucleus, but is limited to this. can't The predetermined region can be any region of the brain that correlates with the particular cognitive ability. The predetermined region may include, for example, at least one of the angular gyrus and the basal ganglia instead of or in addition to at least one of the superior parietal lobule and the caudate nucleus.

(Configuration of judgment system)
FIG. 3 is a diagram showing an example of the determination system 1 according to the embodiment of the invention. The determination system 1 includes a determination device 10 that predicts whether or not a person has a specific cognitive ability (for example, receptivity to 3D images) using a predictor 14a, and a learning process of a prediction model 22b implemented in the predictor 14a. and a learning device 20 that performs Note that the predictor 14a that implements the prediction model 22b may also be called a prediction model, a prediction program, a prediction algorithm, or the like.

Further, in this example, the determination device 10 using the predictor 14a learned by machine learning by the learning device 20 will be described. Determining the specific cognitive ability based on a theoretically derived relational expression or an empirically derived relational expression regarding the relationship between gray matter volume and specific cognitive ability (e.g., receptivity to 3D images) may be

Also, in this example, the determination device 10 and the learning device 20 are separate devices, but the determination device 10 and the learning device 20 may be configured as an integrated device. For example, the determination device 10 and the learning device 20 may be implemented as different operations of a PLC (Programmable Logic Controller).

<Learning device>
The learning device 20 includes, as a functional configuration, a generation unit 21 that generates learning data 22a, a storage unit 22 that stores learning data 22a and a prediction model 22b, and a prediction model 22b by machine learning using the learning data 22a. and a learning unit 23 for performing the learning process. The generation unit 21 has a brain image acquisition unit 211 , a gray matter volume calculation unit 212 , a performance acquisition unit 213 , and a subjective index value acquisition unit 214 .

The brain image acquisition unit 211 acquires a brain image of each user. The brain image may be, for example, an image captured by a Magnetic Resonance Imaging (MRI) device (eg, brain images I1, I2, etc. in FIG. 2). A brain image may be called a brain anatomy image or the like. The MRI machine may be, for example, a 3 Tesla MRI machine.

The gray matter volume calculation unit 212 calculates the gray matter volume of a predetermined region of the user's brain based on the brain image acquired by the brain image acquisition unit 211 . Specifically, the gray matter volume calculator 212 sets a predetermined region of the brain image as the region of interest and calculates the gray matter volume in the region of interest. The predetermined region is, for example, a region corresponding to at least a portion of the superior parietal lobule and the caudate nucleus, but is not limited to this, and may be a region corresponding to at least a portion of the angular gyrus and the basal ganglia. good. The gray matter volume may be assessed using, for example, the Voxel-based-Morp hometry (VBM).

The score acquisition unit 213 acquires a score (for example, correct answer rate or score) of a predetermined task (for example, a perceptual task using 3D images) regarding each user's specific cognitive ability. The performance of the perceptual task using the 3D video may be, for example, the performance of the MOT task using the 3D video.

The subjective index value acquiring unit 214 acquires a subjective index value (for example, a subjective index value relating to motion sickness) regarding specific cognitive ability of each user. The subjective index value related to motion sickness may be, for example, information indicating a Simulator Sickness Questionnaire (SSQ) value. In the SSQ, the user answers a predetermined number of question items (eg, 16 question items) classified into discomfort, eye strain, and dizziness in a predetermined stage (eg, 4 stages). The SSQ value may be a predetermined index value generated based on the answer. The SSQ value may be generated based on the user's answers after performing the perceptual task.

The storage unit 22 stores learning data 22 a generated by the generation unit 21 and a prediction model 22 b generated by learning processing by the learning unit 23 . The learning data 22a may be data in which the gray matter volume calculated by the gray matter volume calculation unit 212 and the score obtained by the score acquisition unit 213 are associated with each user. Further, the learning data 22a may be data in which the subjective index value acquired by the subjective index value acquiring unit 214 is associated with each user in addition to the gray matter volume and grade.

The learning unit 23 executes learning processing of the prediction model 22b by machine learning using the learning data 22a stored in the storage unit 22 . For example, the learning unit 23 performs learning processing of the prediction model 22b so that the results of each user stored as the learning data 22a are output based on the gray matter volume of each user stored as the learning data 22a. may be performed. In addition, the learning unit 23 predicts that the performance of each user stored as the learning data 22a is output based on the gray matter volume and the subjective index value of each user stored as the learning data 22a. A learning process for the model 22b may be performed.

The prediction model 22b machine-learned by the learning unit 23 may be composed of, for example, a neural network. When the prediction model 22b is configured by a neural network, the learning unit 23 may perform neural network learning processing by, for example, the error backpropagation method.

<Determination device>
The determination device 10 includes a brain image acquisition unit 11 , a gray matter volume calculation unit 12 , a subjective index value acquisition unit 13 and a determination unit 14 . The brain image acquisition unit 11 acquires a brain image of a user whose specific cognitive ability (for example, receptivity to 3D images) is determined. The gray matter volume calculation unit 12 sets a predetermined region of the brain image acquired by the brain image acquisition unit 11 as the region of interest, and calculates the gray matter volume in the region of interest. The subjective index value acquiring unit 13 acquires a subjective index value relating to specific cognitive ability of the user (for example, a subjective index value relating to motion sickness). The details of the brain image acquisition unit 11, the gray matter volume calculation unit 12, and the subjective index value acquisition unit 13 are explained in the brain image acquisition unit 211, the gray matter volume calculation unit 212, and the subjective index value acquisition unit 214, respectively. As I said.

Based on the gray matter volume calculated by the gray matter volume calculation unit 12, the determination unit 14 determines whether the user has a specific cognitive ability (for example, receptivity to 3D images). Also, the determination unit 14 may include a predictor 14 a generated by machine learning in the learning device 20 . It should be noted that the predictor 14a may implement new prediction models generated by distillation or generated by distillation. In the distillation, a new prediction model may be generated using input/output data to the prediction model 22b machine-learned by the learning device 20 .

Specifically, the determination unit 14 determines whether the user has a specific cognitive ability (for example, 3D receptivity to video). In addition to the gray matter volume, the determination unit 14 also inputs the subjective index value acquired by the subjective index value acquisition unit 13 to the predictor 14a to generate a prediction value based on the user-specified (eg, receptivity to 3D images). The predicted value may be a predicted value of the performance of a predetermined task related to each user's specific cognitive ability (for example, a predicted value of the correct answer rate of an MOT task using 3D video).

The determination result by the determination unit 14 may be output to the output unit 10e, which will be described later, and displayed to the user. Alternatively, the determination result may be transmitted to another device via the communication unit 10c.

<Hardware configuration>
Next, the hardware configuration of each device in the determination system 1 (for example, at least one of the determination device 10 and the learning device 20) will be described. As shown in FIG. 4, each device in the determination system 1 includes a CPU (Central Processing Unit) 10a corresponding to an arithmetic device, a storage device 10b, a communication section 10c, an input section 10d, and an output section 10e. have. These components are connected to each other via a bus so that data can be sent and received. In this example, the determination device 10 and the learning device 20 are composed of separate computers, but may be composed of a single computer.

The CPU 10a is a control unit that controls the execution of programs stored in the storage device 10b and performs data calculation and processing. The CPU 10a is an arithmetic device (calculation unit ). The CPU 10a receives various input data from the input unit 10d and/or the communication unit 10c, outputs (for example, displays) the calculation results of the input data to the output unit 10e, stores them in the storage device 10b, or communicates them. and transmitted via the unit 10c.

The storage device 10b is at least one of memory, HDD (Hard Disk Drive), and SSD (Solid State Drive). The storage device 10 b of the learning device 20 may constitute the storage unit 22 . The storage device 10b of the determination device 10 may store determination programs executed by the CPU 10a.

The communication unit 10c is an interface that connects each device in the determination system 1 to an external device. Note that when the determination device 10 and the learning device 20 are configured as an integrated device, the communication unit 10c includes inter-process communication between the process operating as the determination device 10 and the process operating as the learning device 20. good.

The input unit 10d receives data input from the user, and may include at least one of a keyboard, mouse, touch panel, and microphone, for example.

The output unit 10e outputs the calculation result by the CPU 10a, and may be configured by at least one of a display such as an LCD (Liquid Crystal Display) and a speaker.

The determination program may be provided by being stored in a computer-readable storage medium such as the storage device 10b, or may be provided via the network N connected by the communication unit 10c. The storage medium storing the determination program may be a non-transitory computer readable medium. The non-temporary storage medium is not particularly limited, and may be, for example, a USB memory, CD-ROM, DVD, or other storage medium.

In the determination device 10, the CPU 10a executes the determination program, thereby realizing the operations of the brain image acquisition unit 11, the gray matter volume calculation unit 12, the subjective index value acquisition unit 13, and the determination unit 14 described with reference to FIG. be done. It should be noted that these physical configurations are examples, and do not necessarily have to be independent configurations. For example, the determination device 10 may include an LSI (Large-Scale Integration) in which the CPU 10a and the storage device 10b are integrated.

In addition, the brain image acquisition units 11 and 211, the subjective index value acquisition units 13 and 214, and the result acquisition unit 213 respectively receive the brain image received by the communication unit 10c, the subjective index value, and the predetermined task related to the specific cognitive ability. (For example, MOT task using 3D video) results may be acquired, or brain images, subjective index values, and results stored in a computer-readable storage medium such as the storage device 10b may be obtained.

(Operation of judgment system)
<Learning action>
FIG. 5 is a flow chart showing an example of the learning operation according to this embodiment. Note that FIG. 5 assumes acceptability for 3D images as an example of specific cognitive ability, and the learning operation of the prediction model 22b that predicts acceptability for the 3D images will be described.

As shown in FIG. 5, in step S101, the learning device 20 acquires a brain image of each user. For example, the learning device 20 may acquire brain images captured by an MRI device (eg, a 3 Tesla MRI device). In step S102, the learning device 20 sets the predetermined region of the brain image acquired in step S101 as the region of interest, and calculates the gray matter volume of the region of interest. The region of interest may be, for example, a region corresponding to at least part of the superior parietal lobule and the caudate nucleus.

In step S103, the learning device 20 acquires a subjective index value (for example, an SSQ value) regarding motion sickness of each user. For example, an SSQ may be administered to each user after a perceptual task (for example, an MOT task using 3D video), and an SSQ value determined based on the answer to the SSQ may be obtained as a subjective index value.

In step S104, the learning device 20 acquires the score of each user's perceptual task using 3D images. As described above, the perceptual performance information may be information (for example, correct answer rate) indicating the performance of a perceptual task (for example, an MOT task using 3D video).

In step S105, the learning device 20 uses learning data 22a that associates the gray matter volume calculated in step S102, the subjective index value obtained in step S103, and the perceptual performance information obtained in step S104. The learning process of the predictive model 22b may be performed by machine learning using conventional techniques.

In addition, in FIG. 5, the order of steps S101, S103, and S104 does not matter. Also, in FIG. 5, step S103 may be omitted. When omitting step S103, in step S105, the learning device 20 performs machine learning using learning data that associates the gray matter volume calculated in step S102 with the grades obtained in step S104, so that the prediction model 22b learning process may be performed.

<Judgment operation>
FIG. 6 is a diagram showing an example of determination operation according to the present embodiment. Note that FIG. 6 assumes acceptability for 3D images as an example of specific cognitive ability, and the determination operation for determining acceptability for the 3D images will be described.

As shown in FIG. 6, in step S201, the determination device 10 acquires brain images of a user whose receptivity to 3D images is to be determined. In step S202, the determination device 10 sets the predetermined region of the brain image acquired in step S201 as the region of interest, and calculates the gray matter volume of the region of interest. In step S203, the determination device 10 acquires a subjective index value related to motion sickness of the user. Details of steps S201, S202, and S203 are the same as steps S101, S102, and S103 in FIG.

In step S204, the determination device 10 inputs the gray matter volume calculated in step S202 and the subjective index value obtained in step S203 to the learned predictor 14a, and outputs a 3D image as the output of the predictor 14a. may be used to generate predictions of perceptual task performance (performance predictions). The predicted grade value may be, for example, a predicted value of the percentage of correct answers or the score of the MOT task.

In step S205, the determination device 10 determines whether or not the predicted performance value generated in step S204 satisfies a predetermined condition. If the predicted result value satisfies a predetermined condition, in step S206, the determination device 10 determines that the user has receptivity to 3D images. On the other hand, if the predicted result value does not satisfy the predetermined condition, in step S207, the determination device 10 determines that the user is not receptive to 3D images.

Note that in FIG. 6, the order of steps S201 and S203 does not matter. Also, in FIG. 6, step S203 may be omitted. When omitting step S203, in step S204, the determination device 10 inputs the gray matter volume calculated in step S202 to the learned predictor 14a, and generates a performance prediction value as the output of the predictor 14a. may

FIG. 7 is a diagram showing an example of the relationship between the performance of the perceptual task and the acceptability for 3D images according to this embodiment. As shown in FIG. 7, the predetermined condition used to determine whether or not the 3D image is acceptable may be determined based on the perceptual task performance distribution for a predetermined number of subjects. FIG. 7 shows the distribution of results when 78 subjects performed a perceptual task using 3D images (here, MOT task).

For example, as shown in FIG. 7, when the results of the perceptual task are distributed, the predetermined condition to be judged to have receptivity to the 3D image is that the result is a correct answer rate of 70% or more. good.

For example, when using the conditions in FIG. 7, if the predicted score value output from the predictor 14a in step S204 in FIG. Judged to be acceptable. On the other hand, when the predicted result value output from the predictor 14a in step S204 of FIG. 6 indicates a correct answer rate of less than 70%, it is determined that the user corresponding to the predicted result value does not have receptivity to 3D images. be.

Note that FIG. 7 is only an example, and the predetermined condition for determining that the subject has acceptability for 3D images is not limited to a correct answer rate of 70% or more. The predetermined condition may be any condition as long as it is determined based on the performance of the perceptual task (for example, percentage of correct answers) and a predetermined threshold.

FIG. 8 is a diagram showing an example of the effect of determining the acceptability of a 3D image according to this embodiment. FIG. 8 shows the distribution of growth rates when a predetermined number (here, 48 subjects) of subjects undergo sports training using 3D images for a predetermined period (for example, one month).

In FIG. 8, 22 out of 48 subjects had an elongation rate of less than 10%. Based on the brain images of the 22 subjects, regions corresponding to the superior parietal lobule and the caudate nucleus were set as regions of interest, and gray matter volumes were calculated. When the gray matter volume was input to the learned predictor 14a and the receptivity to the 3D image of the 22 subjects was determined in advance, it was determined that 19 subjects had no receptivity to the 3D image. Ta.

Therefore, even if sports training using 3D video is performed for a user who is determined not to be receptive to 3D video using the learned predictor 14a, the effect of the training cannot be obtained appropriately. Probability is high. Therefore, by conducting training using 3D video for users who are determined to have receptivity to 3D video, it is possible to efficiently obtain training implementation effects.

In addition, in FIG. 8, sports training is shown as an example of an action using 3D images. It can be any action that is performed. By using the learned predictor 14a to consider the acceptability determination result for the 3D video when performing the action using the 3D video, the effect of performing the action can be obtained more efficiently.

As described above, according to the determination system according to the present embodiment, the acceptability of the user for 3D images is determined based on the gray matter volume of the predetermined region of the user's brain calculated based on the brain image of the user. I can judge. Therefore, by performing an action using a 3D image for a user determined to have receptivity to the 3D image, it is possible to efficiently obtain the effect of performing the action.

In the above embodiment, a specific cognitive ability (for example, receptivity to 3D images) is determined based on the gray matter volume in a predetermined region of the brain, and the parameters used for the determination are brain function and brain structure. is not limited to gray matter volume, as long as it is a parameter that indicates at least one of For example, instead of or in addition to the gray matter volume, the diffusion anisotropy (Fractional Anisotropy (FA) value) of the predetermined region of the brain may be calculated. The diffusion anisotropy may reflect qualitative and quantitative changes in the white matter fibers of the brain. The diffusion anisotropy may be calculated based on the diffusion weighted image. The diffusion-weighted image may be obtained, for example, by MRI using a gradient magnetic field (Motion Probing gradient (MPG)) applied to detect the diffusion of hydrogen atoms. Alternatively, parameters determined based on at least one of resting brain functional connectivity, myelin maps, and ultrastructural images may be used.

The embodiments described above are for facilitating understanding of the present invention, and are not intended to limit and interpret the present invention. Each element included in the embodiment and its arrangement, materials, conditions, shape, size, etc. are not limited to those illustrated and can be changed as appropriate. Also, it is possible to partially replace or combine the configurations shown in different embodiments.

DESCRIPTION OF SYMBOLS 1... Determination system 10... Determination apparatus 20... Learning apparatus 11... Brain image acquisition part 12... Gray matter volume calculation part 13... Subjective index value acquisition part 14... Judgment part 14a... Predictor 21 Generating unit 22 Storage unit 22a Learning data 22b Prediction model 23 Learning unit 211 Brain image acquisition unit 212 Gray matter volume calculation unit 213 Results acquisition unit 214 Subjective Index value acquisition unit 10a CPU 10b storage device 10c communication unit 10d input unit 10e output unit I1, I2 brain image V 3D video X section plane 100 HMD 101... First display unit, 102... Second display unit, 103... Sensor

Claims (8)

an acquisition unit that acquires a user's brain image;
a computing unit that computes a gray matter volume in a predetermined region of the user's brain based on the brain image;
a determination unit that determines whether the user has specific cognitive abilities including receptivity to 3D images based on the calculated gray matter volume;
A determination device comprising: A predictor generated based on machine learning using the gray matter volume of the predetermined region calculated based on the brain image of each user and the performance of a predetermined task related to the specific cognitive ability of each user,
The determination unit determines whether the user has the specific cognitive ability based on the predicted value of the performance generated by inputting the calculated gray matter volume into the predictor.
The determination device according to claim 1. In the machine learning, in addition to the gray matter volume and the performance, a subjective index value regarding the specific cognitive ability of each user is used,
The acquisition unit acquires a subjective index value related to the specific cognitive ability of the user from whom the brain image was acquired,
wherein the predicted value is generated by inputting the calculated gray matter volume and the obtained subjective index value into the predictor;
The determination device according to claim 2. The predetermined region is a region corresponding to at least part of the superior parietal lobule and the caudate nucleus,
The determination device according to any one of claims 1 to 3. the three-dimensional image is perceived by the user using a predetermined display device;
The predetermined display device comprises a first display unit that displays a 2D image for the user's right eye and a second display unit that displays a 2D image for the left eye of the user.
The determination device according to any one of claims 1 to 4 . The predetermined display device is a wearable display device that is worn on a predetermined site of the user, or a non-wearable display device that is not worn on a predetermined site of the user.
The determination device according to claim 5 . obtaining a brain image of the user;
calculating a gray matter volume of a predetermined region of the user's brain based on the brain image;
determining whether the user has specific cognitive abilities , including receptivity to 3D images, based on the calculated gray matter volume;
A determination method having The calculation unit provided in the determination device,
an acquisition unit that acquires a brain image of a user;
a computing unit that computes a gray matter volume in a predetermined region of the user's brain based on the brain image; A determination unit that determines whether or not it has cognitive ability,
Judgment program that functions as

Images