JP6102526B2 - Moving image generation device for brain function measurement and moving image generation system for brain function measurement - Google Patents

Description

  The present invention relates to a brain function measurement video generation device and a brain function measurement video generation system, and more particularly to a brain function measurement video generation device and a brain function measurement video for confirming measurement results using the brain function measurement device. It relates to the generation system.

  Conventionally, in order to confirm a measurement result using a brain function measurement device, a video of a subject being measured is captured simultaneously with a brain function measurement, and the measurement result and the operation of the subject are collated. (For example, refer to Patent Document 1).

  Patent Document 1 discloses a brain function measurement in which an image of a subject being measured with a video camera is captured simultaneously with a brain function measurement, and the obtained measurement data and the captured image are displayed in synchronization after the brain function measurement is completed. An apparatus is disclosed. The brain function measuring device is an optical measuring device that attaches a light transmitting terminal and a light receiving terminal to a holder attached to the head of a subject and receives measurement light reflected (scattered) in the brain from the light receiving terminal. . The measurement data of the optical measurement device is time-series data of detection signals (measurement light intensity) at a plurality of measurement points. In brain function measurement, a task period in which stimulus is presented to a subject and a rest period in which no stimulus is presented are repeated, and analysis of the measurement result is performed based on the difference in measurement results in each period. Here, for example, if there is a momentary movement of the subject during the measurement, the terminal position of the measuring device is momentarily shifted or the terminal floats from the head, so that noise is mixed into the measurement data. For this reason, the brain function measuring apparatus of Patent Document 1 is configured to arrange the obtained measurement data and video data on the display screen and to reproduce the time series of both data in synchronization after the brain function measurement is completed. Has been. Thereby, when the measurement result (measurement data) is compared with the motion of the subject (captured image) and an abnormal value appears in the measurement data, the abnormal value is noise caused by the motion of the subject being measured. It is possible to verify whether the measured value reflects the brain activity of the subject.

JP 2005-328855 A

  However, since the captured image of the subject is stored in a file storage format of moving image data, it can be reproduced by a general-purpose reference device (such as an information processing device (PC)). Since the data is stored in a dedicated file storage format of a unique format that is different for each manufacturer of the measurement device, in order to refer to the measurement data and the video data in synchronization with each other as in Patent Document 1, a reference device is used. As described above, a device corresponding to the file storage format of both moving image data and measurement data is required. Therefore, in order to refer to both data in the collation work between the brain function measurement results and the motion video of the subject, it is necessary to prepare an information processing device and a brain function measurement device installed with dedicated data reference software (program) There is a problem that the convenience is low for the user.

  The present invention has been made to solve the above-described problems, and one object of the present invention is to reduce the restriction of the reference device in the collation operation between the brain function measurement result and the motion image of the subject. Thus, it is to provide a moving image generating apparatus for brain function measurement and a moving image generating system for measuring brain function that can improve convenience for the user.

In order to achieve the above object, the moving image generating apparatus for brain function measurement according to the first aspect of the present invention is a time-series measurement result data indicating a measurement result related to the brain function of the subject measured by the brain function measuring apparatus , or Among the video data of the measurement result, an input unit that receives input of one of the selected data and the first video of the subject at the time of measurement, and the data input from the brain function measurement device is the measurement result data In this case, the measurement is performed so that the control unit that generates the video data of the measurement result , the measurement timing of the video data of the measurement result , and the first video of the subject at the measurement timing are synchronized with each other. results of the video data and the video processor for editing the first video, and the video recording unit that records a single screen image data edited by the image processing unit as moving image data Provided. The term “synchronization” in the present invention is a concept that includes not only a case where the time is exactly the same but also a case where the time is substantially synchronized. For example, a delay caused by a hardware configuration and information processing, etc. This is a concept that allows an inevitable synchronization shift.

  In the moving image generating apparatus for brain function measurement according to the first aspect of the present invention, as described above, a single screen video data in which the measurement timing of the measurement result and the first video of the subject at the measurement timing are synchronized, and The measurement result is provided by providing a video processing unit that edits the measurement result data and the first video, and a video recording unit that records single screen video data created by the video processing unit as moving image data. The data and the first video of the subject can be combined into a single screen video data, which can be recorded in a file storage format of moving image data that is generally popular. As a result, a general-purpose information processing apparatus in which dedicated software is not installed, or other data that has conventionally had to be referred to by reading data in the respective file storage formats and synchronously reproducing them, and the first video, etc. Can be reproduced (referenced) as moving image data even on a reference device (for example, a reproducing device corresponding to a general moving image file format such as a tablet terminal or a DVD player). Thereby, since the restriction | limiting of the reference apparatus in the collation operation | work with a brain function measurement result and a test subject's motion image | video can be reduced, the convenience for a user can be improved. In addition, since the reference device only needs to handle a single moving image data, the total capacity of the data can be reduced as compared with a case where measurement data and video data are individually prepared as in the past.

  In the above-described brain function measurement moving image generating apparatus according to the first aspect, preferably, the input unit is configured to further receive an input of a second video related to a task at the time of brain function measurement, and the video processing unit The measurement result data, the first video, and the second video are edited so that the measurement timing and the first video and the second video at the measurement timing correspond to each other to be a single screen video data. Has been. With this configuration, by reproducing the video data, not only the brain function measurement result and the motion image of the subject, but also the task information such as whether the current reference time is in the task period or the rest period is simultaneously Since it can be referred to, it is possible to further improve convenience in the collation operation between the brain function measurement result and the motion video of the subject.

  In this case, preferably, the second video related to the task at the time of brain function measurement includes a video of a task display screen that displays the contents of the task presented to the subject during the measurement by the brain function measuring device. With this configuration, in addition to the brain function measurement result and the motion image of the subject, the task such as what task was presented in what mode during the measurement can be obtained by simply reproducing the video data. You can refer to the contents at the same time. Thereby, in the collation operation between the brain function measurement result and the motion video of the subject, it is not necessary to confirm the task contents separately, and more detailed collation operation including the task contents can be performed.

  In the above-described brain function measurement moving image generating apparatus according to the first aspect, preferably, the video processing unit chronologically displays the measurement result display image and the measurement timing of the measurement result display image included in the first video. Corresponding image data for one frame is arranged on the screen and edited as a single frame image data, and screen image data is created from the frame image data for each frame. If comprised in this way, while being able to synchronize a measurement result and a 1st image | video for every frame contained in video data, screen video data can be easily produced from the frame image data for every frame.

  In the above-described moving image generating device for brain function measurement according to the first aspect, preferably, the video processing unit converts the measurement result data acquired through the input unit along with the measurement by the brain function measuring device and the first video in real time. In this way, editing is performed so as to be a single screen image data. With this configuration, moving image data can be recorded simultaneously with the measurement by the brain function measuring device, and after the measurement is completed, the brain function measurement result using the moving image data and the motion image of the subject can be quickly verified. It can be carried out.

  In the moving image generating apparatus for brain function measurement according to the first aspect described above, preferably, the video processing unit obtains corresponding image data in time series from the measurement result data acquired in advance and the data of the first video, respectively. It is configured to be extracted and edited so as to be a single screen video data. If comprised in this way, the measurement result implemented in the past and the data of the 1st picture can be edited as single screen picture data, and can be recorded as animation data. For this reason, for example, even if it is not possible to prepare a brain function measurement video generation device at the site where brain function measurement is performed, the video function data is created afterwards and the brain function measurement results using the video data and the subject's behavior Collation with video can be performed.

The moving image generation system for brain function measurement according to the second aspect of the present invention includes a brain function measuring device that measures a biological signal related to a subject's brain function and generates time-series measurement result data, and images the subject during measurement. The imaging device that generates the first video, and the measurement result data or the video data of the measurement result, one of the selected data and the first video is acquired, and when the acquired data is the measurement result data, the measurement result from generates video data of the measurement results, the measurement result of the measurement timing, the first such image and is synchronized with a single-screen video data to the measurement timing, the measurement result data video data and the first image And a moving image generating device that records the edited single screen image data as moving image data.

  In the moving image generating system for brain function measurement according to the second aspect of the present invention, as described above, the measurement result data and the first video are acquired, and the measurement result measurement timing is synchronized with the first video at the measurement timing. Measurement result data and a first video are edited so as to be a single screen video data, and a video generation device is provided for recording the edited single screen video data as video data. The data and the first video of the subject can be combined into a single screen video data, which can be recorded in a file storage format of moving image data that is generally popular. As a result, a general-purpose information processing apparatus in which dedicated software is not installed, or other data that has conventionally had to be referred to by reading data in the respective file storage formats and synchronously reproducing them, and the first video, etc. It can be reproduced (referenced) as moving image data even with the reference device. Thereby, since the restriction | limiting of the reference apparatus in the collation operation | work with a brain function measurement result and a test subject's motion image | video can be reduced, the convenience for a user can be improved. In addition, since the reference device only needs to handle a single moving image data, the total capacity of the data can be reduced as compared with a case where measurement data and video data are individually prepared as in the past.

  The moving image generating system for brain function measurement according to the second aspect preferably further includes a task presenting device that presents to the subject a second video for displaying the contents of the task during the measurement. The measurement result data, the first video, and the second video are edited so that the measurement timing of the result and the first video and the second video at the measurement timing correspond to each other to be a single screen video data. It is configured. If comprised in this way, the content of the task shown to a test subject during a measurement can be referred simultaneously in addition to a brain function measurement result and a test subject's motion picture only by reproducing animation data. Thereby, in the collation operation between the brain function measurement result and the motion video of the subject, it is not necessary to confirm the task contents separately, and more detailed collation operation including the task contents can be performed.

  According to the present invention, as described above, it is possible to reduce the restriction on the reference device in the collation operation between the brain function measurement result and the motion video of the subject, and to improve the convenience for the user.

1 is a block diagram illustrating an overall configuration of a moving image generating system according to an embodiment of the present invention. It is the schematic diagram which showed an example of the screen image data which a moving image generation system produces | generates. It is the schematic diagram which showed another example of the screen image data which a moving image generation system produces | generates. It is the schematic diagram which showed an example of the measurement result screen of a brain function measuring device. It is a schematic diagram for demonstrating the content of the moving image production | generation process by a moving image production | generation apparatus. It is a flowchart for demonstrating the moving image production | generation process (online process) by a moving image production | generation apparatus. It is a flowchart for demonstrating the moving image production | generation process (offline process) by a moving image production | generation apparatus. It is the schematic diagram which showed the arrangement | positioning image which shows the arrangement position of a probe.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments embodying the present invention will be described below with reference to the drawings.

  First, with reference to FIGS. 1-5, the whole structure of the moving image production | generation system 100 for brain function measurement by one Embodiment of this invention is demonstrated. The moving image generation system 100 is an example of the “brain function measurement moving image generation system” of the present invention.

  As shown in FIG. 1, the moving image generating system 100 for measuring brain function according to the present embodiment includes a brain function measuring device 1, an imaging device 2, a task presenting device 3, and a moving image generating device 4. The moving image generating device 4 is an example of the “brain function measuring moving image generating device” of the present invention.

  First, an outline of brain function measurement by the brain function measuring apparatus 1 will be described. The brain function measurement device 1 is a device (light measurement device) that generates a time-series measurement result data by measuring a biological signal (measurement light intensity) related to the brain function of a subject using measurement light. The brain function measurement apparatus 1 irradiates measurement light in the near infrared wavelength region from a light transmission probe 1a arranged on the head surface of a subject (subject). Then, the measurement light transmitted through the subject (completely transmitted) and / or reflected (reflected in the body) is incident on the light receiving probe 1b disposed on the head surface and detected, thereby measuring light. The intensity (amount of received light) is acquired.

  Here, reflecting the brain activity, if the amount of hemoglobin in the brain increases at the activated site, the amount of measurement light absorbed by hemoglobin increases. For this reason, it is possible to acquire a change in the amount of hemoglobin accompanying brain activity based on the acquired intensity of measurement light. Hemoglobin is divided into oxyhemoglobin bonded to oxygen and deoxyhemoglobin not bonded to oxygen, and the light absorption characteristics are different from each other. Therefore, measurement is performed using measurement light having a plurality of wavelengths (for example, three wavelengths of 780 nm, 803 nm, and 830 nm) in consideration of the difference in light absorption characteristics, and the intensity (light reception amount) of the obtained measurement light of each wavelength is obtained. Based on this, the amount of each hemoglobin and the total amount of hemoglobin are calculated.

  As a result, it is possible to non-invasively acquire a change in hemoglobin amount associated with brain activity, that is, a change in blood flow rate or an activated state of oxygen metabolism, based on the intensity (amount of received light) of the measurement light incident on the light receiving probe 1b. It is. The optical measurement device measures a wide brain region at a plurality of points (measurement channels) using a plurality of light transmitting probes 1a and a plurality of light receiving probes 1b, thereby determining which region of the brain is active. It is possible to acquire a dimensional distribution (map).

  As shown in FIG. 1, an optical fiber cable (hereinafter referred to as an optical fiber) 1c is connected to the light transmitting probe 1a and the light receiving probe 1b, so that measurement light can be emitted or incident at the tip. It is configured. The light transmitting probe 1a and the light receiving probe 1b are attached to a holder 5 for fixing at a predetermined position on the head surface. The user determines the arrangement of the probes according to the part to be measured (the frontal head, the top of the head, the temporal region, the back of the head, etc.) and attaches the probe to the holder 5. The light transmitting probe 1a and the light receiving probe 1b are arranged in the holder 5 so as to be alternately arranged in the row and column directions. Thereby, a measurement channel (measurement point) is formed between the adjacent light transmitting probe 1a and light receiving probe 1b.

  The brain function measurement is usually performed while alternately repeating a task period in which a task corresponding to a measurement purpose (measurement target) is performed and a rest period in which the task is not performed. By comparing the measurement results in both periods, knowledge about brain function can be obtained from changes in brain activity (difference in measured values). Tasks vary depending on the purpose of measurement, such as those that cause the subject to perform some action, or tasks that provide some stimulus to the subject. For example, as shown in FIG. 2, the task is presented by presenting the subject with an image (please say a word starting with “ka”) indicating the task content. On the other hand, in the rest period, as shown in FIG. 3, a method is generally employed in which an image showing only a gazing point (cross mark) is presented on the display screen and only the subject's line of sight is fixed.

  Next, the apparatus configuration of the moving image generation system 100 will be described.

  As shown in FIG. 1, the brain function measuring apparatus 1 includes a light output unit 11, a light detection unit 12, a measurement control unit 13, a main body control unit 14, a communication unit 15, and a main storage unit 16. ing. The brain function measuring apparatus 1 is connected to the display unit 17 and the operation unit 18 and is connected to the light transmitting probe 1a and the light receiving probe 1b via the optical fiber 1c.

  The light output unit 11 is configured to output measurement light to the light transmission probe 1a via the optical fiber 1c. The light output unit 11 includes a semiconductor laser as a light source, and outputs measurement light having a plurality of wavelengths (for example, three wavelengths of 780 nm, 803 nm, and 830 nm) in the near infrared wavelength region. The light detection unit 12 includes a photomultiplier tube as a detector, and is configured to acquire and detect measurement light incident on the light receiving probe 1b through the optical fiber 1c.

  The measurement control unit 13 controls the operation of the light output unit 11 and the light detection unit 12 according to the measurement parameters set by the main body control unit 14, controls the timing of turning on and off the light output unit 11, and detects the light. The received light amount signal is acquired from the unit 12.

  The main body control unit 14 is a computer composed of a CPU, a memory, and the like, and is configured to control the measurement operation of the brain function measuring apparatus 1 by executing various programs stored in the main storage unit 16. Yes. The main body control unit 14 exchanges control signals with the task presenting device 3 via the communication unit 15 to synchronize the task presenting operation of the task presenting device 3 and the measurement operation of the brain function measuring device 1.

  In addition, the main body control unit 14 executes data analysis software (program) to display the obtained measurement data 51 as a waveform for each measurement channel (a time change in measurement values), and each measurement channel. The video (result screen data 52) of the measurement result screen 53 (see FIG. 4) is output to the display unit 17 in the display format of the map display in which the measurement data 51 is displayed in correspondence with the two-dimensional distribution (arrangement position). . The measurement data 51 is time-series data including measurement time (measurement timing) information and measurement value information for each measurement channel, and is configured to be recorded in the main storage unit 16 in a dedicated file storage format. In the present embodiment, the main body control unit 14 also outputs the measurement data 51 or the result screen data 52 to the moving image generation device 4. The measurement data 51 and the result screen data 52 are both examples of the “measurement result data” in the present invention.

  As shown in FIG. 4, in addition to the measurement result display area 53a, the measurement result screen 53 includes various icon areas 53b for performing operation input on the screen, file names of measurement data, various setting information, measurement An information display area 53c such as value information is included. In FIG. 4, the measurement result is displayed in the measurement result display area 53a in a chart display showing the measurement waveform for each measurement channel (1 to 24). In the map display, as shown in FIG. 3, the measurement values of the two-dimensionally distributed measurement channels are represented by hues (indicated by shades of hatching in FIG. 3). The display of these measurement result display areas 53a changes with time as the measurement data is acquired from time to time.

  As illustrated in FIG. 1, the communication unit 15 includes a wired or wireless communication module, and can perform data communication with an external device including the task presentation device 3. The main storage unit 16 is composed of, for example, an HDD (Hard Disk Drive) and stores various programs executed by the main body control unit 14 and can store measurement data obtained as a result of measurement. The display unit 17 is a display device such as a liquid crystal monitor, for example, and the operation unit 18 is an input device including a keyboard and a mouse, for example.

  The imaging device 2 has a function of capturing a subject and generating a camera image 54 at the time of measurement by the brain function measuring device 1. The imaging device 2 is a digital video camera, for example, and includes an imaging unit 21 and a microphone 22. The imaging device 2 is configured to output data (video and audio) of the captured camera video 54 of the subject to the moving image generation device 4. The camera video 54 is an example of the “first video” in the present invention.

  The task presentation device 3 has a function of presenting to the subject a task video 55 that displays the contents of the task during the measurement by displaying a task video 55 related to the task at the time of brain function measurement on the display unit 35. Specifically, the task video 55 is a video of a task display screen that displays the contents of the task presented to the subject during the measurement by the brain function measuring apparatus 1, and the task video 55 is shown in FIG. This is a video that alternately displays an image of “Please say the beginning word” and an image of a rest showing only the point of interest (cross mark). The task video 55 is an example of the “second video” in the present invention.

  The task presentation device 3 is configured by an information processing device (PC), and includes a control unit 31, a storage unit 32, a communication unit 33, and an input / output unit (I / O interface) 34 as shown in FIG. In addition, the display unit 35 and the operation unit 36 are connected.

  The control unit 31 includes a CPU, a memory, and the like, and controls the information processing device to function as the task presentation device 3 by executing task presentation control software (program) installed in the storage unit 32. . Thereby, the control unit 31 is configured to output to the display unit 35 a task video 55 in which the task display screen and the rest screen are alternately displayed during the preset task period and rest period. In the present embodiment, the control unit 31 is configured to output the video data of the task video 55 output to the display unit 35 to the moving image generation device 4.

  The storage unit 32 is composed of, for example, an HDD, stores a program executed by the control unit 31, and stores setting information, image data, and the like for displaying the task video 55. The communication unit 33 includes a wired or wireless communication module, and the control unit 31 exchanges control signals with the brain function measuring device 1 via the communication unit 33. The display unit 35 and the operation unit 36 are input devices including a display device such as a liquid crystal monitor and a keyboard and a mouse, respectively.

  The moving image generating device 4 acquires time-series measurement result data indicating the measurement results of the brain function measuring device 1, data of the camera video 54 of the imaging device 2, and data of the task video 55 of the task presenting device 3, respectively. Each of the acquired data is edited and edited such that the measurement timing of the measurement result and the camera video 54 and task video 55 at the measurement timing are synchronized to a single screen video data 60 (see FIG. 2). The single screen image data 60 is recorded as moving image data 61. Note that the moving image generation device 4 uses the video data (result screen data 52) of the measurement result screen 53 and the measurement data 51 (measurement channel) of the brain function measurement device 1 as measurement result data indicating the measurement result of the brain function measurement device 1. It is possible to handle both measurement time (measurement timing) information and time-series data including measurement value information.

  The moving image generating device 4 is configured by an information processing device (PC), and includes a control unit 41, a storage unit 42, a communication unit 43, and an input / output unit (I / O interface) 44, and a display unit 45 and The operation unit 46 is connected. The control unit 41 includes a CPU, a memory, and the like, and controls the information processing apparatus to function as the moving image generation apparatus 4 by executing various software (programs) installed in the storage unit 42. The control unit 41 and the storage unit 42 are examples of the “video processing unit” and the “video recording unit” of the present invention, respectively. The input / output unit 44 is an example of the “input unit” in the present invention.

  Further, the control unit 41 executes software (program) for moving image generation (editing), thereby obtaining measurement result result screen data 52, data of the camera video 54 of the imaging device 2, and tasks of the task presentation device 3. It functions as a video processing unit for editing data of the video 55. That is, the control unit 41 sets the result screen data such that the measurement timing of the measurement result and the camera video 54 and the task video 55 at the measurement timing correspond to (synchronize) with each other to become a single screen video data 60. 52, the camera video 54 and the task video 55 are edited. When the measurement data 51 is handled, the control unit 41 generates (reproduces) the result screen data 52 from the measurement data 51 and edits the result screen data 52.

  Here, as shown in FIG. 5, each video data of the measurement result result screen data 52, the data of the camera video 54 of the imaging device 2, and the data of the task video 55 of the task presentation device 3 is predetermined per second. Image (still image) data (52a, 54a and 55a) of the number of frames (frame rate, for example, 30 frames / second) and time information (not shown) for each frame. Each of the image data (52a, 54a, and 55a) has predetermined pixel information (such as 2-byte color) at a predetermined resolution (for example, 640 × 480 pixels). Further, the data of the camera video 54 including sound further includes sound data 54b. Accordingly, the image data (52a, 54a and 55a) constituting each video data is input to the moving image generating device 4 by the number of frame rates per second.

  For the video of the result screen data 52, the camera video 54, and the task video 55, the control unit 41 displays image data (52a, 54a, and 55a) for one frame corresponding to the measurement timing of the measurement results in time series. They are arranged so as to be edited as single frame image data 62. The control unit 41 creates a single frame image data 62 for the number of frames included in the video data for the measurement time, and creates a single screen video data 60 from the frame image data 62 for each frame. Is configured to do.

  The editing of the video data is performed by adjusting the size (reducing or reducing) of the image data (52a, 54a and 55a) so that the images of the result screen data 52, the camera video 54, and the task video 55 are arranged in a single screen. Processing such as enlargement), trimming (cutout) of a predetermined area in the screen, and arrangement at a predetermined position. Specifically, it is assumed that image data 52 a (for one frame) of the measurement result screen 53 shown in FIG. 4 is obtained as the result screen data 52 at a certain measurement timing. The control unit 41 trims the measurement result display area 53a (see FIG. 4) from the image data 52a on the measurement result screen 53, and adjusts the size so as to fit in a preset arrangement position with a set dimension. Thereby, in the example of the screen video data 60 (frame image data 62) shown in FIG. 2, the image (image data 52a) of the measurement result display area 53a is displayed large in the left area of the screen.

  Similarly, the control unit 41 adjusts the size of the corresponding image data 54a and 55a (for one frame) of the camera video 54 and the task video 55 so as to fit in the preset layout position. . Thereby, in the example of the screen video data 60 (frame image data 62) shown in FIG. 2, the image of the camera video 54 (image data 54a) is displayed in the upper right area of the screen, and the image of the task video 55 (image data 55a) is displayed. In the lower right area of the screen, each is displayed in substantially the same size smaller than the measurement result display area 53a. Further, the audio data 54b of the camera video 54 becomes the audio data (not shown) of the screen video data 60 as it is. In this way, frame image data 62 is generated. In addition, the arrangement and size of each video in the screen video data 60 are arbitrary, and can be set by a user operation input.

  The control unit 41 simply displays measurement result data (result screen data 52), the camera video 54, and the task video 55 acquired via the input / output unit 44 along with the measurement by the brain function measuring apparatus 1 in real time (online). The screen image data 60 is configured to be edited. In addition, the control unit 41 uses time-series corresponding frame image data (52a, 54a, and 55a) from the measurement result data (measurement data 51) acquired in advance and the data of the camera video 54 and the task video 55, respectively. ) Can be extracted afterwards and edited so as to become a single screen image data 60 offline (non-real time).

  As shown in FIG. 1, the storage unit 42 is composed of, for example, an HDD, and stores programs executed by the control unit 41, setting information for the control unit 41 to edit each video data and create screen video data 60, and the like. I remember it. The storage unit 42 is configured to record (save) the single screen video data 60 edited by the control unit 41 as the moving image data 61. The moving image data 61 is recorded in a file storage format (for example, AVI format or MP4 format) that can be played back by a general-purpose playback device. By transmitting the moving image data 61 via the communication unit 43 or providing it in the form of an information recording medium (DVD or the like) recorded by a reader / writer (not shown), other reference devices (for example, PC, tablet) The screen video data 60 can be referred to by a playback device that supports a general moving image file format, such as a type terminal or a DVD player. The communication unit 43 includes a wired or wireless communication module, and can perform data communication with an external device. The control unit 41 can also acquire the measurement data 51 via the communication unit 43.

  The input / output unit 44 includes time-series measurement result data (result screen data 52 or measurement data 51) indicating a measurement result related to the brain function of the subject measured by the brain function measuring device 1, and a camera image 54 of the subject at the time of measurement. And data of the task video 55 of the task presentation device 3 are received. The input / output unit 44 is an interface such as USB (Universal Serial Bus) or IEEE1394. The display unit 45 and the operation unit 46 are input devices such as a display device such as a liquid crystal monitor and a keyboard and a mouse, respectively.

  The moving image generating system 100 according to the present embodiment is configured as described above. In the present embodiment, the moving image generation system 100 displays the measurement result data (result screen data 52) acquired via the input / output unit 44, the camera video 54, and the task video 55 in a single screen in real time (online). Online processing for editing the video data 60, and previously obtained measurement result data (measurement data 51), camera video 54, and task video 55 data are each offline (non-real-time) single screen video. The moving image data 61 of the screen video data 60 can be created by both offline processing for editing the data 60.

  Next, with reference to FIG. 2, FIG. 3, FIG. 6, and FIG. 7, an outline of the moving image generation processing operation of the moving image generation device 4 in each of the online processing and the offline processing will be described.

  At the time of brain function measurement, the imaging device 2 constantly outputs the data of the camera video 54 to the moving image generation device 4. The brain function measuring apparatus 1 and the task presenting apparatus 3 exchange trigger signals (control signals) for measurement start and measurement end, and synchronize the start and end timing of the measurement operation. During the measurement, the brain function measuring device 1 acquires a trigger signal indicating each start point of the task period and the rest period from the task presentation device 3, and records time information in the measurement data. The video data of the measurement result screen 53 of the brain function measuring device 1 and the data of the task video 55 of the task presenting device 3 are output to the moving image generating device 4 simultaneously with the start of measurement while being synchronized with each other. Since the brain function measurement operation process by the cooperation of the brain function measuring apparatus 1 and the task presenting apparatus 3 is well known, the description thereof is omitted.

  First, the online processing operation will be described. As shown in FIG. 6, in step S1, parameters for recording the moving image data 61 are set. The control unit 41 reads the setting information stored in the storage unit 42, the resolution of the screen video data 60 to be created, the pixel information, the frame rate, and each video in the screen video data 60 (result screen data 52, camera video 54 and task). The arrangement position information of the video 55), the file name of the moving image data 61 to be stored, and the like are set. The control unit 41 also sets measurement data communication conditions (such as an input port number and IP address) and measurement data measurement conditions (such as a sampling interval).

  Subsequently, the control unit 41 waits for data reception via the input / output unit 44 in step S2, and determines whether or not data is received in step S3. If data cannot be received, the control unit 41 returns to step S2 and waits for data reception. When the brain function measuring device 1 and the task presenting device 3 start measurement in synchronization with the user's operation, the video of the measurement result screen 53 (result screen data 52) and the data of the task video 55 are simultaneously received and processed. Advances to step S4.

  In step S4, the control unit 41 checks that the time (time) information of the frame received this time is the same for each of the result screen data 52, the camera video 54, and the task video 55 data. Thereby, it is confirmed that the respective image data (52a, 54a and 55a) of the result screen data 52, the camera video 54, and the task video 55 are obtained in synchronization with the measurement timing of the measurement data. In step S5, the control unit 41 checks the received byte length for each data received this time. That is, the control unit 41 confirms the size (byte length) of the received frame image data and confirms that one frame of image data (52a, 54a, and 55a) is obtained.

  In step S6, the control unit 41 determines that the image data (52a, 54a, and 55a) of the same time frame (same measurement timing) is the result screen data 52, the camera video 54, and the task video from the check results of steps S4 and S5. It is confirmed whether it was obtained for each of 55. When data is insufficient due to a delay caused by the hardware configuration or information processing, the control unit 41 returns to Step S3 and repeats Steps S3 to S6 until necessary data is prepared.

  If the necessary data is available, in step S7, the control unit 41 edits the image data (52a, 54a, and 55a) of the result screen data 52, the camera video 54, and the task video 55 to correspond to one frame. The single screen image data 60 is created. That is, the control unit 41 performs processing such as size adjustment (reduction or enlargement) of each image data (52a, 54a, and 55a), trimming (cutout) of a predetermined area in the screen, and arrangement at a predetermined position. Screen frame image data 62 is created. In step S <b> 8, the control unit 41 stores the created single frame image data 62 for one frame on the memory or in the storage unit 42. When the measurement result screen 53 in the result screen data 52 is a chart display, the display (chart display) shown in the frame image data 62 of FIG. 2 is obtained, and when the measurement result screen 53 is a map display, FIG. The display (map display) shown in the frame image data 62 is obtained. Similarly, when the image data 55a of the task video 55 is a task display video, the display (task display screen) shown in FIG. 2 is obtained. When the image data 55a of the task video 55 is a rest display video, the frame shown in FIG. The image data 62 is displayed (rest screen).

  Subsequently, in step S9, the control unit 41 determines whether or not the brain function measurement is finished. If the brain function measurement is not completed, steps S2 to S8 are repeated, whereby the next one frame of image data (52a, 54a, and 55a) is acquired and a single frame image data 62 of one frame is created. Is done. When the brain function measurement is completed, the output of the video of the measurement result screen 53 and the data of the task video 55 is stopped, and based on this, the control unit 41 determines the end of the measurement. In that case, in step S10, the control unit 41 displays screen image data composed of a set of created single frame image data 62 for one frame (measurement time (seconds) × frame image data 62 corresponding to the number of frame rates). 60 is recorded in the storage unit 42 as one moving image data 61 in a general moving image file storage format. This completes the online (real-time) video data creation.

  Next, the operation of offline processing will be described. In the offline processing, the result screen data 52 (video data of the measurement result screen 53) is not converted into moving image data in the online processing. Therefore, the measurement data 51 of the brain function measuring device 1, the video of the camera video 54, and the task video 55 are displayed. Single screen image data 60 is created from the data and recorded as moving image data 61.

  As shown in FIG. 7, in step S <b> 21, the control unit 41 accepts a user operation and accepts selection of a file stored in advance in the storage unit 42. Here, the video data (moving image file) of the camera video 54 and the task video 55 and the measurement data (data file) of the brain function measuring device 1 are selected. Subsequently, the control unit 41 sets parameters for recording the moving image data 61 in step S22. The contents of the parameter setting are the same as in step S1 of FIG. In step S <b> 23, the control unit 41 reads the data attribute of the selected file (video data or measurement data file type). For the measurement data of the brain function measuring apparatus 1, a program corresponding to the file format of the measurement data is executed to generate a video of the measurement result screen 53 (result screen data 52), and image data for one frame (52a, 52a, 54a and 55a) are extracted. As for video data (moving image file), one frame of image data (52a, 54a and 55a) is extracted by a moving image editing program.

  In step S24, the control unit 41 reads image data (52a, 54a, and 55a) for one frame from the selected video data (moving image file). In step S25, the control unit 41 reads measurement data at the same time as the frame read in step S24 from the selected measurement data 51 file.

  Subsequently, in step S26, the control unit 41 generates and edits the video of the measurement result display area 53a of the measurement result screen 53 from the measurement data 51, and edits the respective frame image data of the camera video 54 and the task video 55. The screen image data 60 for one frame is created by editing. The editing contents are the same as in step S7 in FIG. In step S <b> 27, the control unit 41 stores the created single frame image data 62 for one frame on the memory or in the storage unit 42.

  Subsequently, in step S28, the control unit 41 determines whether the creation and storage of the frame image data 62 for the total number of frames of the selected file (video data) has been completed. By repeating steps S24 to S28, screen image data 60 is created. When the creation and storage of the screen image data 60 is completed, in step S29, the control unit 41 records the created screen image data 60 in the storage unit 42 as one moving image data 61 in a general-purpose moving image file storage format. Thereby, the offline video data creation ends.

  In the present embodiment, as described above, the measurement result data and the camera video 54 are set so that the measurement timing of the measurement result and the camera video 54 of the subject at the measurement timing are synchronized to a single screen video data 60. And a storage unit 42 that records the single screen video data 60 created by the control unit 41 as the moving image data 61, thereby providing time-series measurement result data ( The measurement data 51 or the result screen data 52) and the subject's camera video 54 can be combined into a single screen video data 60, which can be recorded in a file storage format of the generally popular moving image data 61. As a result, the measurement result and the camera video 54 can be reproduced (referenced) as the moving image data 61 by a general-purpose information processing apparatus in which dedicated software is not installed or other reference devices. Thereby, since the restriction | limiting of the reference apparatus in the collation operation | work with a brain function measurement result and a test subject's motion image | video can be reduced, the convenience for a user can be improved. Further, since the reference device only needs to handle the single moving image data 61, the total capacity of the data can be reduced as compared with the case where the measurement data and the video data are individually prepared as in the conventional case. .

  Further, in the present embodiment, as described above, measurement is performed so that the measurement timing of the measurement result and the camera video 54 and task video 55 at the measurement timing correspond to each other and become a single screen video data 60. The control unit 41 is configured to edit the result data, the camera video 54, and the task video 55. Thus, only by reproducing the moving image data 61, not only the brain function measurement result and the motion image of the subject, but also the task information such as whether the current reference time is in the task period or the rest period can be referred to at the same time. Therefore, it is possible to further improve the convenience in the collation operation between the brain function measurement result and the motion video of the subject.

  Further, in the present embodiment, as described above, the task video 55 related to the task at the time of brain function measurement is displayed on the task display screen that displays the contents of the task presented to the subject during the measurement by the brain function measurement device 1. Let it be a video. As a result, if the moving image data 61 is reproduced, it is possible to simultaneously refer to the task content in addition to the brain function measurement result and the motion video of the subject. In the work, it is not necessary to confirm the contents of the task separately, and a more detailed collation work including the contents of the task can be performed.

  In the present embodiment, as described above, the control unit 41 measures the image data 52a of the measurement result screen 53 while being included in the image data 52a (result screen data 52) of the measurement result screen 53 and the camera video 54. The image data 54a corresponding to one frame corresponding to the timing (frame) in time series is arranged on the screen and edited as a single frame image data 62, and the screen image data 60 is converted by the frame image data 62 for each frame. Configure to create. As a result, the result screen data 52 and the camera video 54 can be synchronized in units of frames included in the video data, and the screen video data 60 can be easily created.

  In the present embodiment, as described above, the control unit 41 uses the measurement result data acquired through the input / output unit 44 along with the measurement by the brain function measuring apparatus 1 and the camera image 54 in real time. The screen image data 60 is configured to be edited. Thereby, the moving image data 61 can be recorded simultaneously with the measurement by the brain function measuring device 1, and after the measurement is completed, the brain function measurement result using the moving image data 61 and the motion image of the subject are quickly collated. be able to.

  In the present embodiment, as described above, the control unit 41 extracts the corresponding image data 52a and 54a in time series from the previously acquired measurement data 51 and the data of the camera video 54, respectively. Thus, editing is performed so that the single screen image data 60 is obtained. Thus, for example, even when the moving image generating device 4 cannot be prepared at the site where brain function measurement is performed, the moving image data 61 is created afterwards, and the brain function measurement result using the moving image data 61 and the action of the subject Collation with video can be performed.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is shown not by the above description of the embodiments but by the scope of claims for patent, and further includes all modifications within the meaning and scope equivalent to the scope of claims for patent.

  For example, in the above-described embodiment, an example in which the moving image generating device 4 is configured by an information processing device (PC) has been shown, but the present invention is not limited to this. In the present invention, a moving image generating device may be configured by the brain function measuring device 1. That is, by installing a program for causing the main memory unit 16 of the brain function measuring device 1 to function as a moving image generating device and causing the main body control unit 14 to execute the program, the brain function measuring device 1 functions as a moving image generating device. You may comprise. In this case, it is not necessary to separately provide an information processing device as the moving image generating device 4, and the moving image generating system 100 may be configured by the brain function measuring device 1, the imaging device 2, and the task presenting device 3 that also function as the moving image generating device. it can. Similarly, the moving image generating device may be configured by the task presenting device 3.

  Moreover, in the said embodiment, as shown in FIG.2 and FIG.3, each image of the result screen data 52 (measurement result display area 53a), the camera image | video 54, and the task image | video 55 is arranged one by one, Although the example which comprised the single screen image data 60 (frame image data 62) was shown, this invention is not limited to this. In the present invention, for example, a single screen video data may be constituted by only two images of the result screen data 52 (measurement result display area 53a) and the camera video 54. In addition, for example, by using a plurality of imaging devices, camera images obtained by imaging a subject from a plurality of directions are obtained, and in addition to the result screen data and task images, these plurality of camera images are converted into a single screen image data. May be included.

  In addition, video or images other than the result screen data 52, the camera video 54, and the task video 55 may be included in a single screen video data. For example, as shown in FIG. 8, an arrangement image 156 showing the arrangement positions of the light transmitting probe 1a and the light receiving probe 1b with respect to the head (holder 5) of the subject may be included in a single screen image data. In the arrangement image 156, a numbered annular mark indicates a light transmitting point (light transmitting probe 1a), and a numbered circular mark indicates a light receiving point (light receiving probe 1b). . A columnar mark connecting between the light transmitting point and the light receiving point represents the measurement channel C. In the arrangement image 156 of FIG. 8, 24 measurement channels C are formed by the eight light transmitting probes 1a and the light receiving probes 1b, respectively, and are arranged in a rectangular shape as a whole around the top of the subject's head. Is displayed. The arrangement image 156 is displayed according to the arrangement position of the probe. For example, when the probes are respectively arranged on the left and right temporal regions, two display areas indicating the probe arrangement positions are displayed in the arrangement image 156 near the left and right temporal regions. By including the arrangement image 156 in the screen video data in this way, it is possible to grasp at a glance on the screen video data which part of the display of the result screen data 52 indicates the measurement result.

  Moreover, in the said embodiment, although the example using the brain function measuring device 1 which consists of an optical measuring device was shown, this invention is not limited to this. As the brain function measuring device, an electroencephalograph may be used in addition to the optical measuring device. Since an electroencephalograph attaches an electrode to the subject's head, if there is an instantaneous movement of the subject during measurement, noise will be mixed into the measurement data, so the measurement result (measurement data) and the action of the subject (captured image) Need to be matched. Therefore, even when an electroencephalograph is used as the brain function measuring device, it is possible to reduce the restriction of the reference device in the collation work between the brain function measurement result and the motion image of the subject, thereby improving the convenience for the user. be able to.

  In the above embodiment, as an example of the task video 55, an example in which a video for instructing speech (please say a word starting with “ka”) has been described, but the present invention is not limited to this. Since various tasks are set according to the purpose of measurement, any task content displayed as a task video may be used.

  In the above embodiment, online processing for editing video data and result screen data so as to become a single screen video data 60 in real time (online), and offline (non-real time) video data and measurement data acquired in advance. ), The example in which the moving image generating device 4 is configured so as to be able to perform both offline processing for editing so as to become the single screen video data 60 is shown, but the present invention is not limited to this. In the present invention, the moving image generating device 4 may be configured to perform only one of online processing and offline processing.

  In the above embodiment, an example is shown in which the input of each video data of the result screen data 52, the camera video 54, and the task video 55 is received by the input / output unit 44 having an interface such as USB and IEEE1394. However, the present invention is not limited to this. For example, each video data input may be received by wireless communication via the communication unit 43. The input / output unit 44 may be other interface such as HDMI (registered trademark) other than USB and IEEE1394.

  Moreover, in the said embodiment, although the control signal was exchanged between the brain function measuring apparatus 1 and the task presentation apparatus 3, and the example comprised so that it might operate | move synchronously was shown, this invention is limited to this. Absent. In the present invention, in addition to the brain function measuring device 1 and the task presenting device 3, the imaging device 2 and the moving image generating device 4 may be configured to exchange control signals (trigger signals). In this case, the measurement timing for creating the moving image data 61 can be easily synchronized with high accuracy.

DESCRIPTION OF SYMBOLS 1 Brain function measuring device 2 Imaging device 3 Task presentation apparatus 4 Movie production | generation apparatus (Movie production | generation apparatus for brain function measurement)
41 Control unit (video processing unit)
42 Storage unit (video recording unit)
44 Input / output unit (input unit)
51 Measurement data (Measurement result data)
52 Result screen data (Measurement result data)
54 Camera image (first image)
55 Task video (second video)
60 screen image data 61 movie data 62 frame image data 100 movie generation system (movie generation system for brain function measurement)

Claims (8)

Either time-series measurement result data indicating measurement results related to the brain function of the subject measured by the brain function measurement device , or one of the selected data from the video data of the measurement result, and the number of the subject at the time of measurement An input unit for receiving input with one video;
When the data input from the brain function measuring device is the measurement result data, a control unit that generates video data of the measurement result;
The measurement timing of the video data of the measurement result, the as with the first image of a subject is synchronized single screen image data is in the measurement timing, and the image data of the measurement result and said first image A video processor to edit;
A moving image generating apparatus for brain function measurement, comprising: a video recording unit that records a single piece of the screen video data edited by the video processing unit as moving image data. The input unit is configured to further receive an input of a second video related to a task during brain function measurement,
The measurement result data, so that the video processing unit is a single screen video data in a state where the measurement timing of the measurement result and the first video and the second video at the measurement timing correspond to each other, The moving image generating apparatus for brain function measurement according to claim 1, configured to edit the first video and the second video.   3. The brain according to claim 2, wherein the second video related to a task at the time of brain function measurement includes a video of a task display screen that displays a content of a task presented to a subject during measurement by the brain function measuring device. Movie generator for function measurement.   The video processing unit arranges, on the screen, the display image of the measurement result and the image data for one frame included in the first video and corresponding to the measurement timing of the display image of the measurement result in time series. The brain function measurement according to any one of claims 1 to 3, wherein the screen function data is configured to be edited as single frame image data and to generate the screen video data from the frame image data for each frame. Movie generator.   The video processing unit edits the measurement result data and the first video acquired through the input unit in accordance with the measurement by the brain function measuring device so as to be a single screen video data in real time. The moving image generating apparatus for brain function measurement according to claim 1, configured as described above.   The video processing unit extracts time-series corresponding image data from the previously acquired measurement result data and the first video data, respectively, and edits the single screen video data. The moving image generating apparatus for brain function measurement according to any one of claims 1 to 5, configured as described above. A brain function measuring device that measures biological signals related to the brain function of the subject and generates time-series measurement result data; and
An imaging device for imaging a subject and generating a first video during measurement;
One of the measurement result data or the measurement result video data and the first video selected are acquired. When the acquired data is the measurement result data, the measurement result video data is obtained from the measurement result. And the video data of the measurement result data and the first video so that the measurement timing of the measurement result and the first video at the measurement timing are synchronized to a single screen video data. A moving image generating system for brain function measurement, comprising: a moving image generating device that edits and records the edited single screen image data as moving image data. A task presentation device for presenting to the subject a second video displaying the task content during the measurement;
The moving image generation device includes the measurement result data so that the measurement timing of the measurement result and the first video and the second video at the measurement timing correspond to each other in a single screen video data. The moving image generating system for brain function measurement according to claim 7, configured to edit the first video and the second video.

Images