WO2018179346A1 - Manipulation system, server, control method in manipulation system, and program - Google Patents

Manipulation system, server, control method in manipulation system, and program Download PDF

Info

Publication number
WO2018179346A1
WO2018179346A1 PCT/JP2017/013623 JP2017013623W WO2018179346A1 WO 2018179346 A1 WO2018179346 A1 WO 2018179346A1 JP 2017013623 W JP2017013623 W JP 2017013623W WO 2018179346 A1 WO2018179346 A1 WO 2018179346A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
electroencephalogram
command
database
operation system
Prior art date
Application number
PCT/JP2017/013623
Other languages
French (fr)
Japanese (ja)
Inventor
哲也 田辺
Original Assignee
本田技研工業株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 本田技研工業株式会社 filed Critical 本田技研工業株式会社
Priority to PCT/JP2017/013623 priority Critical patent/WO2018179346A1/en
Publication of WO2018179346A1 publication Critical patent/WO2018179346A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer

Definitions

  • the present invention relates to an operation system using a brain wave, a server, a control method in the operation system, and a program.
  • Patent Document 1 describes that a load on an operator's work is determined based on information detected by a biological information sensor, and a work environment is controlled based on the determination result.
  • Patent Document 2 describes changing a control parameter of a two-wheeled vehicle in accordance with a human eye movement.
  • Patent Document 3 describes that a time difference between an acquired electroencephalogram signal and an electrocardiogram signal is calculated and a driver's psychological state is determined based on the calculated time difference.
  • Patent Document 4 describes that a computer or a game machine is operated using an electroencephalogram.
  • Systems and devices that use human biological information can also be applied to vehicles such as motorcycles.
  • a vehicle such as a two-wheeled vehicle, for example, it is extremely difficult for a passenger who is a driver to operate the device while traveling. It is conceivable to apply a system or apparatus using human biological information to the above-described case while traveling to enable operation of the apparatus while traveling.
  • Patent Document 1 describes a helmet equipped with an electroencephalogram sensor and a blink sensor, and describes a control configuration in accordance with the driver's fatigue level.
  • Patent Document 2 describes a helmet equipped with an eye camera, and describes a control configuration corresponding to a difference in rider's attention level.
  • Patent Document 3 describes a helmet equipped with an electroencephalogram sensor and an eye camera, and describes a control configuration according to the driver's degree of tension.
  • Patent Document 4 describes that a computer or a game machine is operated using an electroencephalogram as an electrical signal.
  • An object of the present invention is to provide an operation system, a server, a control method in the operation system, and a program that allow an occupant to operate the apparatus without causing an operation operation of the apparatus using brain waves.
  • An operation system includes a database for accumulating brain wave data, a generation unit that generates correlation data associated with a command for operating the apparatus based on the brain wave data accumulated in the database, Obtaining means for obtaining electroencephalogram data corresponding to the brain wave signal of the passenger, and sending means for sending the command based on the electroencephalogram data obtained by the obtaining means and the correlation data generated by the generating means; And a control means for operating the apparatus when the command is sent by the sending means.
  • the server includes a database for accumulating brain wave data, a generation means for generating correlation data associated with a command for operating the apparatus based on the brain wave data accumulated in the database, and a vehicle Acquiring means for acquiring the electroencephalogram data corresponding to the brain wave signals of the passengers, and transmitting means for transmitting the command based on the electroencephalogram data acquired by the acquiring means and the correlation data generated by the generating means It is characterized by providing.
  • control method corresponds to a generation step of generating correlation data associated with a command for operating the apparatus based on the electroencephalogram data stored in the database, and an electroencephalogram signal of a vehicle occupant
  • the program according to the present invention corresponds to a generation step of generating correlation data associated with a command for operating the apparatus based on the electroencephalogram data stored in the database, and an electroencephalogram signal of a vehicle occupant.
  • the command is transmitted in the transmission step, When transmitted, the computer is caused to execute a control step of operating the apparatus.
  • FIG. 1 is a diagram showing an overall configuration of an operation system using an electroencephalogram in the present embodiment.
  • the operation system according to the present embodiment can transmit an imaging command to a camera provided in the helmet of the passenger based on brain wave data transmitted from the passenger of the moving vehicle.
  • the operation system 100 includes motorcycles 101, 102, 103, radio base stations 104, 105, 106, a network 107, and a server 108.
  • the motorcycle 101 is a so-called saddle riding type vehicle, and includes a three-wheeled vehicle in addition to a two-wheeled vehicle.
  • Wireless communication is performed between each of the motorcycles 101 to 103 and the wireless base stations 104 to 106, and communication via the network 107 is performed between the wireless base stations 104 to 106 and the server 108.
  • the network 107 is, for example, the Internet or a dedicated communication network, and may partially include a telephone communication network such as a mobile phone.
  • the wireless base stations 104 to 106 are wireless communication devices that are installed in public facilities such as traffic lights and communicate with the motorcycles 101 to 103, and are configured as wireless base stations of a mobile phone communication network, for example.
  • the radio base stations 104 to 106 are configured in predetermined area units, for example. Each rider of the motorcycles 101 to 103 wears helmets 109, 110, and 111.
  • the motorcycles 101 to 103 are equipped with controllers 112, 113, and 114 for realizing the operation of the operation system 100.
  • the controllers 112 to 114 include the helmets 109 to 111, the radio base stations 104 to 106, and the like. Communicate between.
  • FIG. 1 three motorcycles 101 to 103 and three radio base stations 104 to 106 are shown. However, the number is not limited to the number shown in the figure, and a plurality of motorcycles are one radio base station. Can be supported.
  • FIG. 1 only one server 108 is shown, but it may be configured by a plurality of devices. The server 108 may be configured as a cloud.
  • the controller 112 of the motorcycle 101 can operate as a mobile terminal.
  • the controller 112 may associate, for example, the contractor ID of the rider's mobile terminal with the own device by setting via the user interface screen from the rider.
  • the server 108 may make it possible to use the rider's personal information and service information managed by the mobile phone system.
  • the motorcycle 101, the helmet 109, the controller 112, and the wireless base station 104 will be described as representative examples of the motorcycle 101 to 103, the helmet 109 to 111, the controller 112 to 114, and the wireless base station 104 to 106.
  • FIG. 2 is a diagram showing a communication network in the operation system 100.
  • FIG. 2 conceptually shows that a data communication network is included.
  • communication 201 is performed between the helmet 109 and the controller 112.
  • the communication 201 includes the electroencephalogram data transmitted from the helmet 109 and the imaging command transmitted from the controller 112.
  • communication 202 is performed between the controller 112 and the radio base station 104.
  • the communication 202 includes brain wave data and driving information of the motorcycle 101.
  • the driving information of the motorcycle 101 is speed information of the motorcycle 101, for example.
  • Communication 203 is performed between the wireless base station 104 and the server 108.
  • the radio base station 104 operates to relay between the controller 112 and the server 108, and the communication 203 includes brain wave data and driving information of the motorcycle 101.
  • FIG. 3 is a diagram illustrating an example of a block configuration of the helmet 109.
  • the helmet 109 is configured with a sensor group 304 for acquiring rider's biological information.
  • the sensor group 304 includes an electroencephalogram sensor that detects a rider's electroencephalogram signal.
  • the camera 305 is attached to the helmet 109 and photographs a front subject (for example, a landscape). In the operation system 100, the camera 305 captures a subject in accordance with a capturing command from the controller 112.
  • the eye tracking camera 306 is a camera for acquiring rider's line-of-sight information. Sensor signals from the sensor group 304 and imaging data from the camera 305 and the eye tracking camera 306 are transmitted to the control board 300.
  • the image memory 308 stores imaging data captured by the camera 305 and brain wave data at the time of imaging.
  • the image memory 308 is, for example, an SD card, and is configured so that it can be attached to and removed from the memory I / F 307.
  • the controller I / F 309 operates as an interface for performing communication with the controller 112, and transmits, for example, a sensor signal from the sensor group 304 to the controller 112. Note that communication with the controller 112 via the controller I / F 309 may be performed wirelessly or may be performed by wire.
  • the control board 300 is configured inside the helmet 109, for example.
  • a processor 301, a memory 302, a storage unit 303, a memory I / F 307, and a controller I / F 309 are configured on the control board 30, and are connected to each other via a bus 310 so as to communicate with each other.
  • the processor 301 reads out the program stored in the storage unit 303 to the memory 302 and executes it, and can control each block in the helmet 109 in an integrated manner.
  • FIG. 4 is a diagram illustrating an example of a block configuration of the controller 112.
  • the controller 112 may be configured separately from the controller including the ECU of the motorcycle 101 or may be configured as a part.
  • the display unit 404 is a display panel, for example, and displays a user interface screen such as a photo selection screen described later.
  • the GPS 405 is a GPS oscillator and is used for acquiring position information of the motorcycle 101.
  • the controller I / F 406 operates as an interface for performing communication with the helmet 109.
  • the I / F 407 operates as an interface for performing communication with the radio base station 104.
  • An image memory 308 removed from the helmet 109 can be attached to the memory I / F 408. Accordingly, the processor 401 can display a list of shooting data stored in the image memory 308 on the display unit 404.
  • the 4 further includes a processor 401, a memory 402, and a storage unit 403, and each block is connected to be communicable with each other via a bus 409.
  • the processor 401 reads out the program stored in the storage unit 403 to the memory 402 and executes it, and can comprehensively control each block on the control board.
  • the processor 401 can cooperate with the ECU of the motorcycle 101, and can control each block on the control board in accordance with, for example, the speed information of the motorcycle 101. Control can be requested. Further, the processor 401 can execute an algorithm used as AI (Artificial Intelligence).
  • AI Artificial Intelligence
  • FIG. 5 is a diagram illustrating an example of a block configuration of the radio base station 104.
  • the I / F 504 operates as an interface for performing communication with the controller 112.
  • a network (NW) I / F 505 operates as an interface for communicating with the server 108 via the network 107.
  • Each block shown in FIG. 5 includes a processor 501, a memory 502, and a storage unit 503, and is connected to be communicable with each other via a bus 506.
  • the processor 501 reads the program stored in the storage unit 503 into the memory 502 and executes it, and controls the radio base station 104 in an integrated manner.
  • the processor 501 converts a communication procedure with the controller 112 and a communication procedure with the server 108.
  • FIG. 6 is a diagram illustrating an example of a block configuration of the server 108.
  • the NWI / F 604 operates as an interface for communicating with the radio base station 104 via the network 107.
  • a database (DB) 605 stores brain wave data associated with the captured image data.
  • the DB 605 will be described later.
  • the DB 606 stores captured image data.
  • Each block shown in FIG. 6 includes a processor 601, a memory 602, and a storage unit 603, and is connected to be communicable with each other via a bus 607.
  • the processor 601 reads the program stored in the storage unit 603 into the memory 602 and executes it, and controls the server 108 in an integrated manner. Further, the processor 601 can execute an algorithm used as an AI.
  • DB 605 and DB 606 are shown as separate databases, but may be configured as an integrated database.
  • FIG. 7 is a view showing the appearance of the helmet 109.
  • a plurality of electrodes 701 corresponding to the electroencephalogram sensor 308 are configured from the top of the helmet 109 along the frontal and occipital regions and from the top of the helmet 109 along the temporal region.
  • a 20-channel EEG signal may be detected by 20 electrodes.
  • the electrode 701 detects an electrical signal of a predetermined frequency band component that appears on the scalp according to activity in the brain, for example, emotion, and transmits the detected electrical signal to the control board 704.
  • the control board 704 corresponds to the control board 300 in FIG.
  • the control board 704 converts the potential signal into digital data by an amplifier (not shown) and an AD converter, and transmits the digital data to the controller 112 as brain wave data corresponding to the brain wave signal.
  • the electroencephalogram data is acquired as a set of data corresponding to the plurality of electrodes 701 as, for example, discrete data of time and potential ( ⁇ V) acquired at a predetermined sampling period.
  • the camera 702 corresponds to the camera 305 in FIG. As described with reference to FIG. 3, the camera 702 shoots a subject according to a shooting command from the controller 112 without the rider's instruction and without causing the rider to perform a shooting operation. As shown in FIG. 3, the camera 702 is configured in front of the helmet 109 so as to generally capture the direction of the rider's line of sight.
  • the eye tracking camera 703 corresponds to the eye tracking camera 306 in FIG.
  • the eye tracking camera 306 photographs the periphery of the rider's eyes.
  • other configurations may be used as long as it is possible to acquire rider's line-of-sight information.
  • an eye tracking sensor that touches a portion corresponding to the pad portion of the glasses and acquires line-of-sight information by a change in the eye muscle resistivity may be used.
  • the DB 606 stores shooting data taken by a motorcycle rider who enjoys the service of the operation system 100 with the camera 702 of the helmet 109.
  • each imaging data is associated with brain wave data at the time of imaging and stored in the DB 605. That is, the DBs 605 and 606 are constructed as a big database of combinations of imaging data and electroencephalogram data transmitted from each rider.
  • the processor 601 of the server 108 acquires the electroencephalogram data corresponding to each electrode 701 transmitted from the controller 112 in S611, in order to acquire a continuous potential signal from the discrete data of time and potential, S612 Smoothing is performed at.
  • the processor 601 performs spectrum analysis on the smoothed potential signal.
  • the processor 601 extracts two-dimensional feature values of each frequency band and each electrode 701, and stores the extracted feature values in the DB 605. To do.
  • FIG. 8 is a diagram showing a processing sequence of the operation system 100 in the present embodiment.
  • the controller 112 may transmit the activation information of the helmet 109 and the driving information of the motorcycle 101 to the server 108 via the wireless base station 104.
  • the server 108 receives the information, the server 108 starts providing the operation system 100 service to the motorcycle 101.
  • the wireless base station 104 that relays data is not shown.
  • electroencephalogram data is transmitted from the helmet 109 to the controller 112.
  • imaging data of the eye tracking camera 306 is transmitted as line-of-sight information.
  • the controller 112 transmits driving information of the motorcycle 101 to the server 108 together with the electroencephalogram data and line-of-sight information.
  • the drive information is speed information of the motorcycle 101, for example.
  • the server 108 determines whether to transmit an imaging command based on the electroencephalogram data transmitted from the controller 112.
  • the DBs 605 and 606 are constructed as a big database of a combination of imaging data and brain wave data transmitted from each rider.
  • the imaging data captured by the camera 305 and the electroencephalogram data at the time of imaging are transmitted from each rider as needed, and the DBs 605 and 606 are updated.
  • the camera 305 is not photographed by the rider's photographing operation.
  • the rider selects what he thinks is good from the plurality of photographed data on the photograph selection screen displayed on the display unit 404 of the motorcycle 101. That is, the DBs 605 and 606 store imaging data and brain wave data that each rider thinks are really good.
  • the processor 601 of the server 108 analyzes the feature quantity of the electroencephalogram data stored in the DB 605, learns the similarity of the electroencephalogram data, and acquires the feature quantity representing the similar pattern. Then, the processor 601 holds the acquired feature amount representing the similar pattern in a predetermined storage area of the server 108 as correlation data corresponding to the shooting event.
  • the processor 601 of the server 108 determines whether or not to send the imaging command by comparing the feature quantity of the electroencephalogram data transmitted from the controller 112 with the correlation data. When it is determined that the feature amount of the electroencephalogram data transmitted from the controller 112 is within a predetermined similar range to the correlation data, it is determined that an imaging command is transmitted.
  • the server 108 transmits the shooting command to the controller 112 in S804.
  • the controller 112 transmits a shooting command from the camera 305 of the helmet 109 to the helmet 109.
  • the processor 301 of the helmet 109 performs shooting with the camera 305.
  • the captured imaging data is stored in the image memory 308 together with the electroencephalogram data at the time of imaging.
  • the feature quantities of the electroencephalogram data that each rider feels as good shooting timing are analyzed, the feature quantities that become similar patterns are learned, and the correlation data corresponding to the shooting event is obtained. Save it.
  • the electroencephalogram data newly transmitted to the server 108 and the correlation data are within a predetermined similar range, it is determined that the rider currently feels that the imaging timing is good and an imaging command is transmitted. .
  • the rider can perform shooting at a desired timing without causing a shooting operation.
  • the rider After shooting in S806, for example, it is assumed that the rider gets off the motorcycle 101 at a break point. Therefore, the rider removes the image memory 308 of the helmet 109 from the memory I / F 307 and attaches it to the memory I / F 408 of the controller 112.
  • the processor 401 displays a list of shooting data stored in the image memory 308 on the display unit 404 as a photo selection screen.
  • FIG. 11 is a diagram showing an example of the photo selection screen 1101. As shown in FIG. 11, a plurality of thumbnail images 1102 are displayed. A thumbnail image 1102 corresponds to shooting data stored in the image memory 308. The thumbnail image 1102 is displayed with date / time information 1104 added thereto. On the photo selection screen 1101, a rider's selection can be accepted by a check box 1103. In addition, when a large number of shooting data is stored in the image memory 308, it cannot be displayed on one screen. In that case, the previous page or the subsequent page can be displayed by the transition button 1105.
  • the rider can shoot at a desired timing without causing a shooting operation.
  • due to factors such as the angle of the camera 702 it is not known whether all the photographed data is truly desirable. Therefore, in this embodiment, by accepting a rider's selection on the photo selection screen 1101, only truly desirable shooting data can be specified.
  • step S807 the controller 112 accepts selection of shooting data on the photo selection screen 1101. This is not shown in FIG. 11, but the selection may be confirmed by a confirmation button or the like.
  • step S ⁇ b> 808 the controller 112 transmits the selected imaging data and electroencephalogram data at the time of imaging to the server 108.
  • the server 108 registers the transmitted imaging data and the electroencephalogram data at the time of imaging in the DB 605.
  • step S809 the server 108 analyzes the feature amounts including the brain wave data newly registered in step S808, and learns the feature amounts that are similar patterns.
  • step S810 the server 108 corrects the correlation data corresponding to the shooting event based on the feature amount obtained by learning.
  • the feature amount learning in step S809 may be performed periodically at predetermined time intervals, or may be performed every time shooting data is accumulated every predetermined number of sheets.
  • the shooting data may be selected by other configurations.
  • the shooting data may be selected on the cloud.
  • the imaging data and brain wave data are transmitted to the server 108.
  • the transmission of imaging data and electroencephalogram data may be performed directly to the server 108 or may be performed via the controller 112.
  • the rider accesses the storage location of the shooting data of the server 108 from his / her portable terminal.
  • the storage location at that time may be a photo database different from the DBs 605 and 606.
  • a photo selection screen 1101 as shown in FIG. 11 is displayed, and the rider performs a selection operation on the photo selection screen 1101.
  • the server 108 stores the imaging data and brain wave data selected in the photo database in the DB 605.
  • FIG. 9 is a flowchart showing processing of the control board 300 in the helmet 109.
  • the processor 301 determines whether or not the automatic shooting mode is set. The determination in S901 may be performed by, for example, a hard switch provided on the control board 300, or may be performed based on an instruction indicating that one of the modes is received from the controller 112. If it is determined in S901 that the mode is not the automatic shooting mode, the process of FIG. 9 is terminated. If it is determined that the mode is the automatic shooting mode, the process proceeds to S902.
  • the processor 301 converts an electrical signal (electroencephalogram signal) detected by the electrode 701 into electroencephalogram data by an amplifier and an AD converter, and transmits the electroencephalogram data to the controller 112. Then, the processor 301 transmits imaging data obtained by the eye tracking camera 306 to the controller 112 as line-of-sight information.
  • an electrical signal electroencephalogram signal
  • the processor 301 transmits imaging data obtained by the eye tracking camera 306 to the controller 112 as line-of-sight information.
  • step S ⁇ b> 903 the processor 301 determines whether a shooting command has been received from the controller 112. If it is determined that the imaging command has not been received, the process of S902 is repeated. On the other hand, if it is determined that a shooting command has been received, the processor 301 performs shooting with the camera 305 in step S904. In step S905, the processor 301 stores the imaging data in the image memory 308 together with the electroencephalogram data at the time of imaging. After S905, the processing from S901 is repeated.
  • FIG. 10 is a flowchart showing the processing of the controller 112.
  • the processing in FIG. 10 starts when the image memory 308 is attached to the memory I / F 408.
  • the processor 401 acquires imaging data and brain wave data stored in the image memory 308, and displays a photo selection screen 1101 on the display unit 404.
  • the processor 402 selects shooting data based on the selection result of the check box 1103 on the photo selection screen 1101.
  • the processor 401 transmits the selected imaging data and brain wave data to the server 108 and registers them in the DB 605 of the server 108. Then, the process of FIG. 10 is complete
  • FIG. 12 is a flowchart showing the processing of the server 108 before learning the electroencephalogram data. If the amount of electroencephalogram data stored in the DB 605 of the server 108 is not sufficient, correlation data cannot be generated by learning feature amounts of similar patterns. In that case, the determination is performed on the electroencephalogram data transmitted from the controller 112 according to a predetermined determination condition. For example, when the potential represented by the electroencephalogram data is included in a predetermined activation range, the electroencephalogram data is determined to satisfy the determination condition, and an imaging command is transmitted.
  • a predetermined determination condition For example, when the potential represented by the electroencephalogram data is included in a predetermined activation range, the electroencephalogram data is determined to satisfy the determination condition, and an imaging command is transmitted.
  • the processor 601 receives drive information.
  • the drive information here is the drive information transmitted from the controller 112 in S802 of FIG.
  • the processor 601 determines whether or not the drive information satisfies a condition. For example, the processor 601 determines whether or not the speed information of the motorcycle 101 is equal to or higher than a predetermined speed. If it is determined that the drive information satisfies the condition, the processes of S1203 and S1204 are executed. If it is determined that the drive information does not satisfy the condition, the process of FIG. In S1203, the processor 601 receives brain wave data and line-of-sight information.
  • the processor 601 determines whether or not the electroencephalogram data satisfies a condition. For example, the processor 601 determines whether or not the potential represented by the electroencephalogram data is included in a predetermined activation region.
  • FIG. 13 is a diagram illustrating an example of the case where the potential represented by the electroencephalogram data is included in the activation region.
  • the determination condition as to whether or not it is included in the activation region may be looser than the determination condition as to whether the correlation data is within a predetermined similarity range. Thereby, it is possible to easily collect brain wave data for acquiring correlation data.
  • the processor 601 transmits an imaging command to the controller 112 in step S1206. After S1206, the processing from S1201 is repeated. On the other hand, if it is determined that it is not included in the activation area, the process proceeds to S1205.
  • the processor 601 determines whether or not the line-of-sight information satisfies a condition. For example, the processor 601 determines whether the rider's line of sight is constant for a predetermined time, for example, 10 seconds. As a result, even when it is determined that the electroencephalogram data is not included in the activation area, it is possible to detect the case where the rider keeps looking at a certain direction for some reason, and the electroencephalogram data for acquiring the correlation data is collected. Can be easier. If it is determined in step S1205 that the line-of-sight information satisfies the condition, the processor 601 transmits an imaging command to the controller 112 in step S1206. If it is determined that the line-of-sight information does not satisfy the condition, the processing from S1201 is repeated.
  • FIG. 14 is a flowchart showing the processing of the server 108 after learning the electroencephalogram data, that is, after acquiring the correlation data. Steps S1401 to S1403 are the same as those described in steps S1201 to S1203 in FIG.
  • the processor 601 compares the electroencephalogram data with the correlation data.
  • the processor 601 determines whether the electroencephalogram data is included in a predetermined similarity range of the correlation data. If it is determined that it falls within the predetermined similar range, the process proceeds to S1407.
  • S1407 is the same as the description in S1206 of FIG.
  • S1406 is the same as the description in S1205 of FIG.
  • the determinations in S1205 in FIG. 12 and S1406 in FIG. 14 are performed when it is determined in S1204 and S1405 that the electroencephalogram data does not satisfy the condition.
  • the determination as to whether the line-of-sight information satisfies the condition may be performed before S1204 and S1405. In this case, if it is determined that the line-of-sight information satisfies the condition, the processing proceeds to S1206 and S1407, and the processor 601 transmits an imaging command to the controller 112. That is, imaging is performed regardless of the electroencephalogram data.
  • determinations in S1204 and S1405 are performed. When it is determined that the electroencephalogram data satisfies the condition, imaging is performed in S1206 and S1407, and when it is determined that the electroencephalogram data does not satisfy the condition, the processes from S1201 and S1401 are repeated.
  • FIG. 15 is a flowchart showing processing for storing the imaging data and brain wave data transmitted from the controller 112 in the DBs 605 and 606. The process of FIG. 15 is started when imaging data and electroencephalogram data are transmitted from the controller 112 in S808 of FIG.
  • the processor 601 monitors reception of shooting data as registration data in the DB 605, and if it is determined that shooting data has been received, the processor 601 proceeds to S1502. In S1502, the processor 601 registers imaging data and brain wave data in the DBs 605 and 606.
  • the processor 601 determines whether or not the data amount of the electroencephalogram data is sufficient for learning the similarity. If it is determined that the amount is sufficient, the processing proceeds to S1504. If it is determined that the amount is not sufficient, the processing from S1501 is repeated.
  • the processor 601 learns the similarity by analyzing the feature quantity of the electroencephalogram data corresponding to each rider registered in the DB 605, and in S1505, the feature quantity representing the similar pattern corresponds to the shooting event. Obtained as correlation data. If the process of S1504 is the first process for the electroencephalogram data registered in the DB 605, the acquired feature quantity representing the similar pattern is used as correlation data corresponding to the imaging event as a predetermined storage area of the server 108. And the process of FIG. 15 is terminated. On the other hand, if it is not the first process, the correlation data stored in a predetermined storage area of the server 108 is corrected (updated) based on the acquired feature amount representing the similar pattern, and then the process of FIG. Exit.
  • the rider can take a picture when it is considered that the rider has a good shooting timing without causing a shooting operation. Then, the rider selects shooting data that is subjectively good from the shooting result, and the selected shooting data is transmitted to the server 108. With such a configuration, only shooting data that the rider thinks is truly good shooting timing can be registered in the DB 606 of the server 108. Then, the server 108 learns the similarity of the electroencephalogram data corresponding to the imaging data that each rider thinks is good, acquires the feature amount representing the similarity pattern, and stores it as the latest correlation data. When the server 108 determines that the electroencephalogram data transmitted from the rider is similar to the correlation data, the server 108 transmits an imaging command to the controller 112 of the motorcycle 101.
  • the brain wave data is transmitted from the controller 112 of the motorcycle 101 to the server 108 and an operation command is received, so that the camera 305 performs imaging.
  • this embodiment may be applied to other operations.
  • an example applied to other operations will be described with respect to differences from the operations of the above-described embodiment.
  • FIG. 16 is a flowchart showing processing in which the server 108 transmits an operation command of the apparatus using brain wave data.
  • S1601 to S1602 are the same as those described in S1201 to S1202 of FIG.
  • the processor 601 receives the electroencephalogram data.
  • the processor 601 compares the electroencephalogram data with the correlation data.
  • the correlation data here is data corresponding to an operation event of a predetermined apparatus.
  • the predetermined device operation event is, for example, operating a music playback device (stereo or the like) or a broadcast receiving device (radio or the like).
  • the processor 601 determines whether the electroencephalogram data is included in a predetermined similarity range of the correlation data. If it is determined to be included in the predetermined similar range, the process proceeds to S1606.
  • step S ⁇ b> 1606 the processor 601 transmits an operation command for a predetermined apparatus to the controller 112. After S1606, the processing from S1601 is repeated. On the other hand, if it is determined that it is not included in the predetermined similar range, the processing from S1601 is repeated.
  • the controller 112 When the controller 112 receives the operation command in S1606, the controller 112 transmits the operation command to a predetermined device.
  • the helmet 109 is configured with a music playback device and a speaker that can be loaded with an SD card recording music, and the rider can listen to the music played back with the music playback device through the speaker. ing.
  • the controller 112 transmits a reproduction command from the music reproduction device of the helmet 109 to the helmet 109.
  • the processor 301 of the helmet 109 performs music playback by the music playback device of the helmet 109.
  • the operation of FIG. 16 described above corresponds to the processing after acquiring the correlation data of FIG.
  • the process proceeds to S1605, and it is determined whether the electroencephalogram data satisfies the condition.
  • the processor 601 may determine whether or not the potential represented by the electroencephalogram data is included in a predetermined activation area, as in S1204 of FIG.
  • the timing is not preferable for the rider even if the music reproduction is performed by transmitting the operation command in S1606.
  • the processor 601 of the server 108 does not register the electroencephalogram data when the reproduction is stopped in the DB 605 and excludes it from the similarity learning target.
  • the processor 601 of the server 108 registers the electroencephalogram data at the time of reproduction in the DB 605 and sets it as a similarity learning target.
  • the controller 112 transmits a playback command from the broadcast receiving device of the helmet 109 to the helmet 109.
  • the processor 301 of the helmet 109 performs reproduction using the radio of the helmet 109.
  • Other operations are the same as those in the music reproducing apparatus.
  • the music playback device and the broadcast receiving device have been described as being configured in the helmet 109, they may be configured in the main body of the motorcycle 101. In that case, when receiving the reproduction command, the controller 112 performs reproduction by the music reproduction device or the broadcast reception device.
  • the predetermined device may be a vehicle component of the motorcycle 101. That is, the operation command and the motorcycle 101 (vehicle) may be linked.
  • the operation command and the motorcycle 101 may be linked.
  • the command for controlling the accelerator or brake of the motorcycle 101 is operated in S1606. It may be transmitted as a command.
  • the correlation data in this case is, for example, brain wave data emitted during sleep. If it is determined in S1605 that the electroencephalogram data is included in the predetermined similar range of the correlation data, in S1606, the processor 601 transmits an accelerator or brake control command to the controller 112 as an operation command. Examples of such control commands include suppression of accelerator output and suppression of sudden changes in accelerator and brake.
  • the controller 112 When the controller 112 receives the control command, the controller 112 performs each control described above in cooperation with the ECU of the motorcycle 101 and the like. Further, the processor 601 may transmit a command for sounding a warning sound to the speaker or the like of the helmet 109 together with the accelerator or brake control command.
  • the electroencephalogram data generated during sleep is taken as an example, but electroencephalogram data representing a physical state such as fatigue may be used.
  • advice information such as prompting a break may be transmitted to the controller 112 of the motorcycle 101 in S1606.
  • the controller 112 may display guidance display and break location information on the display unit 404 based on the advice information.
  • the helmet 109 may be configured with a speaker, and the controller 112 may reproduce the voice guidance on the helmet 109 speaker.
  • the electroencephalogram data emitted during sleep and the electroencephalogram data representing a physical state such as fatigue may not be obtained by similarity learning, but may be provided as a predetermined reference.
  • the operation system (100) of the above embodiment includes: Based on the acquisition means for acquiring the electroencephalogram data corresponding to the electroencephalogram signal of the passenger of the vehicle (101, 102, 103) (S802), the database (605) for accumulating the electroencephalogram data, and the electroencephalogram data accumulated in the database Generating means for generating correlation data associated with a command for operating the apparatus (S1503, S1504), based on the electroencephalogram data acquired by the acquiring means and the correlation data generated by the generating means Transmission means for transmitting the command (S804), and control means for operating the apparatus when the command is transmitted by the transmission means (S805).
  • the determination unit further comprises a determination unit (S803) that determines whether to send the command based on the electroencephalogram data acquired by the acquisition unit and the correlation data generated by the generation unit. When it is determined that the command is transmitted by the means, the transmission means transmits the command.
  • the camera can be shot.
  • the apparatus is an imaging unit (702), and further includes a storage unit (S808) for storing brain wave data corresponding to an electroencephalogram signal at the time of imaging by the imaging unit in the database.
  • electroencephalogram data corresponding to an electroencephalogram signal at the time of imaging can be accumulated in a database.
  • the electroencephalogram data stored in the database is associated with imaging data captured by the imaging means.
  • electroencephalogram data and photographs can be associated with each other and managed in a database.
  • the electroencephalogram stored in the database further comprises selection means (S807, 1101) for the imaging means to select imaging data for generating the correlation data among the imaging data captured by the imaging means.
  • the data is associated with the imaging data selected by the selection unit, and the generation unit generates the correlation data based on the electroencephalogram data associated with the imaging data selected by the selection unit.
  • gaze detection means S902, 703 for detecting a gaze signal of the passenger, and when the gaze signal detected by the gaze detection means satisfies a condition, the transmission means transmits the command. It is characterized by doing.
  • the camera can shoot.
  • the electroencephalogram data stored in the database includes electroencephalogram data corresponding to an electroencephalogram signal of a passenger of a vehicle (102, 103) different from the passenger of the vehicle (101), and the generation means includes the database
  • the correlation data is generated by analyzing the similarity of the electroencephalogram data corresponding to the passengers of a plurality of vehicles stored in the vehicle.
  • correlation data can be generated based on brain wave signals detected from a plurality of passengers.
  • the operation system includes a server (108) and a motorcycle (101, 102, 103) as the vehicle.
  • the server includes the acquisition unit, the database, the generation unit, and the transmission unit.
  • the motorcycle includes the control unit, and the acquisition unit acquires brain wave data corresponding to a brain wave signal detected by a helmet (109, 110, 111) worn by the passenger.
  • a configuration including a motorcycle, a server, and a helmet can be realized as an operation system.
  • the apparatus is characterized in that it is a photographing means (702) configured in the helmet (109, 110, 111).
  • the photographing unit can photograph a rider in the direction of the eyes of the rider.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Psychology (AREA)
  • Human Computer Interaction (AREA)
  • General Physics & Mathematics (AREA)
  • Psychiatry (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

Provided is a manipulation system which makes it possible to cause, using brain waves, a device to operate without an occupant initiating a manipulation operation. Correlation data associated with an instruction for causing the device to operate are generated on the basis of brain wave data accumulated in a database. When the instruction is transmitted on the basis of both the brain wave data acquired with respect to the occupant of a vehicle and the generated correlation data, the device is caused to operate.

Description

操作システム、サーバ、操作システムにおける制御方法およびプログラムOperation system, server, control method and program in operation system
　本発明は、脳波を利用した操作システム、サーバ、操作システムにおける制御方法およびプログラムに関する。 The present invention relates to an operation system using a brain wave, a server, a control method in the operation system, and a program.
　人間の生体情報を利用したシステムや装置がある。特許文献１には、生体情報センサにより検出した情報に基づいて、作業者の作業に対する負荷を判定し、判定結果に基づいて、作業環境を制御することが記載されている。特許文献２には、人の眼球運動に応じて、二輪車の制御パラメータを変更することが記載されている。特許文献３には、取得した脳波信号と心電信号との時間差を演算して、この演算された時間差に応じて運転者の心理状態を判定することが記載されている。特許文献４には、脳波を利用してコンピュータやゲーム機を操作することが記載されている。 There are systems and devices that use human biological information. Patent Document 1 describes that a load on an operator's work is determined based on information detected by a biological information sensor, and a work environment is controlled based on the determination result. Patent Document 2 describes changing a control parameter of a two-wheeled vehicle in accordance with a human eye movement. Patent Document 3 describes that a time difference between an acquired electroencephalogram signal and an electrocardiogram signal is calculated and a driver's psychological state is determined based on the calculated time difference. Patent Document 4 describes that a computer or a game machine is operated using an electroencephalogram.
特開平０８－１７３４０７号公報Japanese Patent Laid-Open No. 08-173407 特開２０１４－１１３２２７号公報JP 2014-113227 A 特開２０１７－０００６４９号公報JP 2017-000649 A 特開２００２－１６６０５０号公報JP 2002-166050 A
　人間の生体情報を利用したシステムや装置は、二輪車のような車両にも適用可能である。一方、二輪車のような車両においては、例えば、運転者である搭乗者は、走行中に装置を操作する動作を行うことは安全上、極めて難しい。人間の生体情報を利用したシステムや装置を走行中における上記のようなケースに適用し、走行中の装置の動作を可能にすることが考えられる。 Systems and devices that use human biological information can also be applied to vehicles such as motorcycles. On the other hand, in a vehicle such as a two-wheeled vehicle, for example, it is extremely difficult for a passenger who is a driver to operate the device while traveling. It is conceivable to apply a system or apparatus using human biological information to the above-described case while traveling to enable operation of the apparatus while traveling.
　特許文献１には、脳波センサとまばたきセンサを搭載したヘルメットが記載され、運転者の疲労度に応じた制御構成が記載されている。特許文献２には、アイカメラを搭載したヘルメットが記載され、ライダーの注意レベルの違いに応じた制御構成が記載されている。特許文献３には、脳波センサとアイカメラを搭載したヘルメットが記載され、運転者の緊張度合いに応じた制御構成が記載されている。特許文献４には、脳波を電気信号として利用してコンピュータやゲーム機を操作することが記載されている。 Patent Document 1 describes a helmet equipped with an electroencephalogram sensor and a blink sensor, and describes a control configuration in accordance with the driver's fatigue level. Patent Document 2 describes a helmet equipped with an eye camera, and describes a control configuration corresponding to a difference in rider's attention level. Patent Document 3 describes a helmet equipped with an electroencephalogram sensor and an eye camera, and describes a control configuration according to the driver's degree of tension. Patent Document 4 describes that a computer or a game machine is operated using an electroencephalogram as an electrical signal.
　しかしながら、いずれの特許文献にも、走行中において、運転者が操作動作を起こすことなく装置を動作させることに人間の生体情報を利用するケースについては触れられていない。 However, none of the patent documents mentions a case in which human biological information is used to operate the device without causing the driver to perform an operation while driving.
　本発明は、脳波を利用し、搭乗者が装置の操作動作を起こすことなく装置を動作させることを可能とする操作システム、サーバ、操作システムにおける制御方法およびプログラムを提供することを目的とする。 An object of the present invention is to provide an operation system, a server, a control method in the operation system, and a program that allow an occupant to operate the apparatus without causing an operation operation of the apparatus using brain waves.
　本発明に係る操作システムは、脳波データを蓄積するデータベースと、前記データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成手段と、車両の搭乗者の脳波信号に対応する脳波データを取得する取得手段と、前記取得手段により取得された脳波データと前記生成手段により生成された前記相関データとに基づいて、前記指令を発信する発信手段と、前記発信手段により前記指令が発信されると、前記装置を動作させる制御手段とを備えることを特徴とする。 An operation system according to the present invention includes a database for accumulating brain wave data, a generation unit that generates correlation data associated with a command for operating the apparatus based on the brain wave data accumulated in the database, Obtaining means for obtaining electroencephalogram data corresponding to the brain wave signal of the passenger, and sending means for sending the command based on the electroencephalogram data obtained by the obtaining means and the correlation data generated by the generating means; And a control means for operating the apparatus when the command is sent by the sending means.
　また、本発明に係るサーバは、脳波データを蓄積するデータベースと、前記データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成手段と、車両の搭乗者の脳波信号に対応する脳波データを取得する取得手段と、前記取得手段により取得された脳波データと前記生成手段により生成された前記相関データとに基づいて、前記指令を発信する発信手段とを備えることを特徴とする。 Further, the server according to the present invention includes a database for accumulating brain wave data, a generation means for generating correlation data associated with a command for operating the apparatus based on the brain wave data accumulated in the database, and a vehicle Acquiring means for acquiring the electroencephalogram data corresponding to the brain wave signals of the passengers, and transmitting means for transmitting the command based on the electroencephalogram data acquired by the acquiring means and the correlation data generated by the generating means It is characterized by providing.
　また、本発明に係る制御方法は、データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成ステップと、車両の搭乗者の脳波信号に対応する脳波データを取得する取得ステップと、前記取得ステップにおいて取得された脳波データと前記生成ステップにおいて生成された前記相関データとに基づいて、前記指令を発信する発信ステップと、前記発信ステップにおいて前記指令が発信されると、前記装置を動作させる制御ステップとを有することを特徴とする。 Further, the control method according to the present invention corresponds to a generation step of generating correlation data associated with a command for operating the apparatus based on the electroencephalogram data stored in the database, and an electroencephalogram signal of a vehicle occupant An acquisition step of acquiring the electroencephalogram data, a transmission step of transmitting the command based on the electroencephalogram data acquired in the acquisition step and the correlation data generated in the generation step, and the command in the transmission step And a control step of operating the device when the message is transmitted.
　また、本発明に係るプログラムは、データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成ステップと、車両の搭乗者の脳波信号に対応する脳波データを取得する取得ステップと、前記取得ステップにおいて取得された脳波データと前記生成ステップにおいて生成された前記相関データとに基づいて、前記指令を発信する発信ステップと、前記発信ステップにおいて前記指令が発信されると、前記装置を動作させる制御ステップとをコンピュータに実行させることを特徴とする。 The program according to the present invention corresponds to a generation step of generating correlation data associated with a command for operating the apparatus based on the electroencephalogram data stored in the database, and an electroencephalogram signal of a vehicle occupant. Based on the acquisition step of acquiring electroencephalogram data, the electroencephalogram data acquired in the acquisition step and the correlation data generated in the generation step, the command is transmitted in the transmission step, When transmitted, the computer is caused to execute a control step of operating the apparatus.
　本発明によれば、脳波を利用し、搭乗者が操作動作を起こすことなく装置を動作させることを可能とする。 According to the present invention, it is possible to operate the apparatus using an electroencephalogram without causing the passenger to perform an operation.
脳波による操作システムの全体構成を示す図である。It is a figure which shows the whole structure of the operation system by an electroencephalogram. 操作システムにおける通信網を示す図である。It is a figure which shows the communication network in an operation system. ヘルメットのブロック構成を示す図である。It is a figure which shows the block structure of a helmet. コントローラのブロック構成を示す図である。It is a figure which shows the block configuration of a controller. 無線基地局のブロック構成を示す図である。It is a figure which shows the block configuration of a radio base station. サーバのブロック構成を示す図である。It is a figure which shows the block configuration of a server. ヘルメットの外観を示す図である。It is a figure which shows the external appearance of a helmet. 操作システムの処理シーケンスを示す図である。It is a figure which shows the process sequence of an operation system. ヘルメットにおける制御基板の処理を示すフローチャートである。It is a flowchart which shows the process of the control board in a helmet. コントローラの処理を示すフローチャートである。It is a flowchart which shows the process of a controller. 写真選択画面を示す図である。It is a figure which shows a photograph selection screen. 脳波データの学習前のサーバの処理を示すフローチャートである。It is a flowchart which shows the process of the server before learning of electroencephalogram data. 脳波データが表す電位が活性化域に含まれる場合を示す図である。It is a figure which shows the case where the electric potential which electroencephalogram data represents is contained in an activation area. 脳波データの学習後のサーバの処理を示すフローチャートである。It is a flowchart which shows the process of the server after learning of electroencephalogram data. ＤＢへの登録処理を示すフローチャートである。It is a flowchart which shows the registration process to DB. 装置の動作指令を送信する処理を示すフローチャートである。It is a flowchart which shows the process which transmits the operation command of an apparatus.
　図１は、本実施形態における脳波による操作システムの全体構成を示す図である。本実施形態における操作システムは、移動中の車両の搭乗者から送信される脳波データに基づいて、搭乗者のヘルメットに設けられたカメラに撮影指令を送信することができる。図１に示すように、操作システム１００は、バイク１０１、１０２、１０３と、無線基地局１０４、１０５、１０６と、ネットワーク１０７と、サーバ１０８とを含む。本実施形態では、バイク１０１は、いわゆる鞍乗り型車両であり、二輪車両の他、三輪車両等も含む。各バイク１０１～１０３と無線基地局１０４～１０６との間では、無線通信が行われ、無線基地局１０４～１０６とサーバ１０８との間では、ネットワーク１０７を介した通信が行われる。ネットワーク１０７は、例えば、インターネットや専用通信網であり、また、一部、携帯電話等の電話通信網を含んでも良い。無線基地局１０４～１０６は、例えば、信号機等の公共施設に設置され、バイク１０１～１０３と通信する無線通信装置であり、例えば携帯電話通信網の無線基地局として構成される。無線基地局１０４～１０６は、例えば、所定のエリア単位に構成される。バイク１０１～１０３の各ライダーは、ヘルメット１０９、１１０、１１１を着用している。また、バイク１０１～１０３には、操作システム１００の動作を実現するためのコントローラ１１２、１１３、１１４が搭載されており、コントローラ１１２～１１４は、ヘルメット１０９～１１１と、無線基地局１０４～１０６との間で通信する。 FIG. 1 is a diagram showing an overall configuration of an operation system using an electroencephalogram in the present embodiment. The operation system according to the present embodiment can transmit an imaging command to a camera provided in the helmet of the passenger based on brain wave data transmitted from the passenger of the moving vehicle. As shown in FIG. 1, the operation system 100 includes motorcycles 101, 102, 103, radio base stations 104, 105, 106, a network 107, and a server 108. In the present embodiment, the motorcycle 101 is a so-called saddle riding type vehicle, and includes a three-wheeled vehicle in addition to a two-wheeled vehicle. Wireless communication is performed between each of the motorcycles 101 to 103 and the wireless base stations 104 to 106, and communication via the network 107 is performed between the wireless base stations 104 to 106 and the server 108. The network 107 is, for example, the Internet or a dedicated communication network, and may partially include a telephone communication network such as a mobile phone. The wireless base stations 104 to 106 are wireless communication devices that are installed in public facilities such as traffic lights and communicate with the motorcycles 101 to 103, and are configured as wireless base stations of a mobile phone communication network, for example. The radio base stations 104 to 106 are configured in predetermined area units, for example. Each rider of the motorcycles 101 to 103 wears helmets 109, 110, and 111. Also, the motorcycles 101 to 103 are equipped with controllers 112, 113, and 114 for realizing the operation of the operation system 100. The controllers 112 to 114 include the helmets 109 to 111, the radio base stations 104 to 106, and the like. Communicate between.
　図１では、バイク１０１～１０３の３台、無線基地局１０４～１０６の３台が示されているが、図に示される台数に限定されず、また、複数台のバイクが１つの無線基地局に対応可能である。また、図１では、サーバ１０８は１つのみ示されているが、複数の装置により構成されても良い。また、サーバ１０８は、クラウドとして構成されても良い。 In FIG. 1, three motorcycles 101 to 103 and three radio base stations 104 to 106 are shown. However, the number is not limited to the number shown in the figure, and a plurality of motorcycles are one radio base station. Can be supported. In FIG. 1, only one server 108 is shown, but it may be configured by a plurality of devices. The server 108 may be configured as a cloud.
　ネットワーク１０７が携帯電話等の電話通信網を含む場合には、バイク１０１のコントローラ１１２は、携帯端末として動作可能である。その場合、コントローラ１１２は、ライダーからのユーザインタフェース画面を介した設定により、例えばライダーの携帯端末の契約者ＩＤ等と自装置とを対応付けるようにしても良い。そのような構成により、サーバ１０８が、携帯電話システムで管理されているライダーの個人情報やサービス情報を利用可能とするようにしても良い。 When the network 107 includes a telephone communication network such as a mobile phone, the controller 112 of the motorcycle 101 can operate as a mobile terminal. In this case, the controller 112 may associate, for example, the contractor ID of the rider's mobile terminal with the own device by setting via the user interface screen from the rider. With such a configuration, the server 108 may make it possible to use the rider's personal information and service information managed by the mobile phone system.
　以下、特に断らない限り、バイク１０１、ヘルメット１０９、コントローラ１１２、無線基地局１０４を、バイク１０１～１０３、ヘルメット１０９～１１１、コントローラ１１２～１１４、無線基地局１０４～１０６の代表例として説明する。 Hereinafter, unless otherwise specified, the motorcycle 101, the helmet 109, the controller 112, and the wireless base station 104 will be described as representative examples of the motorcycle 101 to 103, the helmet 109 to 111, the controller 112 to 114, and the wireless base station 104 to 106.
　図２は、操作システム１００における通信網を示す図である。図２では、データ通信網を含むことを概念的に示している。操作システム１００においては、ヘルメット１０９とコントローラ１１２との間で通信２０１が行われる。通信２０１には、ヘルメット１０９から送信された脳波データや、コントローラ１１２から送信された撮影指令が含まれる。また、コントローラ１１２と無線基地局１０４との間で通信２０２が行われる。通信２０２には、脳波データや、バイク１０１の駆動情報が含まれる。バイク１０１の駆動情報とは、例えば、バイク１０１の速度情報である。 FIG. 2 is a diagram showing a communication network in the operation system 100. FIG. 2 conceptually shows that a data communication network is included. In the operation system 100, communication 201 is performed between the helmet 109 and the controller 112. The communication 201 includes the electroencephalogram data transmitted from the helmet 109 and the imaging command transmitted from the controller 112. In addition, communication 202 is performed between the controller 112 and the radio base station 104. The communication 202 includes brain wave data and driving information of the motorcycle 101. The driving information of the motorcycle 101 is speed information of the motorcycle 101, for example.
　無線基地局１０４とサーバ１０８との間で通信２０３が行われる。無線基地局１０４は、コントローラ１１２とサーバ１０８との間を中継するように動作し、通信２０３には、脳波データや、バイク１０１の駆動情報が含まれる。 Communication 203 is performed between the wireless base station 104 and the server 108. The radio base station 104 operates to relay between the controller 112 and the server 108, and the communication 203 includes brain wave data and driving information of the motorcycle 101.
　図１及び図２に示す操作システム１００では、バイク１０１のライダーの発する脳波データが所定の条件を満たすときに、ヘルメット１０９に設けられたカメラに撮影指令が発せられる。そのような構成により、例えば、ライダーがバイク１０１を運転中に撮影タイミングに適切であろう風景を見たときに、ライダーが撮影動作を起こすことなく、自動的に撮影を行うことができる。 In the operation system 100 shown in FIGS. 1 and 2, when the electroencephalogram data emitted by the rider of the motorcycle 101 satisfies a predetermined condition, an imaging command is issued to the camera provided in the helmet 109. With such a configuration, for example, when a rider views a landscape that is appropriate for shooting timing while driving the motorcycle 101, the rider can automatically perform shooting without causing a shooting operation.
　図３は、ヘルメット１０９のブロック構成の一例を示す図である。ヘルメット１０９には、ライダーの生体情報を取得するためのセンサ群３０４が構成されている。センサ群３０４は、ライダーの脳波信号を検出する脳波センサを含む。カメラ３０５は、ヘルメット１０９に取り付けられており、前方の被写体（例えば風景）を撮影する。操作システム１００においては、カメラ３０５は、コントローラ１１２からの撮影指令により被写体を撮影する。アイトラッキングカメラ３０６は、ライダーの視線情報を取得するためのカメラである。センサ群３０４からのセンサ信号、カメラ３０５及びアイトラッキングカメラ３０６からの撮影データは、制御基板３００に送信される。 FIG. 3 is a diagram illustrating an example of a block configuration of the helmet 109. The helmet 109 is configured with a sensor group 304 for acquiring rider's biological information. The sensor group 304 includes an electroencephalogram sensor that detects a rider's electroencephalogram signal. The camera 305 is attached to the helmet 109 and photographs a front subject (for example, a landscape). In the operation system 100, the camera 305 captures a subject in accordance with a capturing command from the controller 112. The eye tracking camera 306 is a camera for acquiring rider's line-of-sight information. Sensor signals from the sensor group 304 and imaging data from the camera 305 and the eye tracking camera 306 are transmitted to the control board 300.
　画像メモリ３０８は、カメラ３０５により撮影された撮影データ及びその撮影時の脳波データを保存する。画像メモリ３０８は、例えばＳＤカードであって、メモリＩ／Ｆ３０７に対して装着及び取外しが可能に構成されている。 The image memory 308 stores imaging data captured by the camera 305 and brain wave data at the time of imaging. The image memory 308 is, for example, an SD card, and is configured so that it can be attached to and removed from the memory I / F 307.
　コントローラＩ／Ｆ３０９は、コントローラ１１２との間の通信を行うためのインタフェースとして動作し、例えば、センサ群３０４からのセンサ信号をコントローラ１１２に送信する。なお、コントローラＩ／Ｆ３０９を介したコントローラ１１２との間の通信は、無線により行われても良いし、有線により行われても良い。 The controller I / F 309 operates as an interface for performing communication with the controller 112, and transmits, for example, a sensor signal from the sensor group 304 to the controller 112. Note that communication with the controller 112 via the controller I / F 309 may be performed wirelessly or may be performed by wire.
　制御基板３００は、例えば、ヘルメット１０９の内部に構成されている。制御基板３０上には、プロセッサ３０１、メモリ３０２、記憶部３０３、メモリＩ／Ｆ３０７、コントローラＩ／Ｆ３０９が構成され、バス３１０を介して相互に通信可能に接続されている。プロセッサ３０１は、記憶部３０３に記憶されたプログラムをメモリ３０２に読み出して実行し、ヘルメット１０９内の各ブロックを統括的に制御可能である。 The control board 300 is configured inside the helmet 109, for example. A processor 301, a memory 302, a storage unit 303, a memory I / F 307, and a controller I / F 309 are configured on the control board 30, and are connected to each other via a bus 310 so as to communicate with each other. The processor 301 reads out the program stored in the storage unit 303 to the memory 302 and executes it, and can control each block in the helmet 109 in an integrated manner.
　図４は、コントローラ１１２のブロック構成の一例を示す図である。コントローラ１１２は、例えば、バイク１０１のＥＣＵを含むコントローラとは別として構成されても良いし、一部として構成されても良い。表示部４０４は、例えば表示パネルであり、後述する写真選択画面等のユーザインタフェース画面を表示する。ＧＰＳ４０５は、ＧＰＳ発振器であり、バイク１０１の位置情報を取得するために用いられる。 FIG. 4 is a diagram illustrating an example of a block configuration of the controller 112. For example, the controller 112 may be configured separately from the controller including the ECU of the motorcycle 101 or may be configured as a part. The display unit 404 is a display panel, for example, and displays a user interface screen such as a photo selection screen described later. The GPS 405 is a GPS oscillator and is used for acquiring position information of the motorcycle 101.
　コントローラＩ／Ｆ４０６は、ヘルメット１０９との間の通信を行うためのインタフェースとして動作する。Ｉ／Ｆ４０７は、無線基地局１０４との間の通信を行うためのインターフェースとして動作する。メモリＩ／Ｆ４０８には、ヘルメット１０９から外された画像メモリ３０８を装着可能である。それにより、プロセッサ４０１は、画像メモリ３０８に格納された撮影データの一覧を表示部４０４に表示することができる。 The controller I / F 406 operates as an interface for performing communication with the helmet 109. The I / F 407 operates as an interface for performing communication with the radio base station 104. An image memory 308 removed from the helmet 109 can be attached to the memory I / F 408. Accordingly, the processor 401 can display a list of shooting data stored in the image memory 308 on the display unit 404.
　図４のコントローラ１１２はさらに、プロセッサ４０１、メモリ４０２、記憶部４０３を含み、各ブロックは、バス４０９を介して相互に通信可能に接続されている。プロセッサ４０１は、記憶部４０３に記憶されたプログラムをメモリ４０２に読み出して実行し、制御基板上の各ブロックを統括的に制御可能である。プロセッサ４０１は、バイク１０１のＥＣＵと連携可能であり、例えば、バイク１０１の速度情報に応じて制御基板上の各ブロックを制御することができ、また、バイク１０１のＥＣＵに対して加速や減速の制御を要求することができる。また、プロセッサ４０１は、ＡＩ（Ａｒｔｉｆｉｃｉａｌ　Ｉｎｔｅｌｌｉｇｅｎｃｅ）として用いられるアルゴリズムを実行可能である。 4 further includes a processor 401, a memory 402, and a storage unit 403, and each block is connected to be communicable with each other via a bus 409. The processor 401 reads out the program stored in the storage unit 403 to the memory 402 and executes it, and can comprehensively control each block on the control board. The processor 401 can cooperate with the ECU of the motorcycle 101, and can control each block on the control board in accordance with, for example, the speed information of the motorcycle 101. Control can be requested. Further, the processor 401 can execute an algorithm used as AI (Artificial Intelligence).
　図５は、無線基地局１０４のブロック構成の一例を示す図である。Ｉ／Ｆ５０４は、コントローラ１１２との間の通信を行うためのインタフェースとして動作する。ネットワーク（ＮＷ）Ｉ／Ｆ５０５は、ネットワーク１０７を介してサーバ１０８と通信を行うためのインタフェースとして動作する。 FIG. 5 is a diagram illustrating an example of a block configuration of the radio base station 104. The I / F 504 operates as an interface for performing communication with the controller 112. A network (NW) I / F 505 operates as an interface for communicating with the server 108 via the network 107.
　図５に示す各ブロックは、プロセッサ５０１、メモリ５０２、記憶部５０３を含んで構成され、バス５０６を介して相互に通信可能に接続されている。プロセッサ５０１は、記憶部５０３に記憶されたプログラムをメモリ５０２に読み出して実行し、無線基地局１０４を統括的に制御する。プロセッサ５０１は、例えば、コントローラ１１２との間での通信手順と、サーバ１０８との間での通信手順とを変換する。 Each block shown in FIG. 5 includes a processor 501, a memory 502, and a storage unit 503, and is connected to be communicable with each other via a bus 506. The processor 501 reads the program stored in the storage unit 503 into the memory 502 and executes it, and controls the radio base station 104 in an integrated manner. For example, the processor 501 converts a communication procedure with the controller 112 and a communication procedure with the server 108.
　図６は、サーバ１０８のブロック構成の一例を示す図である。ＮＷＩ／Ｆ６０４は、ネットワーク１０７を介して無線基地局１０４と通信を行うためのインタフェースとして動作する。データベース（ＤＢ）６０５には、撮影された撮影データに対応付けられた脳波データが保存されている。ＤＢ６０５については後述する。ＤＢ６０６には、撮影された撮影データが保存されている。 FIG. 6 is a diagram illustrating an example of a block configuration of the server 108. The NWI / F 604 operates as an interface for communicating with the radio base station 104 via the network 107. A database (DB) 605 stores brain wave data associated with the captured image data. The DB 605 will be described later. The DB 606 stores captured image data.
　図６に示す各ブロックは、プロセッサ６０１、メモリ６０２、記憶部６０３を含んで構成され、バス６０７を介して相互に通信可能に接続されている。プロセッサ６０１は、記憶部６０３に記憶されたプログラムをメモリ６０２に読み出して実行し、サーバ１０８を統括的に制御する。また、プロセッサ６０１は、ＡＩとして用いられるアルゴリズムを実行可能である。図６では、ＤＢ６０５とＤＢ６０６は、別個のデータベースとして示されているが、一体のデータベースとして構成されても良い。 Each block shown in FIG. 6 includes a processor 601, a memory 602, and a storage unit 603, and is connected to be communicable with each other via a bus 607. The processor 601 reads the program stored in the storage unit 603 into the memory 602 and executes it, and controls the server 108 in an integrated manner. Further, the processor 601 can execute an algorithm used as an AI. In FIG. 6, DB 605 and DB 606 are shown as separate databases, but may be configured as an integrated database.
　図７は、ヘルメット１０９の外観を示す図である。ヘルメット１０９の頭頂部から前頭部及び後頭部に沿って、また、頭頂部から側頭部に沿って、脳波センサ３０８に対応する電極７０１が複数構成されている。例えば２０個の電極により２０チャネルの脳波信号を検出するようにしても良い。電極７０１は、脳内の活動、例えば感情に応じて頭皮上に現れる所定の周波数帯域成分の電気信号を検出し、検出した電気信号を制御基板７０４に送信する。制御基板７０４は、図３の制御基板３００に対応する。制御基板７０４は、電位信号を不図示のアンプ及びＡＤ変換器によりデジタルデータに変換し、脳波信号に対応する脳波データとしてコントローラ１１２に送信する。脳波データは、例えば、所定のサンプリング周期で取得された時間と電位（μＶ）の離散的なデータとして、複数の電極７０１に対応するデータの組として取得される。 FIG. 7 is a view showing the appearance of the helmet 109. A plurality of electrodes 701 corresponding to the electroencephalogram sensor 308 are configured from the top of the helmet 109 along the frontal and occipital regions and from the top of the helmet 109 along the temporal region. For example, a 20-channel EEG signal may be detected by 20 electrodes. The electrode 701 detects an electrical signal of a predetermined frequency band component that appears on the scalp according to activity in the brain, for example, emotion, and transmits the detected electrical signal to the control board 704. The control board 704 corresponds to the control board 300 in FIG. The control board 704 converts the potential signal into digital data by an amplifier (not shown) and an AD converter, and transmits the digital data to the controller 112 as brain wave data corresponding to the brain wave signal. The electroencephalogram data is acquired as a set of data corresponding to the plurality of electrodes 701 as, for example, discrete data of time and potential (μV) acquired at a predetermined sampling period.
　カメラ７０２は、図３のカメラ３０５に対応する。図３で説明したように、カメラ７０２は、ライダーの指示を介さずに、また、ライダーが撮影動作を起こすことなく、コントローラ１１２からの撮影指令により被写体の撮影を行う。図３に示すように、カメラ７０２は、概ねライダーの視線の方向を撮影するようにヘルメット１０９の前方に構成されている。 The camera 702 corresponds to the camera 305 in FIG. As described with reference to FIG. 3, the camera 702 shoots a subject according to a shooting command from the controller 112 without the rider's instruction and without causing the rider to perform a shooting operation. As shown in FIG. 3, the camera 702 is configured in front of the helmet 109 so as to generally capture the direction of the rider's line of sight.
　アイトラッキングカメラ７０３は、図３のアイトラッキングカメラ３０６に対応する。アイトラッキングカメラ３０６は、ライダーの目の周辺を撮影する。本実施形態では、ライダーの視線情報を取得することが可能であれば、他の構成が用いられても良い。例えば、メガネのパッド部分に相当する箇所に触れるようにし、眼筋の抵抗率の変化により視線情報を取得するアイトラッキングセンサが用いられても良い。 The eye tracking camera 703 corresponds to the eye tracking camera 306 in FIG. The eye tracking camera 306 photographs the periphery of the rider's eyes. In the present embodiment, other configurations may be used as long as it is possible to acquire rider's line-of-sight information. For example, an eye tracking sensor that touches a portion corresponding to the pad portion of the glasses and acquires line-of-sight information by a change in the eye muscle resistivity may be used.
　本実施形態におけるＤＢ６０５、６０６の一例について説明する。ＤＢ６０６は、操作システム１００のサービスを享受するバイクのライダーがヘルメット１０９のカメラ７０２で撮影した撮影データを蓄積している。また、それぞれの撮影データには、その撮影時の脳波データが関連づけられてＤＢ６０５に保存されている。つまり、ＤＢ６０５及び６０６は、各ライダーから送信された撮影データ及び脳波データの組み合わせのビッグデータベースとして構築されている。 An example of the DBs 605 and 606 in this embodiment will be described. The DB 606 stores shooting data taken by a motorcycle rider who enjoys the service of the operation system 100 with the camera 702 of the helmet 109. In addition, each imaging data is associated with brain wave data at the time of imaging and stored in the DB 605. That is, the DBs 605 and 606 are constructed as a big database of combinations of imaging data and electroencephalogram data transmitted from each rider.
　サーバ１０８のプロセッサ６０１は、Ｓ６１１において、コントローラ１１２から送信された各電極７０１に対応する脳波データを取得すると、時間と電位との離散的なデータから連続的な電位信号を取得するために、Ｓ６１２において平滑化を行う。そして、プロセッサ６０１は、Ｓ６１３において、平滑化した電位信号をスペクトル解析し、Ｓ６１４において、各周波数帯域と各電極７０１の２次元で表される特徴量を抽出し、抽出した特徴量をＤＢ６０５に格納する。 When the processor 601 of the server 108 acquires the electroencephalogram data corresponding to each electrode 701 transmitted from the controller 112 in S611, in order to acquire a continuous potential signal from the discrete data of time and potential, S612 Smoothing is performed at. In S613, the processor 601 performs spectrum analysis on the smoothed potential signal. In S614, the processor 601 extracts two-dimensional feature values of each frequency band and each electrode 701, and stores the extracted feature values in the DB 605. To do.
　図８は、本実施形態における操作システム１００の処理シーケンスを示す図である。ここでは、ライダーがバイク１０１に乗車し、操作システム１００のサービスを享受できる状態であるとする。例えば、ライダーがヘルメット１０９の電源を投入すると、コントローラ１１２が、ヘルメット１０９の起動情報およびバイク１０１の駆動情報を、無線基地局１０４を介してサーバ１０８に送信するようにしても良い。サーバ１０８は、それらの情報を受信すると、バイク１０１に対する操作システム１００のサービスの提供を開始する。なお、図８では、データを中継する役割の無線基地局１０４の図示は省略している。 FIG. 8 is a diagram showing a processing sequence of the operation system 100 in the present embodiment. Here, it is assumed that the rider gets on the motorcycle 101 and can enjoy the service of the operation system 100. For example, when the rider turns on the power of the helmet 109, the controller 112 may transmit the activation information of the helmet 109 and the driving information of the motorcycle 101 to the server 108 via the wireless base station 104. When the server 108 receives the information, the server 108 starts providing the operation system 100 service to the motorcycle 101. In FIG. 8, the wireless base station 104 that relays data is not shown.
　まず、Ｓ８０１において、ヘルメット１０９から脳波データが、コントローラ１１２に送信される。また、Ｓ８０１では、脳波データの他にも、アイトラッキングカメラ３０６の撮影データが視線情報として送信される。Ｓ８０２において、コントローラ１１２は、脳波データ、視線情報とともに、バイク１０１の駆動情報をサーバ１０８に送信する。ここで、駆動情報とは、例えば、バイク１０１の速度情報である。Ｓ８０３において、サーバ１０８は、コントローラ１１２から送信された脳波データに基づいて、撮影指令を送信するか否かを判定する。 First, in S801, electroencephalogram data is transmitted from the helmet 109 to the controller 112. In S801, in addition to the electroencephalogram data, imaging data of the eye tracking camera 306 is transmitted as line-of-sight information. In step S <b> 802, the controller 112 transmits driving information of the motorcycle 101 to the server 108 together with the electroencephalogram data and line-of-sight information. Here, the drive information is speed information of the motorcycle 101, for example. In step S <b> 803, the server 108 determines whether to transmit an imaging command based on the electroencephalogram data transmitted from the controller 112.
　ここで、本実施形態における撮影指令の判定処理について説明する。図６において説明したように、ＤＢ６０５及び６０６は、各ライダーから送信された撮影データ及び脳波データの組み合わせのビッグデータベースとして構築されている。また、操作システム１００においては、随時、各ライダーからカメラ３０５で撮影された撮影データと、その撮影時の脳波データとが送信され、ＤＢ６０５及び６０６が更新されている。本実施形態では、カメラ３０５は、ライダーの撮影動作によっては撮影されない。しかしながら、ライダーは、撮影された複数の撮影データから、良いと思うものをバイク１０１の表示部４０４に表示された写真選択画面上で選択する。つまり、ＤＢ６０５及び６０６には、各ライダーが真に良いと思う撮影データと脳波データが格納されている。 Here, the determination process of the imaging command in the present embodiment will be described. As described in FIG. 6, the DBs 605 and 606 are constructed as a big database of a combination of imaging data and brain wave data transmitted from each rider. In the operation system 100, the imaging data captured by the camera 305 and the electroencephalogram data at the time of imaging are transmitted from each rider as needed, and the DBs 605 and 606 are updated. In the present embodiment, the camera 305 is not photographed by the rider's photographing operation. However, the rider selects what he thinks is good from the plurality of photographed data on the photograph selection screen displayed on the display unit 404 of the motorcycle 101. That is, the DBs 605 and 606 store imaging data and brain wave data that each rider thinks are really good.
　サーバ１０８のプロセッサ６０１は、ＤＢ６０５に格納されている脳波データの特徴量を分析し、それらの脳波データの類似性を学習し、類似パターンを表す特徴量を取得する。そして、プロセッサ６０１は、取得された類似パターンを表す特徴量を、撮影イベントと対応する相関データとしてサーバ１０８の所定の記憶領域に保持する。 The processor 601 of the server 108 analyzes the feature quantity of the electroencephalogram data stored in the DB 605, learns the similarity of the electroencephalogram data, and acquires the feature quantity representing the similar pattern. Then, the processor 601 holds the acquired feature amount representing the similar pattern in a predetermined storage area of the server 108 as correlation data corresponding to the shooting event.
　Ｓ８０３では、サーバ１０８のプロセッサ６０１は、コントローラ１１２から送信された脳波データの特徴量と相関データとを比較することで、撮影指令を送信するか否かを判定する。コントローラ１１２から送信された脳波データの特徴量が相関データと所定の類似範囲にあると判断した場合には、撮影指令を送信すると判定する。 In S803, the processor 601 of the server 108 determines whether or not to send the imaging command by comparing the feature quantity of the electroencephalogram data transmitted from the controller 112 with the correlation data. When it is determined that the feature amount of the electroencephalogram data transmitted from the controller 112 is within a predetermined similar range to the correlation data, it is determined that an imaging command is transmitted.
　撮影指令を送信すると判定した場合には、Ｓ８０４において、サーバ１０８は、コントローラ１１２に撮影指令を送信する。Ｓ８０５において、コントローラ１１２は、ヘルメット１０９のカメラ３０５による撮影指令をヘルメット１０９に送信する。Ｓ８０６において、ヘルメット１０９のプロセッサ３０１は、カメラ３０５による撮影を行う。撮影された撮影データは、その撮影時の脳波データとともに画像メモリ３０８に保存される。 If it is determined that the shooting command is to be transmitted, the server 108 transmits the shooting command to the controller 112 in S804. In step S <b> 805, the controller 112 transmits a shooting command from the camera 305 of the helmet 109 to the helmet 109. In step S <b> 806, the processor 301 of the helmet 109 performs shooting with the camera 305. The captured imaging data is stored in the image memory 308 together with the electroencephalogram data at the time of imaging.
　本実施形態においては、ビッグデータベースとして構築され各ライダーが良い撮影タイミングと感じた脳波データの特徴量を分析して、それらの類似パターンとなる特徴量を学習し、撮影イベントと対応する相関データとして保存しておく。そして、新たにサーバ１０８に送信されてきた脳波データと相関データとが所定の類似範囲にあると判断すると、現在、ライダーが良い撮影タイミングであると感じていると判断して撮影指令を送信する。その結果、ライダーは、撮影動作を起こすことなく、望ましいタイミングで撮影を行うことができる。 In this embodiment, it is constructed as a big database, the feature quantities of the electroencephalogram data that each rider feels as good shooting timing are analyzed, the feature quantities that become similar patterns are learned, and the correlation data corresponding to the shooting event is obtained. Save it. When it is determined that the electroencephalogram data newly transmitted to the server 108 and the correlation data are within a predetermined similar range, it is determined that the rider currently feels that the imaging timing is good and an imaging command is transmitted. . As a result, the rider can perform shooting at a desired timing without causing a shooting operation.
　Ｓ８０６で撮影された後、例えば、ライダーは、休憩地点でバイク１０１を降りるとする。そこで、ライダーは、ヘルメット１０９の画像メモリ３０８をメモリＩ／Ｆ３０７から取り外し、コントローラ１１２のメモリＩ／Ｆ４０８に装着する。プロセッサ４０１は、画像メモリ３０８の装着を検出すると、画像メモリ３０８に格納されている撮影データの一覧を写真選択画面として表示部４０４に表示する。 After shooting in S806, for example, it is assumed that the rider gets off the motorcycle 101 at a break point. Therefore, the rider removes the image memory 308 of the helmet 109 from the memory I / F 307 and attaches it to the memory I / F 408 of the controller 112. When detecting that the image memory 308 is attached, the processor 401 displays a list of shooting data stored in the image memory 308 on the display unit 404 as a photo selection screen.
　図１１は、写真選択画面１１０１の一例を示す図である。図１１に示すように、サムネイル画像１１０２が複数表示される。サムネイル画像１１０２は、画像メモリ３０８に格納されている撮影データに対応する。また、サムネイル画像１１０２には、日時情報１１０４が付与されて表示される。そして、写真選択画面１１０１上では、チェックボックス１１０３により、ライダーの選択を受け付けることができる。また、画像メモリ３０８に多数の撮影データが格納されている場合には、一つの画面では表示しきれない。その場合には、遷移ボタン１１０５により、先行ページもしくは後続ページを表示することができる。 FIG. 11 is a diagram showing an example of the photo selection screen 1101. As shown in FIG. 11, a plurality of thumbnail images 1102 are displayed. A thumbnail image 1102 corresponds to shooting data stored in the image memory 308. The thumbnail image 1102 is displayed with date / time information 1104 added thereto. On the photo selection screen 1101, a rider's selection can be accepted by a check box 1103. In addition, when a large number of shooting data is stored in the image memory 308, it cannot be displayed on one screen. In that case, the previous page or the subsequent page can be displayed by the transition button 1105.
　本実施形態では、ライダーは、撮影動作を起こすことなく、望ましいタイミングで撮影を行うことができる。しかしながら、カメラ７０２のアングル等の要因により、撮影された撮影データが全て真に望ましい写真であるかは分からない。そこで、本実施形態では、写真選択画面１１０１上でライダーの選択を受け付けることにより、真に望ましい撮影データのみを特定することができる。 In this embodiment, the rider can shoot at a desired timing without causing a shooting operation. However, due to factors such as the angle of the camera 702, it is not known whether all the photographed data is truly desirable. Therefore, in this embodiment, by accepting a rider's selection on the photo selection screen 1101, only truly desirable shooting data can be specified.
　Ｓ８０７において、コントローラ１１２は、写真選択画面１１０１上で、撮影データの選択を受け付ける。これは、図１１では不図示であるが、確定ボタン等により選択を確定するようにしても良い。そして、Ｓ８０８において、コントローラ１１２は、選択された撮影データ及びその撮影時の脳波データをサーバ１０８に送信する。サーバ１０８は、送信された撮影データ及びその撮影時の脳波データをＤＢ６０５に登録する。 In step S807, the controller 112 accepts selection of shooting data on the photo selection screen 1101. This is not shown in FIG. 11, but the selection may be confirmed by a confirmation button or the like. In step S <b> 808, the controller 112 transmits the selected imaging data and electroencephalogram data at the time of imaging to the server 108. The server 108 registers the transmitted imaging data and the electroencephalogram data at the time of imaging in the DB 605.
　Ｓ８０９において、サーバ１０８は、Ｓ８０８で新たに登録された脳波データを含めて特徴量を分析し、それらの類似パターンとなる特徴量を学習する。そして、Ｓ８１０において、サーバ１０８は、学習して得た特徴量に基づいて、撮影イベントと対応する相関データを修正する。Ｓ８０９での特徴量の学習は、所定の時間間隔で定期的に行われても良いし、撮影データが所定枚数ごとに蓄積する度に行われても良い。 In step S809, the server 108 analyzes the feature amounts including the brain wave data newly registered in step S808, and learns the feature amounts that are similar patterns. In step S810, the server 108 corrects the correlation data corresponding to the shooting event based on the feature amount obtained by learning. The feature amount learning in step S809 may be performed periodically at predetermined time intervals, or may be performed every time shooting data is accumulated every predetermined number of sheets.
　上記では、Ｓ８０７でコントローラ１１２が写真選択画面１１０１上で撮影データの選択を受け付ける構成を説明した。しかしながら、他の構成により撮影データの選択が行われても良い。例えば、撮影データの選択をクラウド上で行うようにしても良い。その場合には、Ｓ８０６で撮影が行われると、その撮影データ及び脳波データがサーバ１０８に送信される。撮影データ及び脳波データの送信は、直接、サーバ１０８に対して行われても良いし、コントローラ１１２を介して行われても良い。そして、ライダーは、休憩地点でバイク１０１を降りたときに、自身の携帯端末からサーバ１０８の撮影データの格納場所にアクセスする。その際の格納場所は、ＤＢ６０５及び６０６とは異なる写真データベースでも良い。アクセスすると、図１１に示すような写真選択画面１１０１が表示され、ライダーは、写真選択画面１１０１上での選択操作を行う。サーバ１０８は、写真データベースで選択された撮影データ及び脳波データをＤＢ６０５に格納する。 In the above description, the configuration in which the controller 112 accepts selection of shooting data on the photo selection screen 1101 has been described in S807. However, the shooting data may be selected by other configurations. For example, the shooting data may be selected on the cloud. In that case, when imaging is performed in S806, the imaging data and brain wave data are transmitted to the server 108. The transmission of imaging data and electroencephalogram data may be performed directly to the server 108 or may be performed via the controller 112. Then, when the rider gets off the motorcycle 101 at the break point, the rider accesses the storage location of the shooting data of the server 108 from his / her portable terminal. The storage location at that time may be a photo database different from the DBs 605 and 606. When accessed, a photo selection screen 1101 as shown in FIG. 11 is displayed, and the rider performs a selection operation on the photo selection screen 1101. The server 108 stores the imaging data and brain wave data selected in the photo database in the DB 605.
　図９は、ヘルメット１０９における制御基板３００の処理を示すフローチャートである。Ｓ９０１において、プロセッサ３０１は、自動撮影モードであるか否かを判定する。Ｓ９０１の判定は、例えば、制御基板３００に設けられたハードスイッチにより判定しても良いし、コントローラ１１２から受信したいずれかのモードである旨の指示に基づいて判定しても良い。Ｓ９０１で自動撮影モードでないと判定された場合、図９の処理を終了し、自動撮影モードであると判定された場合、Ｓ９０２に進む。 FIG. 9 is a flowchart showing processing of the control board 300 in the helmet 109. In step S901, the processor 301 determines whether or not the automatic shooting mode is set. The determination in S901 may be performed by, for example, a hard switch provided on the control board 300, or may be performed based on an instruction indicating that one of the modes is received from the controller 112. If it is determined in S901 that the mode is not the automatic shooting mode, the process of FIG. 9 is terminated. If it is determined that the mode is the automatic shooting mode, the process proceeds to S902.
　Ｓ９０２において、プロセッサ３０１は、電極７０１で検出された電気信号（脳波信号）をアンプ及びＡＤ変換器により脳波データに変換し、コントローラ１１２に送信する。そして、プロセッサ３０１は、アイトラッキングカメラ３０６による撮像データを視線情報としてコントローラ１１２に送信する。 In S902, the processor 301 converts an electrical signal (electroencephalogram signal) detected by the electrode 701 into electroencephalogram data by an amplifier and an AD converter, and transmits the electroencephalogram data to the controller 112. Then, the processor 301 transmits imaging data obtained by the eye tracking camera 306 to the controller 112 as line-of-sight information.
　Ｓ９０３において、プロセッサ３０１は、コントローラ１１２から撮影指令を受信したか否かを判定する。撮影指令を受信していないと判定した場合、Ｓ９０２の処理を繰り返す。一方、撮影指令を受信したと判定した場合、Ｓ９０４において、プロセッサ３０１は、カメラ３０５による撮影を行う。そして、Ｓ９０５において、プロセッサ３０１は、撮影データを、その撮影時の脳波データとともに画像メモリ３０８に保存する。Ｓ９０５の後、Ｓ９０１からの処理を繰り返す。 In step S <b> 903, the processor 301 determines whether a shooting command has been received from the controller 112. If it is determined that the imaging command has not been received, the process of S902 is repeated. On the other hand, if it is determined that a shooting command has been received, the processor 301 performs shooting with the camera 305 in step S904. In step S905, the processor 301 stores the imaging data in the image memory 308 together with the electroencephalogram data at the time of imaging. After S905, the processing from S901 is repeated.
　図１０は、コントローラ１１２の処理を示すフローチャートである。図１０の処理は、画像メモリ３０８がメモリＩ／Ｆ４０８に装着されると開始する。Ｓ１００１において、プロセッサ４０１は、画像メモリ３０８に格納されている撮影データ及び脳波データを取得し、表示部４０４に写真選択画面１１０１を表示する。Ｓ１００２において、プロセッサ４０２は、写真選択画面１１０１上でのチェックボックス１１０３の選択結果に基づいて撮影データを選択する。Ｓ１００３において、プロセッサ４０１は、選択した撮影データ及び脳波データをサーバ１０８へ送信し、サーバ１０８のＤＢ６０５に登録する。その後、図１０の処理を終了する。 FIG. 10 is a flowchart showing the processing of the controller 112. The processing in FIG. 10 starts when the image memory 308 is attached to the memory I / F 408. In step S <b> 1001, the processor 401 acquires imaging data and brain wave data stored in the image memory 308, and displays a photo selection screen 1101 on the display unit 404. In step S <b> 1002, the processor 402 selects shooting data based on the selection result of the check box 1103 on the photo selection screen 1101. In step S <b> 1003, the processor 401 transmits the selected imaging data and brain wave data to the server 108 and registers them in the DB 605 of the server 108. Then, the process of FIG. 10 is complete | finished.
　図１２は、脳波データの学習前のサーバ１０８の処理を示すフローチャートである。サーバ１０８のＤＢ６０５に格納されている脳波データのデータ量が十分でない場合、類似パターンの特徴量を学習して相関データを生成することができない。その場合には、コントローラ１１２から送信された脳波データに対して予め定められた判定条件により判定を行う。例えば、その脳波データが表す電位が所定の活性化域に含まれる場合、その脳波データは、判定条件を満たすとし、撮影指令を送信する。 FIG. 12 is a flowchart showing the processing of the server 108 before learning the electroencephalogram data. If the amount of electroencephalogram data stored in the DB 605 of the server 108 is not sufficient, correlation data cannot be generated by learning feature amounts of similar patterns. In that case, the determination is performed on the electroencephalogram data transmitted from the controller 112 according to a predetermined determination condition. For example, when the potential represented by the electroencephalogram data is included in a predetermined activation range, the electroencephalogram data is determined to satisfy the determination condition, and an imaging command is transmitted.
　Ｓ１２０１において、プロセッサ６０１は、駆動情報を受信する。ここでの駆動情報は、図８のＳ８０２でコントローラ１１２から送信された駆動情報である。Ｓ１２０２において、プロセッサ６０１は、駆動情報が条件を満たすか否かを判定する。例えば、プロセッサ６０１は、バイク１０１の速度情報が所定の速度以上であるか否かを判定する。駆動情報が条件を満たすと判定した場合、Ｓ１２０３及びＳ１２０４の処理を実行し、駆動情報が条件を満たさないと判定した場合、図１２の処理を終了する。Ｓ１２０３において、プロセッサ６０１は、脳波データと視線情報を受信する。 In S1201, the processor 601 receives drive information. The drive information here is the drive information transmitted from the controller 112 in S802 of FIG. In S1202, the processor 601 determines whether or not the drive information satisfies a condition. For example, the processor 601 determines whether or not the speed information of the motorcycle 101 is equal to or higher than a predetermined speed. If it is determined that the drive information satisfies the condition, the processes of S1203 and S1204 are executed. If it is determined that the drive information does not satisfy the condition, the process of FIG. In S1203, the processor 601 receives brain wave data and line-of-sight information.
　Ｓ１２０４において、プロセッサ６０１は、脳波データが条件を満たすか否かを判定する。例えば、プロセッサ６０１は、脳波データが表す電位が所定の活性化域に含まれるか否かを判定する。図１３は、脳波データが表す電位が活性化域に含まれる場合の一例を示す図である。活性化域に含まれるか否かの判定条件は、上述した相関データと所定の類似範囲にあるかについての判定条件よりも緩くても良い。これにより、相関データを取得するための脳波データを収集しやすくすることができる。活性化域に含まれると判定された場合、Ｓ１２０６において、プロセッサ６０１は、撮影指令をコントローラ１１２に送信する。Ｓ１２０６の後、Ｓ１２０１からの処理を繰り返す。一方、活性化域に含まれないと判定された場合、Ｓ１２０５に進む。 In S1204, the processor 601 determines whether or not the electroencephalogram data satisfies a condition. For example, the processor 601 determines whether or not the potential represented by the electroencephalogram data is included in a predetermined activation region. FIG. 13 is a diagram illustrating an example of the case where the potential represented by the electroencephalogram data is included in the activation region. The determination condition as to whether or not it is included in the activation region may be looser than the determination condition as to whether the correlation data is within a predetermined similarity range. Thereby, it is possible to easily collect brain wave data for acquiring correlation data. If it is determined that the image is included in the activation area, the processor 601 transmits an imaging command to the controller 112 in step S1206. After S1206, the processing from S1201 is repeated. On the other hand, if it is determined that it is not included in the activation area, the process proceeds to S1205.
　Ｓ１２０５において、プロセッサ６０１は、視線情報が条件を満たすか否かを判定する。例えば、プロセッサ６０１は、ライダーの視線が所定時間、例えば１０秒間一定しているか否かを判定する。これにより、脳波データが活性化域に含まれないと判定された場合でも、ライダーが何らかの理由により一定方向を見続けた場合を検出することができ、相関データを取得するための脳波データを収集しやすくすることができる。Ｓ１２０５で視線情報が条件を満たすと判定された場合、Ｓ１２０６において、プロセッサ６０１は、撮影指令をコントローラ１１２に送信する。視線情報が条件を満たさないと判定された場合、Ｓ１２０１からの処理を繰り返す。 In S1205, the processor 601 determines whether or not the line-of-sight information satisfies a condition. For example, the processor 601 determines whether the rider's line of sight is constant for a predetermined time, for example, 10 seconds. As a result, even when it is determined that the electroencephalogram data is not included in the activation area, it is possible to detect the case where the rider keeps looking at a certain direction for some reason, and the electroencephalogram data for acquiring the correlation data is collected. Can be easier. If it is determined in step S1205 that the line-of-sight information satisfies the condition, the processor 601 transmits an imaging command to the controller 112 in step S1206. If it is determined that the line-of-sight information does not satisfy the condition, the processing from S1201 is repeated.
　図１４は、脳波データの学習後、即ち、相関データを取得した後のサーバ１０８の処理を示すフローチャートである。Ｓ１４０１～Ｓ１４０３は、図１２のＳ１２０１～Ｓ１２０３における説明と同じである。 FIG. 14 is a flowchart showing the processing of the server 108 after learning the electroencephalogram data, that is, after acquiring the correlation data. Steps S1401 to S1403 are the same as those described in steps S1201 to S1203 in FIG.
　Ｓ１４０４において、プロセッサ６０１は、脳波データと相関データとを比較する。Ｓ１４０５において、プロセッサ６０１は、脳波データが相関データの所定の類似範囲に含まれるか否かを判定する。所定の類似範囲に含まれると判定した場合、Ｓ１４０７に進む。Ｓ１４０７は、図１２のＳ１２０６における説明と同じである。一方、所定の類似範囲に含まれないと判定した場合、Ｓ１４０６に進む。Ｓ１４０６は、図１２のＳ１２０５における説明と同じである。 In S1404, the processor 601 compares the electroencephalogram data with the correlation data. In step S1405, the processor 601 determines whether the electroencephalogram data is included in a predetermined similarity range of the correlation data. If it is determined that it falls within the predetermined similar range, the process proceeds to S1407. S1407 is the same as the description in S1206 of FIG. On the other hand, if it is determined that it is not included in the predetermined similar range, the process proceeds to S1406. S1406 is the same as the description in S1205 of FIG.
　図１２のＳ１２０５、図１４のＳ１４０６の判定は、Ｓ１２０４及びＳ１４０５において脳波データが条件を満たさないと判定された場合に行っていた。しかしながら、視線情報が条件を満たすか否かの判定を、Ｓ１２０４、Ｓ１４０５の前に行うようにしても良い。その場合には、視線情報が条件を満たすと判定された場合には、Ｓ１２０６、Ｓ１４０７に進み、プロセッサ６０１は、撮影指令をコントローラ１１２に送信する。つまり、脳波データに関わらず、撮影を行わせる。一方、視線情報が条件を満たさないと判定された場合には、Ｓ１２０４、Ｓ１４０５の判定が行われる。脳波データが条件を満たすと判定された場合には、Ｓ１２０６、Ｓ１４０７において撮影が行われ、脳波データが条件を満たさないと判定された場合には、Ｓ１２０１、Ｓ１４０１からの処理を繰り返す。 The determinations in S1205 in FIG. 12 and S1406 in FIG. 14 are performed when it is determined in S1204 and S1405 that the electroencephalogram data does not satisfy the condition. However, the determination as to whether the line-of-sight information satisfies the condition may be performed before S1204 and S1405. In this case, if it is determined that the line-of-sight information satisfies the condition, the processing proceeds to S1206 and S1407, and the processor 601 transmits an imaging command to the controller 112. That is, imaging is performed regardless of the electroencephalogram data. On the other hand, when it is determined that the line-of-sight information does not satisfy the condition, determinations in S1204 and S1405 are performed. When it is determined that the electroencephalogram data satisfies the condition, imaging is performed in S1206 and S1407, and when it is determined that the electroencephalogram data does not satisfy the condition, the processes from S1201 and S1401 are repeated.
　図１５は、コントローラ１１２から送信された撮影データ及び脳波データをＤＢ６０５及び６０６に格納する処理を示すフローチャートである。図１５の処理は、図８のＳ８０８においてコントローラ１１２から撮影データ及び脳波データが送信されると開始される。 FIG. 15 is a flowchart showing processing for storing the imaging data and brain wave data transmitted from the controller 112 in the DBs 605 and 606. The process of FIG. 15 is started when imaging data and electroencephalogram data are transmitted from the controller 112 in S808 of FIG.
　Ｓ１５０１において、プロセッサ６０１は、ＤＢ６０５への登録データとしての撮影データの受信を監視し、撮影データを受信したと判定した場合には、Ｓ１５０２に進む。Ｓ１５０２において、プロセッサ６０１は、撮影データ及び脳波データをＤＢ６０５及び６０６に登録する。 In S1501, the processor 601 monitors reception of shooting data as registration data in the DB 605, and if it is determined that shooting data has been received, the processor 601 proceeds to S1502. In S1502, the processor 601 registers imaging data and brain wave data in the DBs 605 and 606.
　Ｓ１５０３において、プロセッサ６０１は、脳波データのデータ量が類似性を学習する上で十分な量であるか否かを判定する。十分な量であると判定された場合には、Ｓ１５０４に進み、十分な量でないと判定された場合には、Ｓ１５０１からの処理を繰り返す。 In S1503, the processor 601 determines whether or not the data amount of the electroencephalogram data is sufficient for learning the similarity. If it is determined that the amount is sufficient, the processing proceeds to S1504. If it is determined that the amount is not sufficient, the processing from S1501 is repeated.
　Ｓ１５０４において、プロセッサ６０１は、ＤＢ６０５に登録されている各ライダーに対応する脳波データの特徴量を分析することによりその類似性を学習し、Ｓ１５０５において、類似パターンを表す特徴量を、撮影イベントに対応する相関データとして取得する。なお、Ｓ１５０４の処理がＤＢ６０５に登録されている脳波データに対する最初の処理である場合には、取得された類似パターンを表す特徴量を、撮影イベントに対応する相関データとしてサーバ１０８の所定の記憶領域に保存し、図１５の処理を終了する。一方、最初の処理でない場合には、取得された類似パターンを表す特徴量に基づいて、サーバ１０８の所定の記憶領域に保存されている相関データを修正（更新）し、その後、図１５の処理を終了する。 In S1504, the processor 601 learns the similarity by analyzing the feature quantity of the electroencephalogram data corresponding to each rider registered in the DB 605, and in S1505, the feature quantity representing the similar pattern corresponds to the shooting event. Obtained as correlation data. If the process of S1504 is the first process for the electroencephalogram data registered in the DB 605, the acquired feature quantity representing the similar pattern is used as correlation data corresponding to the imaging event as a predetermined storage area of the server 108. And the process of FIG. 15 is terminated. On the other hand, if it is not the first process, the correlation data stored in a predetermined storage area of the server 108 is corrected (updated) based on the acquired feature amount representing the similar pattern, and then the process of FIG. Exit.
　以上のように、本実施形態によれば、ライダーは、ライダー個人が撮影動作を起こすことなく、良い撮影タイミングと思われるときに撮影を行うことができる。そして、ライダーがその撮影結果から主観的に良いと思う撮影データを選択し、その選択された撮影データは、サーバ１０８に送信される。そのような構成により、ライダーが真に良い撮影タイミングと思う撮影データのみをサーバ１０８のＤＢ６０６に登録することができる。そして、サーバ１０８は、各ライダーが良いと思う撮影データに各対応する脳波データの類似性を学習し、類似パターンを表す特徴量を取得し、最新の相関データとして保存する。そして、サーバ１０８は、ライダーから送信された脳波データが相関データに類似していると判定した場合、撮影指令をバイク１０１のコントローラ１１２に送信する。 As described above, according to the present embodiment, the rider can take a picture when it is considered that the rider has a good shooting timing without causing a shooting operation. Then, the rider selects shooting data that is subjectively good from the shooting result, and the selected shooting data is transmitted to the server 108. With such a configuration, only shooting data that the rider thinks is truly good shooting timing can be registered in the DB 606 of the server 108. Then, the server 108 learns the similarity of the electroencephalogram data corresponding to the imaging data that each rider thinks is good, acquires the feature amount representing the similarity pattern, and stores it as the latest correlation data. When the server 108 determines that the electroencephalogram data transmitted from the rider is similar to the correlation data, the server 108 transmits an imaging command to the controller 112 of the motorcycle 101.
　また、本実施形態では、バイク１０１のコントローラ１１２から、脳波データをサーバ１０８へ送信し、操作指令を受信することによりカメラ３０５による撮影を行った。しかしながら、本実施形態を他の動作に適用するようにしても良い。以下、他の動作に適用する例を、上記の実施形態の動作と異なる点について説明する。 In this embodiment, the brain wave data is transmitted from the controller 112 of the motorcycle 101 to the server 108 and an operation command is received, so that the camera 305 performs imaging. However, this embodiment may be applied to other operations. Hereinafter, an example applied to other operations will be described with respect to differences from the operations of the above-described embodiment.
　図１６は、サーバ１０８が、脳波データを用いて、装置の動作指令を送信する処理を示すフローチャートである。Ｓ１６０１～Ｓ１６０２は、図１２のＳ１２０１～Ｓ１２０２における説明と同じである。Ｓ１６０３において、プロセッサ６０１は、脳波データを受信する。 FIG. 16 is a flowchart showing processing in which the server 108 transmits an operation command of the apparatus using brain wave data. S1601 to S1602 are the same as those described in S1201 to S1202 of FIG. In S1603, the processor 601 receives the electroencephalogram data.
　Ｓ１６０４において、プロセッサ６０１は、脳波データと相関データとを比較する。ここでの相関データは、所定の装置の動作イベントと対応するデータである。所定の装置の動作イベントとは、例えば、音楽再生装置（ステレオ等）や放送受信装置（ラジオ等）を動作させることである。Ｓ１６０５において、プロセッサ６０１は、脳波データが相関データの所定の類似範囲に含まれるか否かを判定する。所定の類似範囲に含まれると判定した場合、Ｓ１６０６に進む。Ｓ１６０６において、プロセッサ６０１は、所定の装置の動作指令をコントローラ１１２に送信する。Ｓ１６０６の後、Ｓ１６０１からの処理を繰り返す。一方、所定の類似範囲に含まれないと判定した場合、Ｓ１６０１からの処理を繰り返す。 In S1604, the processor 601 compares the electroencephalogram data with the correlation data. The correlation data here is data corresponding to an operation event of a predetermined apparatus. The predetermined device operation event is, for example, operating a music playback device (stereo or the like) or a broadcast receiving device (radio or the like). In step S1605, the processor 601 determines whether the electroencephalogram data is included in a predetermined similarity range of the correlation data. If it is determined to be included in the predetermined similar range, the process proceeds to S1606. In step S <b> 1606, the processor 601 transmits an operation command for a predetermined apparatus to the controller 112. After S1606, the processing from S1601 is repeated. On the other hand, if it is determined that it is not included in the predetermined similar range, the processing from S1601 is repeated.
　コントローラ１１２は、Ｓ１６０６における動作指令を受信すると、動作指令を所定の装置に送信する。本例では、例えば、ヘルメット１０９に、音楽を録音したＳＤカードを装着可能な音楽再生装置及びスピーカが構成され、ライダーが音楽再生装置で再生された音楽をスピーカで聴くことができるように構成されている。その場合、コントローラ１１２は、ヘルメット１０９の音楽再生装置による再生指令をヘルメット１０９に送信する。ヘルメット１０９のプロセッサ３０１は、ヘルメット１０９の音楽再生装置による音楽再生を行う。 When the controller 112 receives the operation command in S1606, the controller 112 transmits the operation command to a predetermined device. In this example, for example, the helmet 109 is configured with a music playback device and a speaker that can be loaded with an SD card recording music, and the rider can listen to the music played back with the music playback device through the speaker. ing. In that case, the controller 112 transmits a reproduction command from the music reproduction device of the helmet 109 to the helmet 109. The processor 301 of the helmet 109 performs music playback by the music playback device of the helmet 109.
　上記の図１６の動作は、図１４の相関データを取得した後の処理に対応する。相関データを取得する前、即ち脳波データの学習前の段階では、Ｓ１６０３で脳波データを受信すると、Ｓ１６０５に進み、脳波データが条件を満たすか否かを判定する。ここでは、例えば、プロセッサ６０１は、図１２のＳ１２０４と同様に、脳波データが表す電位が所定の活性化域に含まれるか否かを判定するようにしても良い。但し、その場合、Ｓ１６０６による動作指令の送信によって音楽再生が行われても、ライダーにとっては好ましくないタイミングである可能性がある。そのため、例えば、ライダーが音楽再生の開始後、所定の時間（例えば、１０秒）内に再生を中止した場合には、再生中止の旨とそのときの脳波データとを、コントローラ１１２を介してサーバ１０８に送信するようにする。そして、サーバ１０８のプロセッサ６０１は、再生が中止されたときの脳波データをＤＢ６０５へ登録しないようにし、類似性の学習対象外とする。また、ライダーが音楽再生の開始後、所定の時間以上、再生を継続した場合には、再生の旨とそのときの脳波データとを、コントローラ１１２を介してサーバ１０８に送信するようにする。そして、サーバ１０８のプロセッサ６０１は、再生されたときの脳波データをＤＢ６０５へ登録し、類似性の学習対象とする。 The operation of FIG. 16 described above corresponds to the processing after acquiring the correlation data of FIG. Before acquiring the correlation data, that is, before learning the electroencephalogram data, if the electroencephalogram data is received in S1603, the process proceeds to S1605, and it is determined whether the electroencephalogram data satisfies the condition. Here, for example, the processor 601 may determine whether or not the potential represented by the electroencephalogram data is included in a predetermined activation area, as in S1204 of FIG. However, in that case, there is a possibility that the timing is not preferable for the rider even if the music reproduction is performed by transmitting the operation command in S1606. Therefore, for example, if the rider stops the playback within a predetermined time (for example, 10 seconds) after the start of the music playback, a message indicating that the playback is stopped and the electroencephalogram data at that time are sent via the controller 112 to the server. 108 to be transmitted. Then, the processor 601 of the server 108 does not register the electroencephalogram data when the reproduction is stopped in the DB 605 and excludes it from the similarity learning target. In addition, when the rider continues playing for a predetermined time or more after the start of music playback, the fact that the rider is playing and the electroencephalogram data at that time are transmitted to the server 108 via the controller 112. Then, the processor 601 of the server 108 registers the electroencephalogram data at the time of reproduction in the DB 605 and sets it as a similarity learning target.
　以上のような構成により、例えば、ライダーが海岸等を走行している際に、ライダーが音楽再生動作を起こすことなく自動的に音楽を再生することができる。 With the configuration as described above, for example, when a rider is traveling on the beach, music can be automatically played back without causing the rider to perform music playback.
　また、音楽再生装置の代わりに放送受信装置が構成されているのであれば、コントローラ１１２は、ヘルメット１０９の放送受信装置による再生指令をヘルメット１０９に送信する。ヘルメット１０９のプロセッサ３０１は、ヘルメット１０９のラジオによる再生を行う。他の動作については、音楽再生装置の場合と同様である。また、音楽再生装置や放送受信装置がヘルメット１０９に構成されているとして説明したが、バイク１０１の本体に構成されていても良い。その場合には、コントローラ１１２は、再生指令を受信すると、音楽再生装置や放送受信装置による再生を行う。 In addition, if a broadcast receiving device is configured instead of the music playback device, the controller 112 transmits a playback command from the broadcast receiving device of the helmet 109 to the helmet 109. The processor 301 of the helmet 109 performs reproduction using the radio of the helmet 109. Other operations are the same as those in the music reproducing apparatus. Further, although the music playback device and the broadcast receiving device have been described as being configured in the helmet 109, they may be configured in the main body of the motorcycle 101. In that case, when receiving the reproduction command, the controller 112 performs reproduction by the music reproduction device or the broadcast reception device.
　また、所定の装置は、バイク１０１の車両構成要素であっても良い。つまり、動作指令とバイク１０１（車両）との連動を実現するようにしても良く、例えば、Ｓ１６０５で脳波データが条件を満たす場合に、Ｓ１６０６で、バイク１０１のアクセルやブレーキを制御する指令を動作指令として送信するようにしても良い。この場合の相関データは、例えば、睡眠時に発せられる脳波データである。Ｓ１６０５で脳波データが相関データの所定の類似範囲に含まれると判定した場合、Ｓ１６０６において、プロセッサ６０１は、アクセルやブレーキの制御指令を動作指令としてコントローラ１１２に送信する。そのような制御指令としては、例えば、アクセルの出力抑制や、アクセルやブレーキの急減な変化の抑制である。コントローラ１１２は、制御指令を受信すると、バイク１０１のＥＣＵ等と連携することにより上記の各制御を行う。また、プロセッサ６０１は、アクセルやブレーキの制御指令と合わせて、ヘルメット１０９のスピーカ等に警告音を鳴動させる指令を送信するようにしても良い。 Further, the predetermined device may be a vehicle component of the motorcycle 101. That is, the operation command and the motorcycle 101 (vehicle) may be linked. For example, when the electroencephalogram data satisfies the condition in S1605, the command for controlling the accelerator or brake of the motorcycle 101 is operated in S1606. It may be transmitted as a command. The correlation data in this case is, for example, brain wave data emitted during sleep. If it is determined in S1605 that the electroencephalogram data is included in the predetermined similar range of the correlation data, in S1606, the processor 601 transmits an accelerator or brake control command to the controller 112 as an operation command. Examples of such control commands include suppression of accelerator output and suppression of sudden changes in accelerator and brake. When the controller 112 receives the control command, the controller 112 performs each control described above in cooperation with the ECU of the motorcycle 101 and the like. Further, the processor 601 may transmit a command for sounding a warning sound to the speaker or the like of the helmet 109 together with the accelerator or brake control command.
　以上のような構成により、例えば、ライダーが居眠りをしているような状態に陥った場合に、事故の発生を回避することができる。また、上記では、睡眠時に発せられる脳波データを例として挙げたが、他にも疲労等の身体状態を表す脳波データであっても良い。その場合、Ｓ１６０６で、休憩を促すなどのアドバイス情報をバイク１０１のコントローラ１１２に送信するようにしても良い。コントローラ１１２は、例えば、アドバイス情報に基づいて、ガイダンス表示と休憩場所の位置情報を表示部４０４に表示しても良い。もしくは、ヘルメット１０９にスピーカを構成し、コントローラ１１２は、音声ガイダンスをヘルメット１０９のスピーカに再生するようにしても良い。睡眠時に発せられる脳波データや、疲労等の身体状態を表す脳波データは、類似性の学習によって得られるのではなく、予め定められた基準として設けられたものであっても良い。 With the configuration as described above, for example, when the rider falls into a state of falling asleep, the occurrence of an accident can be avoided. In the above description, the electroencephalogram data generated during sleep is taken as an example, but electroencephalogram data representing a physical state such as fatigue may be used. In that case, advice information such as prompting a break may be transmitted to the controller 112 of the motorcycle 101 in S1606. For example, the controller 112 may display guidance display and break location information on the display unit 404 based on the advice information. Alternatively, the helmet 109 may be configured with a speaker, and the controller 112 may reproduce the voice guidance on the helmet 109 speaker. The electroencephalogram data emitted during sleep and the electroencephalogram data representing a physical state such as fatigue may not be obtained by similarity learning, but may be provided as a predetermined reference.
　＜実施形態のまとめ＞
　上記実施形態の操作システム（１００）は、
　車両（１０１、１０２、１０３）の搭乗者の脳波信号に対応する脳波データを取得する取得手段と（Ｓ８０２）、脳波データを蓄積するデータベース（６０５）と、前記データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成手段と（Ｓ１５０３、Ｓ１５０４）、前記取得手段により取得された脳波データと前記生成手段により生成された前記相関データとに基づいて、前記指令を発信する発信手段と（Ｓ８０４）、前記発信手段により前記指令が発信されると、前記装置を動作させる制御手段と（Ｓ８０５）、を備えることを特徴とする。 <Summary of Embodiment>
The operation system (100) of the above embodiment includes:
Based on the acquisition means for acquiring the electroencephalogram data corresponding to the electroencephalogram signal of the passenger of the vehicle (101, 102, 103) (S802), the database (605) for accumulating the electroencephalogram data, and the electroencephalogram data accumulated in the database Generating means for generating correlation data associated with a command for operating the apparatus (S1503, S1504), based on the electroencephalogram data acquired by the acquiring means and the correlation data generated by the generating means Transmission means for transmitting the command (S804), and control means for operating the apparatus when the command is transmitted by the transmission means (S805).
　そのような構成により、例えば、ライダーが撮影タイミングが良いと感じたときに、ライダーが撮影動作を起こすことなく、カメラに撮影させる操作システムを実現することができる。 With such a configuration, for example, when the rider feels that the shooting timing is good, it is possible to realize an operation system that allows the camera to shoot without causing the rider to take a shooting operation.
　また、前記取得手段により取得された脳波データと前記生成手段により生成された前記相関データとに基づいて、前記指令を発信するか否かを判定する判定手段（Ｓ８０３）、をさらに備え、前記判定手段により前記指令を発信すると判定された場合、前記発信手段は、前記指令を発信することを特徴とする。 The determination unit further comprises a determination unit (S803) that determines whether to send the command based on the electroencephalogram data acquired by the acquisition unit and the correlation data generated by the generation unit. When it is determined that the command is transmitted by the means, the transmission means transmits the command.
　そのような構成により、例えば、カメラの撮影イベントと相関づけられた相関データと脳波データとの類似が判定された場合にカメラに撮影させることができる。 With such a configuration, for example, when the similarity between the correlation data correlated with the shooting event of the camera and the electroencephalogram data is determined, the camera can be shot.
　また、前記装置は、撮影手段（７０２）であり、前記撮影手段の撮影時の脳波信号に対応する脳波データを前記データベースに格納する格納手段（Ｓ８０８）、をさらに備えることを特徴とする。 Further, the apparatus is an imaging unit (702), and further includes a storage unit (S808) for storing brain wave data corresponding to an electroencephalogram signal at the time of imaging by the imaging unit in the database.
　そのような構成により、例えば、撮影時の脳波信号に対応する脳波データをデータベースに蓄積していくことができる。 With such a configuration, for example, electroencephalogram data corresponding to an electroencephalogram signal at the time of imaging can be accumulated in a database.
　前記データベースに蓄積されている脳波データは、前記撮影手段が撮影した撮影データと関連づけられていることを特徴とする。 The electroencephalogram data stored in the database is associated with imaging data captured by the imaging means.
　そのような構成により、例えば、脳波データと写真とを相互に関連付けてデータベースで管理することができる。 With such a configuration, for example, electroencephalogram data and photographs can be associated with each other and managed in a database.
　また、前記撮影手段が撮影した撮影データのうち、前記生成手段が前記相関データを生成するための撮影データを選択する選択手段（Ｓ８０７、１１０１）、をさらに備え、前記データベースに蓄積されている脳波データは、前記選択手段により選択された撮影データと関連づけられており、前記生成手段は、前記選択手段により選択された撮影データに関連づけられた脳波データに基づいて、前記相関データを生成することを特徴とする。 The electroencephalogram stored in the database further comprises selection means (S807, 1101) for the imaging means to select imaging data for generating the correlation data among the imaging data captured by the imaging means. The data is associated with the imaging data selected by the selection unit, and the generation unit generates the correlation data based on the electroencephalogram data associated with the imaging data selected by the selection unit. Features.
　そのような構成により、例えば、撮影後にライダーが真に良いと感じた写真のみを脳波データと関連づけてデータベースで管理することができる。 With such a configuration, for example, it is possible to manage only a photograph that the rider feels really good after shooting in a database in association with brain wave data.
　また、前記搭乗者の視線信号を検出する視線検出手段（Ｓ９０２、７０３）、をさらに備え、前記視線検出手段により検出された前記視線信号が条件を満たす場合、前記発信手段は、前記指令を発信することを特徴とする。 In addition, it further includes gaze detection means (S902, 703) for detecting a gaze signal of the passenger, and when the gaze signal detected by the gaze detection means satisfies a condition, the transmission means transmits the command. It is characterized by doing.
　そのような構成により、例えば、ライダーの目線が一定方向に一定時間固定された場合に、カメラに撮影させることができる。 With such a configuration, for example, when the rider's line of sight is fixed in a certain direction for a certain period of time, the camera can shoot.
　また、前記データベースに蓄積されている脳波データは、前記車両（１０１）の搭乗者と異なる車両（１０２、１０３）の搭乗者の脳波信号に対応する脳波データを含み、前記生成手段は、前記データベースに蓄積された複数の車両の搭乗者に対応する脳波データの類似性を解析することにより前記相関データを生成することを特徴とする。 The electroencephalogram data stored in the database includes electroencephalogram data corresponding to an electroencephalogram signal of a passenger of a vehicle (102, 103) different from the passenger of the vehicle (101), and the generation means includes the database The correlation data is generated by analyzing the similarity of the electroencephalogram data corresponding to the passengers of a plurality of vehicles stored in the vehicle.
　そのような構成により、例えば、複数の搭乗者から検出された脳波信号を基に、相関データを生成することができる。 With such a configuration, for example, correlation data can be generated based on brain wave signals detected from a plurality of passengers.
　また、前記操作システムは、サーバ（１０８）と、前記車両であるバイク（１０１、１０２、１０３）とを含み、前記サーバは、前記取得手段、前記データベース、前記生成手段、前記発信手段を備え、前記バイクは、前記制御手段を備え、前記取得手段は、前記搭乗者が着用するヘルメット（１０９、１１０、１１１）により検出された脳波信号に対応する脳波データを取得することを特徴とする。 The operation system includes a server (108) and a motorcycle (101, 102, 103) as the vehicle. The server includes the acquisition unit, the database, the generation unit, and the transmission unit. The motorcycle includes the control unit, and the acquisition unit acquires brain wave data corresponding to a brain wave signal detected by a helmet (109, 110, 111) worn by the passenger.
　そのような構成により、例えば、バイクとサーバとヘルメットを含む構成を操作システムとして実現することができる。 With such a configuration, for example, a configuration including a motorcycle, a server, and a helmet can be realized as an operation system.
　また、前記装置は、前記ヘルメット（１０９、１１０、１１１）に構成されている撮影手段（７０２）であることを特徴とする。 Further, the apparatus is characterized in that it is a photographing means (702) configured in the helmet (109, 110, 111).
　そのような構成により、例えば、撮影手段は、ライダーの概ね目線方向を被写体として撮影することができる。 With such a configuration, for example, the photographing unit can photograph a rider in the direction of the eyes of the rider.
　１００　操作システム：　１０８　サーバ：　１０９、１１０、１１１　ヘルメット：　１１２、１１３、１１４　コントローラ 100 Operation system: 108 Server: 109, 110, 111 Helmet: 112, 113, 114 Controller

Claims (12)

  1. 　脳波データを蓄積するデータベースと、
    　前記データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成手段と、
    　車両の搭乗者の脳波信号に対応する脳波データを取得する取得手段と、
    　前記取得手段により取得された脳波データと前記生成手段により生成された前記相関データとに基づいて、前記指令を発信する発信手段と、
    　前記発信手段により前記指令が発信されると、前記装置を動作させる制御手段と、
    　を備えることを特徴とする操作システム。     A database for accumulating brain wave data;
    Generating means for generating correlation data associated with a command for operating the device based on the electroencephalogram data stored in the database;
    Obtaining means for obtaining electroencephalogram data corresponding to an electroencephalogram signal of a vehicle occupant;
    Based on the electroencephalogram data acquired by the acquisition means and the correlation data generated by the generation means, transmission means for transmitting the command;
    When the command is transmitted by the transmission means, control means for operating the device;
    An operation system comprising:
  2. 　前記取得手段により取得された脳波データと前記生成手段により生成された前記相関データとに基づいて、前記指令を発信するか否かを判定する判定手段、をさらに備え、
    　前記判定手段により前記指令を発信すると判定された場合、前記発信手段は、前記指令を発信する、
    　ことを特徴とする請求項１に記載の操作システム。     Determination means for determining whether to send the command based on the electroencephalogram data acquired by the acquisition means and the correlation data generated by the generation means;
    When it is determined that the command is transmitted by the determination unit, the transmission unit transmits the command.
    The operation system according to claim 1.
  3. 　前記装置は、撮影手段であり、
    　前記撮影手段の撮影時の脳波信号に対応する脳波データを前記データベースに格納する格納手段、をさらに備えることを特徴とする請求項１又は２に記載の操作システム。     The apparatus is a photographing means;
    The operation system according to claim 1, further comprising storage means for storing brain wave data corresponding to an electroencephalogram signal at the time of photographing by the photographing means in the database.
  4. 　前記データベースに蓄積されている脳波データは、前記撮影手段が撮影した撮影データと関連づけられていることを特徴とする請求項３に記載の操作システム。 4. The operation system according to claim 3, wherein the electroencephalogram data stored in the database is associated with imaging data captured by the imaging means.
  5. 　前記撮影手段が撮影した撮影データのうち、前記生成手段が前記相関データを生成するための撮影データを選択する選択手段、をさらに備え、
    　前記データベースに蓄積されている脳波データは、前記選択手段により選択された撮影データと関連づけられており、
    　前記生成手段は、前記選択手段により選択された撮影データに関連づけられた脳波データに基づいて、前記相関データを生成する、
    　ことを特徴とする請求項４に記載の操作システム。     A selection means for selecting photographing data for generating the correlation data among the photographing data photographed by the photographing means;
    The electroencephalogram data stored in the database is associated with the imaging data selected by the selection means,
    The generating means generates the correlation data based on brain wave data associated with the imaging data selected by the selecting means;
    The operation system according to claim 4.
  6. 　前記車両の搭乗者の視線信号を検出する視線検出手段、をさらに備え、
    　前記視線検出手段により検出された前記視線信号が条件を満たす場合、前記発信手段は、前記指令を発信する、
    　ことを特徴とする請求項３乃至５のいずれか１項に記載の操作システム。     Line-of-sight detection means for detecting line-of-sight signals of passengers of the vehicle,
    When the line-of-sight signal detected by the line-of-sight detection unit satisfies a condition, the transmission unit transmits the command.
    The operation system according to any one of claims 3 to 5, wherein:
  7. 　前記データベースに蓄積されている脳波データは、前記車両の搭乗者と異なる車両の搭乗者の脳波信号に対応する脳波データを含み、
    　前記生成手段は、前記データベースに蓄積された複数の車両の搭乗者に対応する脳波データの類似性を解析することにより前記相関データを生成する、
    　ことを特徴とする請求項１乃至６のいずれか１項に記載の操作システム。     The electroencephalogram data stored in the database includes electroencephalogram data corresponding to an electroencephalogram signal of a vehicle occupant different from the vehicle occupant,
    The generating means generates the correlation data by analyzing the similarity of electroencephalogram data corresponding to passengers of a plurality of vehicles stored in the database;
    The operation system according to any one of claims 1 to 6, wherein
  8. 　前記操作システムは、サーバと、前記車両である鞍乗り型車両とを含み、
    　前記サーバは、前記取得手段、前記データベース、前記生成手段、前記発信手段を備え、
    　前記鞍乗り型車両は、前記制御手段を備え、
    　前記取得手段は、前記鞍乗り型車両の搭乗者が着用するヘルメットにより検出された脳波信号に対応する脳波データを取得する、
    　ことを特徴とする請求項１乃至７のいずれか１項に記載の操作システム。     The operation system includes a server and a saddle-ride type vehicle that is the vehicle,
    The server includes the acquisition unit, the database, the generation unit, and the transmission unit,
    The saddle riding type vehicle includes the control means,
    The acquisition means acquires electroencephalogram data corresponding to an electroencephalogram signal detected by a helmet worn by a rider of the saddle-ride type vehicle,
    The operation system according to claim 1, wherein:
  9. 　前記装置は、前記ヘルメットに構成されている撮影手段であることを特徴とする請求項８に記載の操作システム。 The operation system according to claim 8, wherein the device is a photographing unit configured in the helmet.
  10. 　脳波データを蓄積するデータベースと、
    　前記データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成手段と、
    　車両の搭乗者の脳波信号に対応する脳波データを取得する取得手段と、
    　前記取得手段により取得された脳波データと前記生成手段により生成された前記相関データとに基づいて、前記指令を発信する発信手段と、
    　を備えることを特徴とするサーバ。     A database for accumulating brain wave data;
    Generating means for generating correlation data associated with a command for operating the device based on the electroencephalogram data stored in the database;
    Obtaining means for obtaining electroencephalogram data corresponding to an electroencephalogram signal of a vehicle occupant;
    Based on the electroencephalogram data acquired by the acquisition means and the correlation data generated by the generation means, transmission means for transmitting the command;
    A server comprising:
  11. 　データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成ステップと、
    　車両の搭乗者の脳波信号に対応する脳波データを取得する取得ステップと、
    　前記取得ステップにおいて取得された脳波データと前記生成ステップにおいて生成された前記相関データとに基づいて、前記指令を発信する発信ステップと、
    　前記発信ステップにおいて前記指令が発信されると、前記装置を動作させる制御ステップと、
    　を有することを特徴とする制御方法。     Generating a correlation data associated with a command for operating the device based on the electroencephalogram data stored in the database;
    An acquisition step of acquiring electroencephalogram data corresponding to an electroencephalogram signal of a vehicle occupant;
    A transmission step of transmitting the command based on the electroencephalogram data acquired in the acquisition step and the correlation data generated in the generation step;
    When the command is transmitted in the transmission step, a control step for operating the device;
    A control method characterized by comprising:
  12. 　データベースに蓄積された脳波データに基づいて、装置を動作させるための指令と関連づけられた相関データを生成する生成ステップと、
    　車両の搭乗者の脳波信号に対応する脳波データを取得する取得ステップと、
    　前記取得ステップにおいて取得された脳波データと前記生成ステップにおいて生成された前記相関データとに基づいて、前記指令を発信する発信ステップと、
    　前記発信ステップにおいて前記指令が発信されると、前記装置を動作させる制御ステップと、
    　をコンピュータに実行させることを特徴とするプログラム。     Generating a correlation data associated with a command for operating the device based on the electroencephalogram data stored in the database;
    An acquisition step of acquiring electroencephalogram data corresponding to an electroencephalogram signal of a vehicle occupant;
    A transmission step of transmitting the command based on the electroencephalogram data acquired in the acquisition step and the correlation data generated in the generation step;
    When the command is transmitted in the transmission step, a control step for operating the device;
    A program that causes a computer to execute.
PCT/JP2017/013623 2017-03-31 2017-03-31 Manipulation system, server, control method in manipulation system, and program WO2018179346A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/013623 WO2018179346A1 (en) 2017-03-31 2017-03-31 Manipulation system, server, control method in manipulation system, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2017/013623 WO2018179346A1 (en) 2017-03-31 2017-03-31 Manipulation system, server, control method in manipulation system, and program

Publications (1)

Publication Number Publication Date
WO2018179346A1 true WO2018179346A1 (en) 2018-10-04

Family

ID=63674502

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/013623 WO2018179346A1 (en) 2017-03-31 2017-03-31 Manipulation system, server, control method in manipulation system, and program

Country Status (1)

Country Link
WO (1) WO2018179346A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021089544A (en) * 2019-12-03 2021-06-10 富士フイルムビジネスイノベーション株式会社 Information processing system and program

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004152002A (en) * 2002-10-30 2004-05-27 Mitsubishi Electric Corp Control unit using brain wave signal
WO2012060039A1 (en) * 2010-11-02 2012-05-10 Ｎｅｃカシオモバイルコミュニケーションズ株式会社 Information processing system and information processing method
WO2013128920A1 (en) * 2012-02-27 2013-09-06 ヤマハ発動機株式会社 Operation-state-sharing device and operation-state-sharing system
JP2017042261A (en) * 2015-08-25 2017-03-02 マツダ株式会社 Electroencephalogram acquisition method and electroencephalogram acquisition device
JP2017061192A (en) * 2015-09-24 2017-03-30 日産自動車株式会社 Support apparatus and support method

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004152002A (en) * 2002-10-30 2004-05-27 Mitsubishi Electric Corp Control unit using brain wave signal
WO2012060039A1 (en) * 2010-11-02 2012-05-10 Ｎｅｃカシオモバイルコミュニケーションズ株式会社 Information processing system and information processing method
WO2013128920A1 (en) * 2012-02-27 2013-09-06 ヤマハ発動機株式会社 Operation-state-sharing device and operation-state-sharing system
JP2017042261A (en) * 2015-08-25 2017-03-02 マツダ株式会社 Electroencephalogram acquisition method and electroencephalogram acquisition device
JP2017061192A (en) * 2015-09-24 2017-03-30 日産自動車株式会社 Support apparatus and support method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021089544A (en) * 2019-12-03 2021-06-10 富士フイルムビジネスイノベーション株式会社 Information processing system and program
JP7320261B2 (en) 2019-12-03 2023-08-03 株式会社Agama-X Information processing system, method, and program

Similar Documents

Publication Publication Date Title
EP3067827B1 (en) Driver distraction detection system
JP6862629B2 (en) Vehicle management server, in-vehicle terminal, watching method, and program in the watching transfer system
US11165955B2 (en) Album generation apparatus, album generation system, and album generation method
JP5176311B2 (en) Image display system, display device, and display method
KR101250929B1 (en) A two ear-piece insertable hearing aid having a vibration pattern alert function
US11057728B2 (en) Information processing apparatus, information processing method, and program
JP2016052881A (en) Travel control system for vehicle
KR101250951B1 (en) Two ears insertable hearing aid having a danger detecting function and system for detecting dangerous factors using thereof
US11250875B2 (en) Behavior support system, behavior support apparatus, behavior support method, and storage medium storing program thereof
CN106060569A (en) Live video broadcast method and device, apparatus and system
CN109885368A (en) A kind of interface display anti-fluttering method and mobile terminal
JP2019086805A (en) In-vehicle system
KR20150137453A (en) Mobile device and control method using brainwave
CN111196230A (en) Driving support device, wearable device, driving support method, and program
JP7287257B2 (en) Image processing device, display system, program and image processing method
KR101741074B1 (en) Automatic recording device and method for car accident
CN111133752B (en) Expression recording system
WO2018179346A1 (en) Manipulation system, server, control method in manipulation system, and program
WO2018075523A1 (en) Audio/video wearable computer system with integrated projector
CN113853571A (en) Information processing apparatus and information processing terminal
CN109361727B (en) Information sharing method and device, storage medium and wearable device
JP5690580B2 (en) Audiovisual terminal, audiovisual authentication system, and control program
CN113965726A (en) Method, device and system for processing traffic video
US11792484B2 (en) Information providing system, information providing method and management apparatus for information providing system for providing information obtained through a robot
US11518036B2 (en) Service providing system, service providing method and management apparatus for service providing system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17902623

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17902623

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP