JP2012185632A - Electronic apparatus, processing system, and processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an electronic apparatus which performs appropriate processing according to a movement of a target person.SOLUTION: An electronic apparatus comprises: a first input unit (31) for inputting a detection result by a biometric sensor (21) which detects change in biological information of a target person; a second input unit (32) for inputting a recognition result by a recognition devices (13, 23, 33) which recognize a movement of the target person; and a processing unit (40) for performing processing according to the movement of the target person based on input results in the first and second input units.

Description

  The present invention relates to an electronic device, a processing system, and a processing program.

  An interface device has been proposed in which a user operates a device by making a gesture toward a camera (for example, Patent Document 1).

JP 2004-246856 A

  However, the conventional interface device cannot always accurately determine whether the action performed by the user is a gesture.

  Therefore, the present invention has been made in view of the above problems, and an object thereof is to provide an electronic device, a processing system, and a processing program capable of performing appropriate processing according to the operation of the subject.

  An electronic apparatus according to the present invention includes a first input unit (31) for inputting a detection result of a biological sensor (21) that detects a change in biological information of a subject, and a recognition device (13, 13) that recognizes the operation of the subject. 23, 33) a second input unit (32) for inputting the recognition result, and a processing unit (40) for performing processing according to the operation of the subject based on the input results of the first and second input units. And an electronic device (19).

  In this case, the recognition device includes a plurality of different sensors, and the processing unit inputs the plurality of inputs that the second input unit inputs even when there is no input of changes in biological information to the first input unit. Based on the recognition result of the sensor, it is possible to perform processing according to the movement of the subject.

  The plurality of sensors includes an imaging device and a contact-type sensor, and includes a control unit (40) that performs imaging by the imaging device when the contact-type sensor recognizes an action of a subject. be able to. In this case, the imaging device may be provided above the contact sensor.

  In the electronic device of the present invention, at least a part of the recognition device may be provided in the vicinity of the biometric sensor. The processing unit may perform a process of outputting a sound by a speaker (15) that outputs a sound toward the target person, and the speaker is directed to output the sound in a limited direction. It may be a sexual speaker.

  Moreover, in the electronic device of the present invention, the processing unit can accept a voice uttered by the subject from a voice input device (16) that inputs voice. In this case, it is good also as providing the audio | voice identification part (34) which identifies the audio | voice received from the said audio | voice input apparatus.

  Moreover, the electronic device of the present invention includes a time measuring unit (20) that measures a time when the biological information of the subject changes and a time when the operation of the subject is recognized, and the processing unit May execute processing according to the input result and the timing result of the timing unit. In addition, when the target person exists in the movable mobile device (50), the processing unit performs the processing in consideration of the detection result of the detection device (11) that detects the movement of the mobile device. It may be done.

  The processing system of the present invention includes a biological sensor (21) for detecting a change in biological information of a subject, a recognition device (13, 23, 33) for recognizing the motion of the subject, and an electronic device (19) of the present invention. And a processing system (100).

  The processing program of the present invention includes a first input step for inputting a detection result of the biosensor (21) for detecting a change in the biometric information of the subject, and a recognition device (13, 23, 33 for recognizing the motion of the subject. ) To the computer (90), the second input step for inputting the recognition result of), and the processing step for performing processing according to the action of the subject based on the input results of the first and second input steps. Is a processing program to be executed.

  In addition, in order to explain the present invention in an easy-to-understand manner, the above description has been made in association with the reference numerals of the drawings representing one embodiment. However, the present invention is not limited to this, and the configuration of an embodiment described later is provided. May be modified as appropriate, or at least a part thereof may be replaced with another component. Further, the configuration requirements that are not particularly limited with respect to the arrangement are not limited to the arrangement disclosed in the embodiment, and can be arranged at a position where the function can be achieved.

  The electronic device, the processing system, and the processing program of the present invention have an effect of being able to perform appropriate processing according to the operation of the subject.

It is a figure showing roughly the composition of the trouble response system concerning one embodiment. It is a figure which shows the example of mounting in the train of a trouble handling system. It is a figure which shows the pressure sensor and biological sensor which were provided in the strap. It is a figure which shows an example of a reference image. It is a hardware block diagram of an electronic device. It is a functional block diagram of an electronic device. It is a flowchart which shows the process of a trouble response system (process and control part of an electronic device).

  Hereinafter, the trouble handling system 100 according to an embodiment will be described in detail. FIG. 1 is a block diagram showing a schematic configuration of the trouble handling system 100. As shown in FIG. 1, the trouble handling system 100 includes a processing device 19, a main body 12, a biosensor 21, a piezoelectric sensor 13, a pressure sensor 23, a vehicle sensor 11, an air conditioning unit 29, and a timer 20. And a flash memory 30.

  FIG. 2 shows an implementation example of the trouble handling system 100. As shown in FIG. 2, in this embodiment, the trouble handling system 100 is provided in a train 50. For example, the processing device 19 and the main body 12 are provided on the ceiling of the train 50, and the piezoelectric sensor 13 is provided on the floor of the train 50. In addition, the living body sensor 21 and the pressure sensor 23 are provided on the straps 22 (see FIG. 3) in the train 50. Furthermore, it is assumed that other devices are also provided in the train 50.

  As shown in FIG. 1, the main body unit 12 includes an imaging unit 14, a speaker 15, a microphone 16, an LED (Light Emitting Diode) 18, and a driving device 9. In the main body 12, each device may be configured as one unit, but at least one device may be separately arranged.

  The imaging unit 14 includes an imaging lens, an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), a control circuit that controls the imaging device, and the like. Since the imaging unit 14 is provided on the ceiling of the train 50 as described above, the imaging unit 14 mainly images the passenger's head. Moreover, the imaging part 14 images the said passenger's face, when a passenger looks at the direction of a ceiling. The reason why the imaging unit 14 mainly images the head is to protect passenger privacy.

  The speaker 15 is used for making an announcement for suppressing a trouble when a trouble occurs in the vehicle, and for asking a passenger whether the trouble has occurred. The speaker 15 is synthesized based on an instruction from the processing device 19 (the processing / control unit 40 (see FIG. 6)) using, for example, a speech synthesis technique such as “Is it all right” or “Please calm down”. Is output. Various speakers can be used as the speaker 15. For example, a directional speaker or a superdirectional speaker that has an ultrasonic transducer and transmits sound only in a limited direction can be used. When a directional speaker is used, a voice can be emitted not to the entire vehicle but to the vicinity where the trouble occurs.

  The microphone 16 collects sound in the vehicle. For example, the microphone 16 collects voices such as “help” and “car” uttered by passengers in the event of trouble and inputs them to the processing device 19 (processing / control unit 40).

  The LED 18 emits light toward the vicinity where the trouble is occurring, and informs the passengers, station staff, and the like that the trouble is occurring.

  The drive device 9 includes, for example, a voice coil motor, and adjusts the position and orientation of the imaging unit 14, the speaker 15, the microphone 16, and the LED 18.

  One or a plurality of main body portions 12 configured as described above can be provided for each vehicle (in FIG. 2, the case of two units is illustrated). The number of the main body units 12 provided can be determined according to the imaging region of the imaging unit 14 (so that the entire vehicle can be imaged), for example.

  The piezoelectric sensor 13 has a piezo element, and electrically detects vibration by converting an externally applied force into a voltage by a piezoelectric effect. If a large number of piezoelectric sensors 13 are arranged so as to cover the entire area in the vehicle, it is possible to detect at which position in the vehicle the vibration is generated.

  For example, it is assumed that a woman in a vehicle is in trouble with being surrounded by a plurality of men and has stepped strongly on the floor (piezoelectric sensor 13) several times. In this case, if the piezoelectric sensor 13 detects the vibration and transmits the detection result to the processing device 19 (processing / control unit 40), the processing device 19 (processing / control unit 40) may cause trouble. That position can be detected. The details of the trouble detection method will be described later.

  The timer 20 has a time measuring function and measures the time when the piezoelectric sensor 13 detects vibration. For example, when the vibration is detected continuously for 5 seconds or more, or when vibration is detected intermittently within a predetermined time (for example, 30 seconds), the timer 20 causes the processing device 19 (processing / control unit 40) to To that effect.

  The biological sensor 21 detects biological information such as heart rate, blood oxygen concentration, blood pressure, and the like, and includes an LED 24, a photo sensor 25, and a sweat sensor 26 as shown in FIG. As shown in FIG. 3, the LED 24, the photosensor 25, and the sweat sensor 26 are provided on the handrail portion 22 a of the strap 22 provided in the train 50. Specifically, a plurality of LEDs 24 and photosensors 25 are alternately arranged on the handrail portion 22a, and a pair of sweating sensors 26 are provided so as to sandwich the LEDs 24 and the photosensors 25.

  The LED 24 and the photosensor 25 detect the heart rate and blood oxygen concentration by receiving light reflected by the photosensor 25 when the light emitted from the LED 24 is applied to the finger. The perspiration sensor 26 measures the impedance of the hand with a plurality of electrodes and detects the perspiration amount. The number and arrangement of the LEDs 24, photosensors 25, and sweating sensors 26 can be set as appropriate.

  Further, it is assumed that a pressure sensor 23 is also provided on the handrail portion 22a of the strap 22 shown in FIG. As the pressure sensor 23, a strain sensor may be used, or a sensor that detects pressure from a change in capacitance may be used. The pressure sensor 23 detects that the passenger is caught in the strap 22 or makes a different grip (gesture) such as strongly holding the strap 22 when the passenger is involved in trouble. Detect that. The number and arrangement of the pressure sensors 23 can be set as appropriate.

  In the present embodiment, a part of the biosensor 21 and the pressure sensor 23 are arranged close to each other as shown in FIG. 3, but they may be provided separately or may be a single unit. It is assumed that the position of the strap 22 is known in advance, and the information on the position of each strap 22 is stored in the flash memory 30 or the like.

  The vehicle sensor 11 includes a vibration sensor that detects vibration of the train itself that occurs when the train travels, stops, and the like. The vehicle sensor 11 may include a temperature sensor that detects the temperature inside the vehicle. The detection result of the vehicle sensor 11 is transmitted to the processing device 19 (processing / control unit 40).

  The air conditioning unit 29 performs air conditioning in the vehicle. In the present embodiment, the processing device 19 (processing / control unit 40) is based on the number of heads (that is, the number of passengers) imaged by the imaging unit 14. ).

  The flash memory 30 is a non-volatile memory that stores various data. In the present embodiment, the flash memory 30 stores a reference image indicating the positions of hands and feet with respect to the passenger's head. Has been. FIG. 4 is a diagram illustrating an example of a reference image. In FIG. 4, the area surrounded by the broken line is the presence range of the hand based on the position of the head (the range that is likely to exist), and the area surrounded by the alternate long and short dash line indicates the position of the head. It is the range of foot presence as a reference.

  Next, the processing device 19 will be described in detail. The processing device 19 controls the entire trouble handling system 100, and determines whether or not a trouble has occurred in the vehicle based on outputs from the biological sensor 21, the piezoelectric sensor 13, the pressure sensor 23, and the like. Further, the processing device 19 causes the main body unit 12 and the like to perform operations and processing that cause the trouble to calm down when trouble occurs.

  FIG. 5 shows a hardware configuration diagram of the processing device 19. As shown in FIG. 5, the processing device 19 includes a CPU 90, a ROM 92, a RAM 94, a storage unit (here, HDD (Hard Disk Drive)) 96, and the like, and each component of the processing device 19 is connected to a bus 98. Has been. In the processing device 19, the functions of each unit in FIG. 6 are realized by the CPU 90 executing the processing program stored in the ROM 92 or the HDD 96.

  FIG. 6 shows a functional block diagram of the processing device 19. As shown in FIG. 6, the processing device 19 has a biological information input unit 31, an operation information input unit 32, a face recognition unit 33, a voice recognition unit 34, and a processing / control unit by the CPU 90 executing a processing program. The function as 40 is demonstrated.

  A detection result detected by the biometric sensor 21 is input to the biometric information input unit 31. The biological information input unit 31 outputs the input information to the processing / control unit 40.

  The operation information input unit 32 receives the detection results detected by the piezoelectric sensor 13 and the pressure sensor 23 and the recognition result of the face recognition unit 33 described later. The operation information input unit 32 outputs the input information to the processing / control unit 40.

  The face recognition unit 33 acquires an image captured by the imaging unit 14 and detects a face image in the image. The face recognition unit 33 determines a face by detecting facial features such as eyes, nose, and mouth as images. In the present embodiment, since the imaging unit 14 is provided on the ceiling of the vehicle, the face recognition unit 33 determines whether the substantially circular image included in the image captured by the imaging unit 14 is the head. It can also be said that the face is determined. The face recognition unit 33 also detects the movement of the head without contact. In some cases, it is difficult to move the face in the car to see the ceiling. Therefore, the face recognizing unit 33 may adopt an algorithm that the face is detected when the passenger raises his chin and the forehead and eyes are imaged by the imaging unit 14.

  The speech recognition unit 34 has a speech recognition dictionary, and identifies speech input from the microphone 16 using the speech recognition dictionary. In the present embodiment, it is assumed that a voice uttered in an emergency such as “help” or “car” is registered in the voice recognition dictionary. It is assumed that not only the voice but also the magnitude (dB) of the collected voice is input from the microphone 16 to the processing / control unit 40. In FIG. 6, the voice input from the microphone 16 to the processing device 19 is input to the voice recognition unit 34 via the processing / control unit 40, but is not limited thereto. A voice may be directly input from the microphone 16 to the voice recognition unit 34.

  The processing / control unit 40 performs various processes using information input from inside or outside of the processing device 19 and controls devices inside or outside the processing device 19. For example, the processing / control unit 40 uses the reference image stored in the flash memory 30, so that the biometric sensor 21, the piezoelectric sensor 13, the pressure is detected based on the head or face image recognized by the face recognition unit 33. It is determined whether the output of the sensor 23 is an output from the same person (one passenger). In this case, it is determined whether or not they are the same person by enlarging, reducing, and rotating the reference image and performing pattern matching with the passenger. Note that the processing / control unit 40 can enlarge or reduce the reference image based on the size of the head. This is because the physique differs depending on the size of the head. The processing / control unit 40 determines the positions of the pressure sensor 23 that has detected the gesture and the biosensor 21 that has detected a change in the biometric information based on the position information of the strap 22 (stored in the flash memory 30 or the like). Can be obtained. Note that men, women, children, and the like may be stored in the flash memory 30 from their average physiques as reference images.

  Further, the processing / control unit 40 determines whether or not the passenger is involved in a trouble in the vehicle based on the detection results of the biosensor 21, the piezoelectric sensor 13, and the pressure sensor 23. Here, the passenger involved in the trouble changes the biological information, or gestures (action) to step on the floor several times, gestures (action) to hold the strap 22 strongly, or raise the face to the ceiling Do gestures (actions) such as watching. In addition, the passenger involved in the trouble may shake his / her feet tremblingly or he / she may unconsciously and strongly grip the strap 22 so that the passenger can unconsciously perform a gesture (action). Therefore, the processing / control unit 40 can determine whether or not trouble has occurred based on information from the biological sensor 21, the piezoelectric sensor 13, the pressure sensor 23, and the face recognition unit 33. In addition, since the piezoelectric sensor 13, the pressure sensor 23, and the face recognition part 33 are detecting the above gestures (action), it can be said that they are gesture (action) detection parts. Hereinafter, the piezoelectric sensor 13, the pressure sensor 23, and the face recognition unit 33 are collectively referred to as a gesture detection unit (13, 23, 33).

  Further, the processing / control unit 40 controls the driving device 9 that drives the main body unit 12 to emit sound or light toward the place where the trouble has occurred, or the sound that is emitted at the place where the trouble has occurred. To collect sound. Furthermore, the processing / control unit 40 turns on a switch such as the microphone 16 at a timing when it is determined that a trouble has occurred based on information from the biological sensor 21, the piezoelectric sensor 13, the pressure sensor 23, and the face recognition unit 33. It is also possible to perform such control (normally the off state is maintained). Thereby, energy saving can be achieved. Further, from the viewpoint of energy saving, the processing / control unit 40 detects that the passenger is caught on the strap by the LED 24 provided on the strap 22 or the pressure sensor 23 near the photo sensor 25. Only the LED 24 may emit light.

  Next, the processing and operation of the trouble handling system 100 configured as described above will be described with reference to the flowchart of FIG.

  In the process of FIG. 7, in step S <b> 10, the biological information input unit 31 inputs biological information input from the biological sensor 21 to the processing / control unit 40. Specifically, the biological information input unit 31 inputs the heart rate and blood oxygen concentration detected by the LED 24 and the photosensor 25 to the processing / control unit 40 and the sweating detected by the sweating sensor 26. Enter the amount. The biological information input from the biological sensor 21 to the biological information input unit 31 may be one piece of information or a plurality of pieces of information. The biological information may include blood pressure information, for example. In step S <b> 10, the biological information input unit 31 repeatedly inputs biological information of a plurality of passengers to the processing / control unit 40.

  Next, in step S <b> 12, the processing / control unit 40 detects a gesture using the detection information of the piezoelectric sensor 13 and the pressure sensor 23 and the recognition result of the face recognition unit 33. In the present embodiment, gesture detection is performed using the piezoelectric sensor 13, the pressure sensor 23, and the face recognition unit 33, but the piezoelectric sensor 13, the pressure sensor 23, and the face recognition unit 17 depend on the trouble detection situation. Gesture detection may be performed using at least one. Moreover, it is good also as performing gesture detection using a sensor different from the piezoelectric sensor 13, the pressure sensor 23, and the face recognition part 17. FIG. The processing / control unit 40 performs imaging by the imaging unit 14 when there is at least one of detection of biological information by the biological sensor 21 and gesture detection by the piezoelectric sensor 13 or the pressure sensor 23, and otherwise The switch of the imaging unit 14 may be turned off (or power is not supplied).

  Note that the execution order of step S10 and step S12 may be switched. In this case, in the strap 22 where there was no output from the pressure sensor 23 in step S10, the detection by the biological sensor 21 in step S12 may be omitted. In this case, power supply to the biosensor 21 may be performed at a pressure detection timing by the pressure sensor 23 provided on the same strap 22.

  Next, in step S14, the processing / control unit 40 determines whether or not a trouble has occurred in the train 50 based on the results of steps S10 and S12. Specifically, first, the processing / control unit 40 uses the reference image (FIG. 4) stored in the flash memory 30 to detect the position of the passenger's head or face, the biosensor 21 and the gesture detection unit (piezoelectric). From the positional relationship (the positional relationship between the hand and foot) of the sensor 13 and the pressure sensor 23), it is determined which biological information of which passenger is changing, which passenger is making a gesture, and whether it is the same passenger. To do. Then, the processing / control unit 40 determines whether or not a trouble has occurred after the passenger has been identified. In this case, the processing / control unit 40 determines that a trouble has occurred when the following criteria (a) to (c) are satisfied.

(A) Regarding the same passenger, when the biosensor 21 detects a change in heart rate or blood oxygen concentration, and at least one of the gesture detectors (13, 23, 33) detects a passenger's gesture.
(B) As a result of detection by the vehicle sensor 11 (temperature sensor), the perspiration sensor 26 detects perspiration of a predetermined amount or more with respect to the same passenger even though the temperature inside the vehicle is not high (for example, 23 ° C. or less). When at least one sensor of the gesture detection unit (13, 23, 33) detects a passenger's gesture.
(C) When there is no output from the pressure sensor 23 (that is, no output from the biological sensor 21 is obtained) with respect to the same passenger, but a gesture of a predetermined amount or more is detected from the piezoelectric sensor 13 and the face recognition unit 33.

In addition, when providing the infrared sensor which can detect a passenger's body temperature in a vehicle, when the process / control part 40 satisfy | fills the judgment criteria of the following (d), the trouble has generate | occur | produced. You may judge.
(D) Regarding the same passenger, there is no output from the pressure sensor 23, but an increase in body temperature is detected by the infrared sensor, and a gesture of a predetermined amount or more is detected from at least one of the piezoelectric sensor 13 and the face recognition unit 17 if you did this.

  Here, the infrared sensor detects the amount of infrared emission energy emitted from the passenger and converts it into temperature, and can detect a wide range of surface temperature distributions. In this case, the occurrence of trouble can be detected by detecting the temperature change of the passenger's head. If a non-contact sensor such as an infrared camera is used, the passenger's biological information can be obtained without giving (grabbing) a special sensor to the passenger.

  In the determinations (a) to (d) above, the processing / control unit 40 determines that the change in the output value of the biosensor 21 has continued for 5 seconds or more based on the time measurement result of the timer 20, or the biosensor 21. When the output value of the biosensor 21 changes intermittently within 30 seconds, it is determined that the output value of the biosensor 21 has changed. However, the present invention is not limited to this, and the processing / control unit 40 changes the biological information when the change in the output value of the biological sensor 21 is large (for example, when the change amount is 10% or more of the original value). It may be determined that In addition, the processing / control unit 40 determines whether the change in the output value of the gesture detection unit (13, 23, 33) continues for 5 seconds or more based on the timing result of the timer 20, or the gesture detection unit (13, 23, 33). If the output value of 33) changes intermittently within 30 seconds, it is determined that the passenger has made a gesture.

  Also, when the train stops suddenly, when shaking is large, or when passengers get on and off, the detection result of the gesture detection unit (13, 23, 33) is the same detection result as when a trouble occurs. There is. In such a case, in order not to determine that a trouble has occurred, the processing / control unit 40 may determine whether or not a trouble has occurred in consideration of the detection result of the vehicle sensor 11. .

  If the determination in step S14 is affirmed by the above determination, the process proceeds to step S22. If the determination is negative, the process proceeds to step S16.

  When the process proceeds to step S16, the processing / control unit 40 determines whether it is not possible to clearly determine whether or not a trouble has occurred (that is, whether confirmation is necessary). Specifically, the processing / control unit 40 makes a determination based on whether one of the following conditions (A) and (B) is satisfied.

(A) While the biosensor 21 detects a change in biometric information, it cannot be confirmed from the detection result of the gesture detection unit (13, 23, 33) that a gesture is being performed.
(B) In the case where it can be confirmed from the detection result of the pressure sensor 23 that the gesture is performed, the biosensor 21 has not detected a change in the biometric information, and the detection result of the piezoelectric sensor 13 and the face recognition unit 33 You can confirm that a gesture is being made from at least one of the recognition results.

  If either of the above conditions (A) and (B) is satisfied, the determination in step S16 is affirmed and the process proceeds to step S18. On the other hand, if neither of the above conditions (A) and (B) is satisfied, it is determined that there is almost no possibility that a trouble has occurred, the determination in step S16 is denied, and the process returns to step S10.

  When the process proceeds to step S18, the processing / control unit 40 confirms the passenger specified in step S14. Specifically, the processing / control unit 40 drives the speaker 15 and the microphone 16 by the driving device 9 and asks the identified passenger using a sound such as “Are you okay?” Using the speaker 15. . Further, the processing / control unit 40 turns on the switch of the microphone 16 at the timing of the inquiry, and obtains a response voice from the passenger to the inquiry. Then, the processing / control unit 40 transmits the response voice to the voice recognition unit 34 and acquires the voice recognition result by the voice recognition unit 34.

  Next, in step S20, the processing / control unit 40 causes trouble based on the recognition result of the speech recognition unit 34 and the outputs of the biosensor 21 and the gesture detection unit (13, 23, 33) between step S16 and step S18. It is determined whether or not this has occurred. For example, the processing / control unit 40 determines that there is no trouble when the recognition result of the voice recognition unit 34 is “OK”, and when it is “help” or “car”, etc. It is determined that a problem has occurred. Further, for example, when there is no response voice from a passenger and a gesture is made by the passenger, it is determined that a trouble has occurred. The processing / control unit 40 may determine whether or not trouble has occurred in consideration of the magnitude (dB) of the collected response voice.

  If the determination in step S20 is negative (when no trouble has occurred), the process returns to step S10. If the determination is positive (if a trouble has occurred), the process proceeds to step S22.

  When the determination in step S20 or the determination in step S14 described above is affirmed and the process proceeds to step S22, the processing / control unit 40 executes trouble suppression processing.

  Specifically, in order to suppress trouble, the processing / control unit 40 controls the driving device 9 to identify the imaging unit 14, the speaker 15, the microphone 16, and the LED 18 (passengers caught in trouble). And around it. Then, the processing / control unit 40 will ask the speaker 15 “How did you do it”, “Is it all right”, etc., or “Some troubles may have occurred from the speaker 15, so record the situation. After the announcement “etc.”, the image picked up by the image pickup unit 14 and the sound collected by the microphone 16 are recorded in the flash memory 30. Moreover, the processing apparatus 19 makes LED18 light-emit, and irradiates light toward the vicinity where the identified passenger exists. If processing / control part 40 performs Step S22, when the molester act is performed in the car, the person who performs the molester act is likely to hesitate to continue the act. Thereby, it becomes possible to suppress generation | occurrence | production of the subsequent molester act. The processing / control unit 40 may perform at least one of the announcement, imaging, recording, and irradiation. For example, the light emission of the LED 18 may be performed only when it can be determined that the boarding rate is high from the imaging result of the imaging unit 14, the detection result of the piezoelectric sensor, or at night.

  Next, in step S24, the processing / control unit 40 checks whether or not the trouble has subsided. In this case, the processing / control unit 40 makes an inquiry using the speaker 15 as in step S18 described above, and the trouble is calmed based on the recognition result of the response speech acquired from the microphone 16 by the speech recognition unit 34. Determine whether or not. Note that the processing / control unit 40 may determine whether or not the trouble has subsided based on the detection results of the biological sensor 21 and the gesture detection unit (13, 23, 33). In this case, if the detection results of the biosensor 21 and the gesture detection unit (13, 23, 33) return to normal values, it can be determined that the trouble has calmed down.

  If the determination in step S24 is affirmative, the process returns to step S10. On the other hand, if the determination in step S24 is negative, the process proceeds to step S26. In step S26, the processing / control unit 40 notifies the station staff at the next stop station of the vehicle in which the trouble has occurred. In this case, the processing / control unit 40 notifies the station staff at the next stop station using a communication function (telephone function, mail function, etc.) realized by the CPU of FIG. 4 or a communication device connected to the processing device. can do.

  By repeatedly executing the above processing during train operation, it is possible to detect occurrence of trouble in the train and to calm down the trouble.

  As described above in detail, according to the present embodiment, the processing / control unit 40 detects the change in the biological information of the passenger in the train input from the biological information input unit 31 and the detection result of the biological sensor 21. Based on the recognition result of the gesture detection unit (13, 23, 33) for recognizing the passenger motion input from the motion information input unit 32, processing corresponding to the passenger motion (gesture) is performed. In other words, the processing / control unit 40 considers the detection result of the biosensor 21 in addition to the operation recognized by the gesture detection unit (13, 23, 33) (determination of occurrence of trouble or processing for calming the trouble). By performing this, it is possible to perform appropriate processing according to the passenger's movement.

  Further, according to the present embodiment, the gesture detection unit (13, 23, 33) has a plurality of different sensors, and the processing / control unit 40 inputs biological information change to the biological information input unit 31. Even if there is not, the process according to the motion of the passenger is performed based on the recognition results of the plurality of sensors input by the motion information input unit 32. Therefore, by performing processing based on the recognition results of motions (gestures) of a plurality of sensors, it is possible to more appropriately perform processing according to passenger motion.

  In addition, according to the present embodiment, the gesture detection unit includes the face recognition unit 33 that is a non-contact sensor, and contact sensors such as the piezoelectric sensor 13 and the pressure sensor 23. The processing / control unit 40 includes: When the contact sensor recognizes the movement of the passenger, the imaging unit 14 performs imaging. Thereby, the imaging unit 14 can turn off the power until the contact-type sensor recognizes the movement of the passenger, so that energy saving can be achieved.

  Moreover, according to this embodiment, since the imaging part 14 is provided above (ceiling part) rather than the contact-type sensor, the imaging part 14 can mainly image a passenger's head. Thereby, it becomes possible to protect a passenger's privacy.

  In addition, according to the present embodiment, since at least a part of the gesture detection unit (the pressure sensor 23 in the present embodiment) is provided in the vicinity of the biological sensor 21, the change in the biological information of the hand that performed the gesture itself can be detected. It can be detected by the biosensor 21. In this case, since the relationship between the gesture and the change in the biological information is high, it is possible to more appropriately perform the process according to the passenger's motion.

  Moreover, according to this embodiment, since the process / control part 40 performs the process which outputs a sound from the speaker 15 toward a passenger as a process according to a passenger | crew's operation | movement, attention to a passenger's inquiry and a passenger's action And so on. As a result, it is possible to appropriately determine whether or not trouble has occurred and to prevent trouble.

  In the present embodiment, the speaker 15 can be a directional speaker that outputs sound in a limited direction. In this case, it is possible to limit the question and attention to the passengers to a specific passenger (passenger who performed the gesture) and passengers around it.

  In addition, according to the present embodiment, the processing / control unit 40 receives the voice uttered by the passenger from the microphone 16 that inputs the voice and causes the voice recognition unit 34 to identify the voice. It is possible to perform appropriate processing (confirmation of trouble occurrence) based on the meaning of the voice.

  Moreover, in this embodiment, the timer 20 which measures the time when the passenger's biometric information is changing and the time when the passenger's gesture is recognized is provided, and the processing / control unit 40 is configured to input the biometric information. The process according to the input result of the part 31 and the operation information input part 32 and the timing result of the timer 20 is executed. For example, the processing / control unit 40 determines that a gesture is performed when a change in the input result is detected continuously for 5 seconds or more, or when the input result changes intermittently within a predetermined time (for example, 30 seconds). Judgment is made and processing is executed. Thereby, it is possible to appropriately determine that the gesture has been performed, and thus it is possible to appropriately execute the process.

  In the present embodiment, the passenger is present in a movable device called a train, and the processing / control unit 40 performs processing in consideration of the detection result of the vehicle sensor 11 that detects the movement of the train. I am going to do that. In other words, by taking into account the movement of the train that occurs during sudden acceleration / deceleration, stoppage, passenger up / down, etc., it is possible to appropriately determine the presence / absence of the passenger's gesture even if there is a passenger movement due to the movement of the train. As a result, appropriate processing can be performed.

  In the present embodiment, the motion information input unit 32 detects the detection result of the pressure sensor 23 that detects the movement of the passenger's hand, the detection result of the piezoelectric sensor 13 that detects the movement of the foot, and the movement of the head. The recognition result of the face recognition unit 33 is input, and the processing / control unit 40 determines whether the hand, foot, and head are part of the same person. It becomes possible to associate detection results of hand, foot, and head movements. In the present embodiment, the processing / control unit 40 performs processing based on the detection results of the movements of the hands, feet, and head of the same person, and therefore performs appropriate processing based on the movements of the hands, feet, and head. It can be carried out.

  In this embodiment, the processing / control unit 40 determines that the hand and the foot are the same based on the position information of the pressure sensor 23 that detects the movement of the hand and the position information of the piezoelectric sensor 13 that detects the movement of the foot. Since it is determined whether or not it belongs to a person, appropriate determination based on position information can be performed.

  Further, in the present embodiment, in addition to the motion information input unit 32, the biometric information input unit 31 that inputs the detection result of the biosensor 21 that detects a change in the biometric information of the passenger is provided. Appropriate processing based on the change can be performed.

  In the present embodiment, the processing / control unit 40 performs processing based on head movement and movements other than the head (hands and feet) input from the face recognition unit 33. It is possible to perform appropriate processing as compared with the case where processing based on either movement or movement other than the head is performed.

  In the above embodiment, the case where the processing device 19 has the functions of the voice recognition unit 34 and the face recognition unit 33 has been described. However, the present invention is not limited to this, and a device (such as a CPU) having a function equivalent to the speech recognition unit 34 or a function equivalent to the face recognition unit 33 may be provided outside the processing device 19. Good.

  In the above embodiment, an acceleration sensor worn by a passenger may be adopted as a part of the gesture detection unit instead of or together with the piezoelectric sensor 13. The acceleration sensor is built in or attached to shoes, for example. Information on the acceleration sensor is registered in the processing device 19 or the flash memory 30. When the passenger makes a gesture of stepping on the floor in the vehicle, the detection result of the acceleration sensor is input to the processing device 19 (operation information input unit 32) by wireless communication or the like. By doing in this way, it becomes possible to perform the same process as the case where the piezoelectric sensor 13 is utilized. Further, if acceleration sensors are provided at the toe portion and the heel portion, respectively, it becomes possible to detect a gesture that the passenger steps on the floor, regardless of the posture in the vehicle. When the acceleration sensor is used in this way, it is possible to detect the occurrence of trouble more accurately.

  In addition, although the said embodiment demonstrated the case where the biosensor 21 was provided in the strap 22, it is not restricted to this. For example, the biometric sensor 21 may be provided on a watch-type accessory or a ring-type accessory that a passenger wears. In this case, the accessory is provided with a wireless unit, and the biological information detected by the biological sensor 21 is wirelessly transmitted using the wireless unit. The biosensor 21 is registered in the processing device 19 in advance. In this case, if the passenger wears the biometric sensor 21, for example, when the passenger is perverted and the heart rate is increased (that is, when the biometric information is changed), the wireless device is used for the processing device 19. It is possible to notify the possibility of the occurrence of a molester act. Further, in this case, since the passenger receiving the molester action does not have to hold the strap, the occurrence of the trouble can be detected more accurately.

  In addition, although the said embodiment demonstrated the case where the processing apparatus 19 was provided in the vehicle, it is not restricted to this. For example, the processing device 19 may be provided outside the vehicle (for example, a station or a railway control facility). In this case, it is necessary that each unit other than the processing device 19 shown in FIG. 1 and the processing device 19 can communicate with each other.

  In the above embodiment, the case where the living body sensor 21 and the pressure sensor 23 are provided on the strap 22 in the vehicle has been described. However, the present invention is not limited to this, and the living body sensor 21 and the pressure sensor are attached to a bar-shaped handrail provided in the vehicle. 23 may be provided.

  The trouble handling system 100 is not limited to being installed in a train, but can be installed in a mobile device that can be boarded by a person such as a bus or an elevator. It can also be installed in banks, commercial facilities, homes, etc.

  The above-described embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

DESCRIPTION OF SYMBOLS 11 Vehicle sensor 13 Piezoelectric sensor 15 Speaker 16 Microphone 19 Processing apparatus 20 Timer 21 Biosensor 23 Pressure sensor 31 Biometric information input part 32 Operation information input part 33 Face recognition part 34 Voice recognition part 40 Processing / control part 50 Train 90 CPU
100 Trouble response system

Claims (13)

A first input unit for inputting a detection result of a biological sensor that detects a change in biological information of the subject;
A second input unit for inputting a recognition result of a recognition device for recognizing the movement of the subject;
An electronic apparatus comprising: a processing unit that performs processing according to the operation of the subject based on input results of the first and second input units. The recognition device has a plurality of different sensors,
The processing unit responds to the operation of the subject based on the recognition results of the plurality of sensors input by the second input unit even when there is no input of changes in biological information to the first input unit. The electronic device according to claim 1, wherein processing is performed. The plurality of sensors includes an imaging device and a contact sensor,
3. The electronic apparatus according to claim 1, further comprising a control unit configured to perform imaging by the imaging device when the contact-type sensor recognizes a motion of the subject.   The electronic apparatus according to claim 3, wherein the imaging device is provided above the contact sensor.   The electronic apparatus according to claim 1, wherein at least a part of the recognition device is provided in the vicinity of the biometric sensor.   The electronic device according to claim 1, wherein the processing unit performs a process of outputting sound with a speaker that outputs sound toward the target person.   The electronic apparatus according to claim 6, wherein the speaker is a directional speaker that outputs the sound in a limited direction.   The electronic device according to any one of claims 1 to 7, wherein the processing unit receives a voice uttered by the subject from a voice input device that inputs voice.   The electronic apparatus according to claim 8, further comprising a voice identification unit that identifies voice received from the voice input device. A time measuring unit for measuring the time when the biological information of the subject is changing and the time when the operation of the subject is recognized;
The electronic device according to any one of claims 1 to 9, wherein the processing unit executes processing according to the input result and a timing result of the timing unit. If the subject is present in a movable mobile device,
The electronic device according to claim 1, wherein the processing unit performs the processing in consideration of a detection result of a detection device that detects a movement of the mobile device. A biological sensor for detecting a change in biological information of the subject,
A recognition device for recognizing the movement of the subject;
A processing system comprising: the electronic device according to claim 1. A first input step of inputting a detection result of a biological sensor that detects a change in biological information of the subject;
A second input step of inputting a recognition result of a recognition device for recognizing the movement of the subject;
A processing program for causing a computer to execute a processing step of performing processing according to the operation of the subject based on input results of the first and second input steps.

Images