JPWO2021106112A1 - Vehicle condition monitoring device and condition monitoring method - Google Patents

Abstract

Provided is a vehicle condition monitoring device that detects some abnormality that has occurred in a situation where a human is present in the vehicle, and further estimates whether the abnormality is a vehicle problem or a human problem. Therefore, the vehicle condition monitoring device is a human condition quantity using a voice signal analysis processing unit that analyzes voice using the voice signal of a microphone installed in the vehicle and a video signal of a camera installed in the vehicle. Incident estimation is performed using the state amount extraction processing unit for extracting the image, the human state amount, the analysis result of the voice signal, and the operation information of the equipment constituting the vehicle, and the incident estimation is generated. It includes a processing unit and an incident information transmission processing unit that transmits the estimated incident information to the outside of the state monitoring device.

Description

The present invention relates to a vehicle condition monitoring device and a condition monitoring method.

Vehicles such as railroad vehicles are required to safely move people and things inside the vehicle to their destinations. For this reason, in a vehicle, an abnormality that hinders movement itself (for example, a failure of equipment constituting the vehicle (for example, a door)) or an abnormality that occurs over time due to wear of a mechanical system (for example, wheel wear of a railroad vehicle). It is necessary to prevent the occurrence of vibration phenomenon) as much as possible. Therefore, when an abnormality occurs, it is common to immediately detect the abnormality and take countermeasures, or to carry out regular maintenance work so that some abnormality does not occur.

For example, in Patent Document 1, regarding the "elevator car abnormality confirmation system", a surveillance camera for photographing the inside of the vehicle, a storage unit for storing the captured image of the surveillance camera, and a display unit for displaying the captured image are provided. It is equipped with a control unit that displays the captured image of the monitoring camera with a predetermined time width when the abnormality detection unit detects an abnormality on the display unit during maintenance, and the predetermined time width is longer than the time when the abnormality detection unit detects the abnormality. It is set to include the captured image of the surveillance camera at the time retroactively, and the storage unit newly captures the image data of the captured image after a predetermined period has passed from the image data of the stored captured image. It is described that the image data of the captured image to be captured is overwritten and stored, and the control unit prohibits overwriting of the captured image of the monitoring camera having a predetermined time width when an abnormality is detected.

Japanese Unexamined Patent Publication No. 2016-141522

The abnormality confirmation system of Patent Document 1 detects an abnormality in the car of an elevator, which is a vehicle, and confirms the abnormality after the fact, but does not consider the case where a human being itself causes an abnormality. .. Further, the abnormality confirmation system of Patent Document 1 has a problem that it is difficult to distinguish whether the problem is a vehicle problem or a human problem with respect to the generated abnormality.

Therefore, the present invention detects a certain abnormality that has occurred in a situation where a human is present in the vehicle, and further, monitors the state of the vehicle to estimate whether the abnormality is a problem of the vehicle or a human problem. The purpose is to provide a system.

In order to solve the above problems, one of the typical vehicle state monitoring devices of the present invention is a voice signal analysis processing unit that analyzes voice using the voice signal of a microphone installed in the vehicle, and the vehicle. The state amount extraction processing unit that extracts the human state amount using the video signal of the camera installed in the vehicle, the human state amount, the analysis result of the voice signal, and the operation information of the equipment constituting the vehicle. It is provided with an incident estimation processing unit that estimates an incident using the above and generates estimated incident information, and an incident information transmission processing unit that transmits the estimated incident information to the outside of the state monitoring device.

According to the present invention, in a situation where a human is present in a vehicle, some abnormality that has occurred is detected, and further, whether the abnormality is a vehicle problem or a human problem is estimated, and the vehicle condition monitoring is performed. It becomes possible to provide the device.

Issues, configurations and effects other than those described above will be clarified by the following description of the embodiments.

It is a figure which shows the structure of the condition monitoring system of a railroad vehicle. It is a flow chart explaining the operation of the state monitoring apparatus. It is a figure of an example which extracts a human state quantity. It is a flow chart explaining the operation of the state monitoring apparatus. It is a flow chart explaining the operation of the state monitoring apparatus. It is a flow chart explaining the operation of the state monitoring apparatus following FIG.

In the present disclosure, a vehicle that transports people or goods is referred to as a "vehicle". "Vehicles" include, for example, railroad vehicles, aircraft, and elevators.
Hereinafter, examples will be described with reference to the drawings. In each figure, the same numbers are assigned to the components having a common function, and the duplicated description thereof will be omitted.

The first embodiment is a condition monitoring device and a condition monitoring method for detecting an incident generated in a railway vehicle and estimating the content thereof by using the information of a microphone and a camera mounted on the railway vehicle.

<Structure of Example 1>
FIG. 1 is a diagram showing a configuration of a condition monitoring system according to the first embodiment.

In the figure, the railroad vehicle 1 is composed of a vehicle body 2 for transporting people 8 and goods and a bogie 4 traveling on a track 3, and a microphone 5 for monitoring the state and a surveillance camera 6 are mounted inside the vehicle body 2. Will be done. The microphone 5 and the surveillance camera 6 may be installed on the ceiling or the side surface of the vehicle body 2, and a plurality of microphones 5 may be installed. Further, the microphone and the surveillance camera may be integrated. Further, the railway vehicle 1 is equipped with a vehicle information control device 7 that manages vehicle operation information such as a traveling position, a traveling speed, and a occupancy rate. Further, the railroad vehicle 1 includes a voice signal detection processing unit 11, a video signal detection processing unit 13, a voice signal analysis processing unit 12, a state quantity extraction processing unit 14, an incident estimation processing unit 15, an incident information transmission processing unit 16, and vehicle information. It is equipped with a status monitoring device 10 including a detection processing unit 17 and an incident information database 18.

The state monitoring device 10 inputs an audio signal 31a from the microphone 5 and a video signal 34a from the monitoring camera 6, inputs a vehicle information signal 37a from the vehicle information control device 7, and outputs estimated incident information 40a to the vehicle information control device 7. Output.

The audio signal detection processing unit 11 inputs the audio signal 31a obtained by the microphone 5, converts it into an audio digital signal 32a for analysis processing, and outputs it to the audio signal analysis processing unit 12.

The video signal detection processing unit 13 inputs the video signal 34a obtained by the surveillance camera 6, converts it into a video digital signal 35a for extracting a human state quantity, and outputs it to the state quantity extraction processing unit 14.

The audio signal analysis processing unit 12 inputs an audio digital signal 32a, and performs incident estimation processing of an audio signal analysis result 33a such as a sound generated by a human, for example, a scream, or a sound generated by a vehicle, for example, an operating sound of an air conditioner. Output to unit 15.

The state quantity extraction processing unit 14 inputs the video digital signal 35a and outputs the state quantity 36a such as human movement to the incident estimation processing unit 15.

The vehicle information detection processing unit 17 inputs a vehicle information signal 37a and outputs vehicle information 38a such as operation data of doors and underfloor equipment to the incident estimation processing unit 15.

The incident estimation processing unit 15 inputs the voice signal analysis result 33a, the state quantity 36a, the vehicle information 38a, and the incident information 42a, and outputs the estimated incident information 39a and 41a to the incident information transmission processing unit 16 and the incident information database 18, respectively. ..

The incident information database 18 inputs estimated incident information 41a, outputs incident information 42a to incident estimation processing unit 15, and outputs incident information 43a to incident information transmission processing unit 16.

The incident information transmission processing unit 16 inputs the estimated incident information 39a and the incident information 43a, and outputs the estimated incident information 40a to the vehicle information control device 7.

The details of the above processing contents will be described separately below.

<Detection flow of Example 1 (incident caused by human)>
FIG. 2 is a flow chart for explaining the operation of the condition monitoring device, and is a flow for detecting an incident generated by a human in particular. It should be noted that FIG. 2 shows the flow after grasping that there are humans in the vehicle to be the condition monitoring target. Regarding this grasp, for example, a surveillance camera installed in the vehicle may be used to calculate the number of people entering and exiting the vehicle, and the number of people in the vehicle may be grasped. The operation of the condition monitoring device 10 will be described with reference to the step numbers in the figure.

Step S101: Extract a human voice from the voice signal of the microphone. For example, focusing on the frequency band in which the human voice is prominent, the human voice is extracted by the difference processing with the audio signal when the human voice is absent. This process is performed by the audio signal analysis processing unit 12.

Step S102: It is determined whether or not the voice extracted in step S101 exceeds the voice threshold set in advance. In this determination, for example, not only the voice level but also how long the voice exceeding the threshold value continues. This processing is performed by the audio signal analysis processing unit 12 or the incident estimation processing unit 15.

Step S103: When the threshold value is exceeded, the state quantity of each person in the vehicle is extracted from the video signal. For example, FIG. 3 shows an example of extracting the state quantity of each human in a vehicle. In FIG. 3, each human 8 is recognized as a moving body 50 from the video signal of the surveillance camera 6 installed in the vehicle. By calculating the moving speed (scalar value) from the amount of movement of each moving body 50 in the vehicle, the average speed of the entire moving body 50 in the vehicle is calculated. At this time, if the railroad vehicle is running and the passengers are seated, the average speed of the moving body 50 is almost equal to zero even if some passengers move in the aisle. Further, when passengers move when departing from or arriving at a station, the average speed of the moving body 50 falls within a certain speed range. On the other hand, it is presumed that the average speed of the moving body 50 will be high under the situation where the human 8 escapes due to an abnormality in the vehicle. In addition, the occupancy rate may be taken into consideration as the amount of human condition. Furthermore, when calculating the moving speed of each moving body 50, the moving body 50 moving in each direction in each direction is individually separated to calculate the average speed in consideration of the moving vector, and the human state quantity. It may be extracted as. Further, the density of the moving body may be extracted as a human state quantity from the position information of each moving body from a bird's-eye view from the upper surface of the vehicle body. This process is carried out by the state quantity extraction process unit 14.

Step S104: To grasp the relationship seen between the human voice exceeding the threshold value and the human state quantity. For example, grasping where the combination of the characteristic value of the human voice obtained in step 101 and the human state quantity obtained in step 103 is located in the space about each of them. back. Further, not only the human voice and the average moving speed of the human but also the position in the space in consideration of the occupancy rate may be grasped, and the state quantity obtained from the human may be considered. This process is performed by the incident estimation processing unit 15.

Step S105: If the presence or absence of an incident can be determined based on the relationship grasped in step 104, which incident corresponds to is selected. For example, an incident level corresponding to the relationship between the human voice and the human state quantity in step 104 may be set in advance as a numerical value, and which incident level corresponds to may be selected. If the incident that has occurred can be estimated from the relationship between the human voice and the human state quantity in step 104 based on the past incident database, not only the incident level but also the estimated incident can be selected. good. For example, humans are rampaging. This process is performed using the incident estimation processing unit 15 and the incident information database 18.

Step S106: Incident information is transmitted based on the result of step S105. For example, information is transmitted to the vehicle information control device, and the driver or crew in the vehicle or the management center on the ground side is contacted. At this time, the audio signal and the video signal at that time are also presented according to the incident level. Further, at that time, the estimated incident information may be shown. Regarding these incident information, the driver is displayed on the screen of the driver's cab, the crew is presented with the information on a smart device such as a tablet, and the ground management center is in the management center. All you have to do is display the information on the screen. This process is performed by the incident information transmission processing unit 16.

<Detection flow of Example 1 (incident generated on the vehicle side)>
FIG. 4 is a flow chart for explaining the operation of the condition monitoring device, and is a flow for detecting an incident on the vehicle side, particularly when a person is absent in the vehicle. It should be noted that FIG. 4 shows the flow after grasping that a human is absent from the vehicle to be the condition monitoring target. Regarding this grasp, in the same manner as described above, for example, a surveillance camera installed in the vehicle may be used to calculate the number of people entering and exiting the vehicle, and the number of people in the vehicle may be grasped. .. The operation of the condition monitoring device 10 will be described with reference to the step numbers in the figure.

Step S111: Analyzing the voice signal of the microphone. For example, a 1/3 octave analysis may be performed to detect the noise level in each frequency band. This process is performed by the audio signal analysis processing unit 12.

Step S112: It may be determined whether or not the sound extracted in step S111 exceeds the preset threshold value. In this determination, for example, not only the sound level but also how long the sound exceeding the threshold value continues. Further, the noise value specified in the specifications may be set as the threshold value. This processing is performed by the audio signal analysis processing unit 12 or the incident estimation processing unit 15.

Step S113: When the threshold value is exceeded, the relationship between the generated sound and the device database is collated. For example, based on the 1/3 octave analysis result, equipment such as air conditioners and operating frequencies may be registered in advance as a database. If the operating frequency of the registered equipment is applicable in the frequency band of the noise level that exceeds the threshold value, the corresponding result can be output. This process is performed by the incident estimation processing unit 15, and the incident information database 18 is used.

Step S114: In step S113, when the registration data of the database related to the threshold value exceeding sound exists, the relationship between the sound and the operation information of the device is analyzed. For example, the operation information of the device in the registration data is received from the vehicle information control device. At this time, it is detected how much the relationship between the device operation information obtained from the past data and the generated sound is different from the relationship obtained this time. This process is performed by the incident estimation processing unit 15 and the vehicle information detection processing unit 17, and the incident information database 18 is used.

Step S115: In step 114, when the operation information of the device and the threshold value exceeding sound are related, the relationship between the sound and the operation information of the device is selected. For example, when the difference between the relationship obtained from the past data and the relationship obtained this time is large, the device is selected as the factor of the sound. This process is performed by the incident estimation processing unit 15 and the vehicle information detection processing unit 17, and the incident information database 18 is used.

Step S116: An incident that occurred on the vehicle side is selected based on the results of step 113, step 114, and step 115. For example, if there is no information in the device database related to the threshold excess sound, it is selected that the incident does not exist in the database in the first place. In addition, if there is information in the device database related to the threshold overtone but no relevance is found, the incident indicates the device in the target database and no relevance is found. To select. In addition, if there is information in the device database related to the threshold overtone and there is a relevance, the target device is selected as an incident. This process is performed using the incident estimation processing unit 15 and the incident information database 18.

<Detection flow of Example 1 (incident that occurs on the vehicle side when there is a human)>
5 and 6 are flow charts for explaining the operation of the condition monitoring device, and in particular, are a flow of detecting an incident on the vehicle side by using a sound generated by a human or a human movement as support information. It should be noted that FIGS. 5 and 6 show the flow after grasping the existence of a human being for the vehicle to be the condition monitoring target. Of the step numbers in the figure, the duplicate step numbers are omitted, and the operation of the condition monitoring device 10 will be described.

Step S121: Extract a sound other than the human voice from the voice signal of the microphone. For example, it may be extracted as a sound other than the human voice by the difference processing with the human voice. This process is performed by the audio signal analysis processing unit 12.

Step S122: When the existence or association of data is not found in step S113 and step S114, the state quantity of each person in the vehicle is extracted. As an example of extracting the state quantity of each human in the vehicle, the individual human 8 is recognized from the video signal of the surveillance camera 6 installed in the vehicle. At this time, pay attention to the human head part and extract where the human face is facing. In this way, it is possible to extract which part the plurality of humans are paying attention to from the direction in which they are facing their faces. This process is carried out by the state quantity extraction process unit 14.

Step S123: Based on the results up to step 122, an incident on the vehicle side based on the characteristics is selected. For example, if there is no information in the device database related to the threshold excess sound, it is selected that the incident does not exist in the database in the first place. In addition, if there is information in the device database related to the threshold overtone but no relevance is found, the incident indicates the device in the target database and no relevance is found. To select. Further, from the characteristics obtained in step S122, where the human was paying attention is also selected. This process is performed using the incident estimation processing unit 15 and the incident information database 18.

Step S124: A vehicle incident is selected based on the results of step 113, step 114, and step 115. For example, if there is information in the device database related to the threshold overtone and the relevance is found, the target device is selected as an incident. This process is performed using the incident estimation processing unit 15 and the incident information database 18.

<Effect of Example 1>
(1) As described above, in the first embodiment, when a person is absent in the vehicle, the cause of the problematic sound can be estimated by collating the generated sound on the vehicle side with the device database, and the crew member or the like can estimate the cause. It will be possible to immediately present information such as factor candidates, generated sounds, and video information inside the vehicle to the parties concerned.

(2) Further, in Example 1, when a human is present in the vehicle, it becomes possible to detect an incident generated by the human by utilizing the relationship between the human voice and the human state quantity. In addition, it becomes possible to immediately present information such as the content of the incident, the generated sound, and the video information in the vehicle to the related persons such as the crew.

(3) Further, in the first embodiment, when a human is present in the vehicle, in addition to collating the sound other than the human voice with the device database, the vehicle side which becomes a problem from the human voice and the human state quantity. It is possible to estimate the factors of the above, and to immediately present information such as factor candidates, generated sounds, and video information in the vehicle to related persons such as crew members.

<Supplementary matters of the embodiment>
In the above-described embodiment, the use for condition monitoring of a railway vehicle has been described, but the use of the present invention is not limited to this. For example, it can be applied to aircraft condition monitoring, elevator condition monitoring and other uses.

The present invention is not limited to the above-described embodiment, and includes various modifications. For example, artificial intelligence may perform part or all of the processing performed by the condition monitoring device 10. The above-mentioned examples have been described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the described configurations.

Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration. For example, by installing an odor sensor or a vibration acceleration detection sensor inside the vehicle body, the factor may be estimated in consideration of the odor or vibration inside the vehicle body. Further, by installing a microphone, an odor sensor, or a vibration acceleration detection sensor in the vicinity of the trolley, it may be possible to detect the sound, odor, or vibration generated by the device installed under the floor.

1 ... railroad car, 2 ... car body (half car body), 3 ... track, 4 ... trolley, 5 ... microphone, 6 ... surveillance camera, 7 ... vehicle information control device, 8 ... human, 10 ... state monitoring device, 11 ... Audio signal detection processing unit, 12 ... Audio signal analysis processing unit, 13 ... Video signal detection processing unit, 14 ... State quantity extraction processing unit, 15 ... Incident estimation processing unit, 16 ... Incident information transmission processing unit, 17 ... Vehicle information detection Processing unit, 18 ... Incident information database

Claims (7)

A voice signal analysis processing unit that analyzes voice using the voice signal of a microphone installed in a vehicle, and
A state quantity extraction processing unit that extracts a human state quantity using the video signal of a camera installed on the vehicle, and a state quantity extraction processing unit.
An incident estimation processing unit that estimates an incident using the human state quantity, the analysis result of the voice signal, and the operation information of the equipment constituting the vehicle, and generates the estimated incident information.
An incident information transmission processing unit that transmits the estimated incident information to the outside of the condition monitoring device, and
Vehicle condition monitoring device. In the vehicle condition monitoring device according to claim 1,
A vehicle condition monitoring device that separates the sound generated by the human and the sound generated by the vehicle from the audio signal, and determines that an incident has occurred when each sound exceeds the respective threshold value. In the vehicle condition monitoring device according to claim 2,
The human is detected from the video signal, and the individual movement speed of the human, the average movement speed of the entire human, or the location and density of the human are obtained from the movement amount of the human, and the state of the human is obtained. A vehicle condition monitoring device that detects as a quantity. In the vehicle condition monitoring device according to claim 3,
A vehicle condition monitoring device that estimates an incident corresponding to a combination of the human state quantity and the characteristic value of the sound generated by the human, and generates the estimated incident information. In the vehicle condition monitoring device according to claim 2,
A vehicle condition monitoring device that estimates an incident from the analysis result of the sound generated by the vehicle and the operation information of the equipment constituting the vehicle, and generates the estimated incident information. In the vehicle condition monitoring device according to claim 4 or 5.
A vehicle condition monitoring device that transmits the estimated incident information, the audio signal used for estimation, and the video signal to the outside of the condition monitoring device. A voice signal analysis processing step that analyzes voice using the voice signal of a microphone installed in a vehicle, and
A state quantity extraction processing step for extracting a human state quantity using the video signal of a camera installed on the vehicle, and a state quantity extraction processing step.
An incident estimation processing step that estimates an incident using the human state quantity, the analysis result of the voice signal, and the operation information of the equipment constituting the vehicle, and generates the estimated incident information.
The incident information transmission processing step for transmitting the estimated incident information, and
Vehicle condition monitoring method.

Images