JP2020140401A - Abnormality monitoring system, and, abnormality monitoring method - Google Patents

Abstract

To improve accuracy in monitoring abnormality of security.SOLUTION: An abnormality monitoring system comprises: a sensor data acquisition unit 48 which acquires sensor data from a sensor which detects abnormality in security of a motorcycle; a periphery information acquisition unit 42 which acquires periphery information indicating a state around the motorcycle; a learning unit 23 which performs learning by artificial intelligence based on the sensor data acquired by the sensor data acquisition unit 48 and the periphery information acquired by the periphery information acquisition unit 42 and generates a learnt model for determining the abnormality in the security and erroneous detection; and a determination unit 24 which uses the learnt model learnt by the learning unit 23 to determine the abnormality in the security and the erroneous detection based on the sensor data acquired by the sensor data acquisition unit 48 and the periphery information acquired by the periphery information acquisition unit 42.SELECTED DRAWING: Figure 1

Description

The present invention relates to an abnormality monitoring system and an abnormality monitoring method.

In order to prevent theft and mischief of moving objects such as vehicles, there is known a technique for monitoring security abnormalities by detecting vibration generated in the moving objects. Further, there is known a technique of operating a necessary anti-theft device such as an alarm signal based on a change in air pressure generated in the vehicle when the door or trunk of the vehicle is opened (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2004-078365

In the conventional technology, when the sensor detects, for example, rain, wind, lightning, contact with neighbors, noise, etc., a security abnormality may be erroneously detected and an alarm may be activated carelessly. .. However, if the threshold value of the sensor is set high in order to suppress erroneous detection, there is a risk that a security abnormality cannot be detected correctly. As described above, there is room for improvement in the accuracy of monitoring the presence or absence of security abnormalities.

The present invention has been made in view of the above, and an object of the present invention is to improve the accuracy of monitoring abnormalities in security.

In order to solve the above-mentioned problems and achieve the object, the abnormality monitoring system according to the present invention includes a sensor data acquisition unit that acquires sensor data from a sensor that detects a security abnormality of the moving body, and a peripheral portion of the moving body. Learning by artificial intelligence based on the peripheral information acquisition unit that acquires the peripheral information indicating the state of, the sensor data acquired by the sensor data acquisition unit, and the peripheral information acquired by the peripheral information acquisition unit, A learning unit that generates a trained model for determining a security abnormality and a false detection, and the sensor data acquired by the sensor data acquisition unit using the trained model learned by the learning unit. It is characterized by including a determination unit for determining a security abnormality and an erroneous detection based on the peripheral information acquired by the peripheral information acquisition unit.

The abnormality monitoring method according to the present invention includes a sensor data acquisition step of acquiring sensor data from a sensor that detects a security abnormality of the moving body, and a peripheral information acquisition step of acquiring peripheral information indicating a state around the moving body. , Learned by artificial intelligence based on the sensor data acquired in the sensor data acquisition step and the peripheral information acquired in the peripheral information acquisition step, and learned to determine a security abnormality and false detection. Based on the learning step for generating a model, the sensor data acquired in the sensor data acquisition step using the learned model learned in the learning step, and the peripheral information acquired in the peripheral information acquisition step. It includes a determination step for determining a security abnormality and a false positive.

According to the present invention, there is an effect that the accuracy of monitoring an abnormality in security can be improved.

FIG. 1 is a schematic view showing an example of a configuration example of the abnormality monitoring system according to the first embodiment. FIG. 2 is a schematic view showing a configuration example of the abnormality monitoring device according to the first embodiment. FIG. 3 is a diagram illustrating an example of sensor data and peripheral information acquired from a plurality of abnormality monitoring devices. FIG. 4 is a flowchart showing a flow of learning processing in the server device according to the first embodiment. FIG. 5 is a flowchart showing a flow of disconnection monitoring processing in the abnormality monitoring device according to the first embodiment. FIG. 6 is a flowchart showing a flow of monitoring processing in the abnormality monitoring device according to the first embodiment. FIG. 7 is a flowchart showing a processing flow in the abnormality monitoring device according to the first embodiment. FIG. 8 is a flowchart showing a processing flow in the abnormality monitoring device according to the first embodiment. FIG. 9 is a flowchart showing a processing flow in the abnormality monitoring device according to the first embodiment. FIG. 10 is a flowchart showing a flow of determination processing in the server device according to the first embodiment. FIG. 11 is a schematic view showing a configuration example of an imaging device for abnormality monitoring according to the second embodiment. FIG. 12 is a schematic view showing a configuration example of the abnormality monitoring device according to the second embodiment. FIG. 13 is a schematic view showing a configuration example of the abnormality monitoring device according to the third embodiment. FIG. 14 is a schematic view showing a configuration example of an imaging device for abnormality monitoring according to a fourth embodiment. FIG. 15 is a schematic view showing a configuration example of the abnormality monitoring device according to the fourth embodiment.

Hereinafter, embodiments of the abnormality monitoring system and the abnormality monitoring method according to the present invention will be described in detail with reference to the accompanying drawings. The present invention is not limited to the following embodiments.

[First Embodiment]
FIG. 1 is a schematic view showing an example of a configuration example of the abnormality monitoring system according to the first embodiment. FIG. 2 is a schematic view showing a configuration example of the abnormality monitoring device according to the first embodiment. The anomaly monitoring system 1 uses a learning model generated by machine learning of artificial intelligence (AI) to prevent theft of moving objects such as motorcycles, bicycles, and small vessels, and security such as mischief. Monitor for abnormalities. As an example of machine learning, a multi-layer neural network (hereinafter referred to as "DNN") is trained as a trained model by learning by deep learning of supervised learning using supervised learning or unsupervised learning not using teacher data. To generate. Deep learning is a type of machine learning that repeatedly updates the weights of DNNs so that the accuracy of judgments using DNNs is improved, and derives the weights of DNNs so that the judgment result using DNNs is good. It is a method. Machine learning methods are not limited.

The abnormality monitoring system 1 monitors, for example, the presence or absence of security abnormalities such as theft of moving objects such as motorcycles, bicycles, and small vessels, and mischief. In the present embodiment, the moving body will be described as a motorcycle, but the present invention is not limited to this. The abnormality monitoring system 1 includes a server device 10 and an abnormality monitoring device 30 arranged on each of a plurality of motorcycles.

The server device 10 uses the sensor data acquired from the plurality of abnormality monitoring devices 30 and peripheral information to generate a DNN as a learned model for determining a security abnormality and a false detection. When the server device 10 acquires the sensor data for determination and the peripheral information from the abnormality monitoring device 30, the server device 10 uses the DNN to determine a security abnormality and an erroneous detection. The server device 10 includes a communication unit 11, a storage unit 12, and a control unit 20.

The communication unit 11 is a communication unit. The communication unit 11 enables communication with the abnormality monitoring device 30. In the present embodiment, the communication unit 11 can always communicate with the abnormality monitoring device 30 when the abnormality monitoring system 1 is operating normally.

The storage unit 12 records data obtained by associating sensor data with peripheral information, which is teacher data used in the abnormality monitoring system 1, acquired from a plurality of abnormality monitoring devices 30. The storage unit 12 stores the DNN model generated by the learning unit 23 of the control unit 20.

Peripheral information includes date and time information, daytime information, seasonal information, weather information indicating the weather conditions around the motorcycle, object detection information indicating the detection of an object around the motorcycle, and sound around the motorcycle. It includes at least one of voice information and position information indicating the parking position of the motorcycle. Peripheral information is, for example, information that indicates that an event is held around the motorcycle, which can be obtained from the Internet, information on the dispatch of police vehicles or ambulance vehicles around the motorcycle, or information on the streets around the motorcycle. It may be the surveillance camera video data taken by the surveillance camera.

In the present embodiment, the peripheral information is acquired from the video data of the periphery of the motorcycle, which is captured corresponding to the acquisition time of the sensor data. For example, by performing image processing on video data, it is possible to acquire weather information, object detection information, and position information around the motorcycle. Further, the peripheral information may be acquired from the voice data (voice information) recorded corresponding to the acquisition time of the sensor data. Further, the peripheral information may be acquired from the position information of the current position of the motorcycle, which is acquired corresponding to the acquisition time of the sensor data.

The sensor data and peripheral information, which are teacher data, will be described with reference to FIG. FIG. 3 is a diagram illustrating an example of sensor data and peripheral information acquired from a plurality of abnormality monitoring devices. For example, in association with data A, which is sensor data acquired from user A, the sensor data acquisition time, whether or not it was a false detection, and peripheral information corresponding to the sensor data acquisition time include position information A and video data A. The voice data A is stored. From the position information A and the weather information acquired from the outside by the server device 10, it can be seen that it was raining heavily in the area around the motorcycle parking position. Further, by performing image processing on the video data A, it can be seen that heavy rain has fallen around the motorcycle. Further, if a loud rain sound is recorded in the voice data A, it can be seen that heavy rain has fallen around the motorcycle.

For example, in association with data B, which is sensor data acquired from user B, the sensor data acquisition time, whether or not it was a false detection, and peripheral information corresponding to the sensor data acquisition time include position information B and video data B. The voice data B is stored. By performing image processing on the video data B, it can be seen that an object existed around the motorcycle.

For example, in association with the data C which is the sensor data acquired from the user C, the sensor data acquisition time, whether or not it was a false detection, and the peripheral information corresponding to the sensor data acquisition time include the position information C and the video data C. The voice data C is stored. By performing image processing on the video data C, it can be seen that a person existed around the motorcycle. Further, if the voice of the person is recorded in the voice data C, it can be seen that the person exists around the motorcycle.

For example, in association with the data D which is the sensor data acquired from the user D, the sensor data acquisition time, whether or not it was a false detection, and the position information D and the video data D as peripheral information corresponding to the sensor data acquisition time. The audio data D is stored. From the position information D and the dispatch information of the police vehicle or ambulance vehicle around the motorcycle, it can be seen that the motorcycle was parked near the fire station at the time when the emergency vehicle was dispatched. Further, by performing image processing on the video data D, it can be seen that an emergency vehicle in which the red light is lit is running around the motorcycle. Further, if the siren sound is recorded in the voice data D, it can be seen that the siren sound is sounding around the motorcycle.

The control unit 20 is, for example, an arithmetic processing unit (control device) composed of a CPU (Central Processing Unit) or the like. The control unit 20 loads the stored program into the memory and executes the instructions included in the program. The control unit 20 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control unit 20 and the like. The control unit 20 includes a communication control unit 21, a data acquisition unit 22, a learning unit 23, a determination unit 24, and a memory control unit 25.

The communication control unit 21 controls communication with the plurality of abnormality monitoring devices 30 via the communication unit 11. The communication control unit 21 constantly monitors the communication status with the abnormality monitoring device 30.

The data acquisition unit 22 acquires sensor data and peripheral information from a plurality of abnormality monitoring devices 30. The data acquisition unit 22 outputs the acquired sensor data and peripheral information to the storage control unit 25.

The learning unit 23 learns by artificial intelligence based on the sensor data and peripheral information, and generates a learned model for determining a security abnormality and a false detection. The learning unit 23 learns the weight of DNN as a learned model for determining a security abnormality and a false detection by using sensor data which is teacher data and peripheral information. The structure and weight of the DNN are called the DNN model. The learning in the learning unit 23 may be executed in the same manner as the learning in the known deep learning.

For example, the learning unit 23 inputs the sensor data in the case of “security abnormality” and peripheral information into the DNN and executes deep learning. Then, the weight during learning is updated based on whether a correct determination result is output if it is a "security abnormality" or an erroneous determination result is output if it is a "false positive". In addition, the learning unit 23 inputs the sensor data and peripheral information in the case of "false positive" to the DNN to execute deep learning. Then, the weight during learning is updated based on whether a correct determination result is output if it is "erroneous detection" or an erroneous determination result is output if it is "security abnormality". By repeating such processing, the learned weight is obtained as a result of learning. The learning unit 23 outputs a DNN model including the weight of the DNN to the storage unit 12.

For example, the learning unit 23 is strong in the area around the bicycle parking position of the motorcycle based on the period when the sensor data exceeds the threshold value, the period when heavy rain is recognized from the video data which is the surrounding information, and the position information and the weather information. Learn false positives due to heavy rain when data is accumulated that matches or is close to the period when it is determined that it was raining, or the period when a loud rain sound was recorded in the audio data. It is possible. In addition, it is possible to learn security abnormalities according to weather information so as not to erroneously detect vibrations caused by heavy rain.

For example, when the learning unit 23 accumulates data in which the period when the sensor data becomes equal to or higher than the threshold value and the period when the strong wind is recognized from the video data which is the peripheral information are coincident or close to each other, the cause is the strong wind. It is possible to learn false positives. In addition, it is possible to learn security abnormalities according to weather information so as not to erroneously detect vibrations caused by strong winds.

For example, the learning unit 23 accumulates data in which the period when the sensor data exceeds the threshold value and the period when the object passing near the motorcycle is recognized from the video data which is the peripheral information coincide with or are close to each other. If so, it is possible to learn false positives caused by the passage of objects. In addition, it is possible to learn security abnormalities according to the object detection information so that vibration due to the passage of the object is not erroneously detected.

For example, the learning unit 23 records a period in which the sensor data exceeds the threshold value, a period in which a person passing near the motorcycle is recognized from the video data which is peripheral information, or a voice of the person is recorded in the voice data C. When data that matches or is close to the period in which it was used is accumulated, it is possible to learn false positives caused by the passage of a person. In addition, it is possible to learn security abnormalities according to the object detection information and the voice information indicating the voice of the person so as not to erroneously detect the vibration caused by the passage of the person.

For example, the learning unit 23 has a period in which the sensor data exceeds the threshold value, a period in which the emergency vehicle is determined to be dispatched from the position information D and the dispatch information of the police vehicle or the ambulance vehicle around the motorcycle, and the peripheral information. When the period when an emergency vehicle is detected from the video data, or the data that matches or is close to the period when the siren sound was recorded in the audio data is accumulated, the false detection caused by the siren sound is learned. It is possible. In addition, it is possible to learn the voice information indicating the siren sound, the position information indicating the bicycle parking position, and the security abnormality according to the dispatch information of the emergency vehicle so as not to erroneously detect the vibration due to the siren sound.

For example, the learning unit 23 can learn erroneous detection due to noise when data is accumulated from the surrounding information that the parking position of the motorcycle is along a highway or under an overpass where vibration is large due to noise. In addition, it is possible to learn security abnormalities according to the bicycle parking position so as not to erroneously detect vibration due to noise.

For example, when the learning unit 23 accumulates data on the date and time when an event with a large vibration due to noise such as a fireworks display is held around the motorcycle from the surrounding information, the learning unit 23 falsely detects the cause of the noise of the event. It is possible to learn. In addition, it is possible to learn security abnormalities according to the date and time of the event so as not to erroneously detect vibration due to the noise of the event.

The learning unit 23 learns by artificial intelligence based on the sensor data, peripheral information, and the judgment information acquired by the operation control unit 53, and generates a learned model for determining a security abnormality and a false detection. May be good. For example, even if the sensor data is equal to or greater than the threshold value, the learning unit 23 can learn the erroneous detection when the operation information indicating that the user has determined that the erroneous detection is accumulated is accumulated.

The learning unit 23 updates the DNN model and outputs it to the storage unit 12 every time data is acquired from the data acquisition unit 22, at a predetermined period, or at a desired timing.

The learning unit 23 acquires an appropriate threshold value of the sensor data used in the processing of the sensor determination unit 49 of the abnormality monitoring device 30 according to the generated DNN model. The acquired appropriate threshold value is transmitted to the abnormality monitoring device 30.

The determination unit 24 will be described. When the determination unit 24 acquires the sensor data for determination and the peripheral information from the abnormality monitoring device 30, the determination unit 24 uses DNN to determine whether the security is abnormal or erroneous detection. More specifically, the determination unit 24 reads the DNN model learned from the storage unit 12 to reproduce the DNN, and determines whether the sensor data for determination and the peripheral information are security abnormalities or false positives. .. The determination in the determination unit 24 may be executed by the same method as the determination in the known deep learning.

The storage control unit 25 stores the sensor data acquired by the data acquisition unit 22 and peripheral information in the storage unit 12 as teacher data to be used in the abnormality monitoring system 1. When the data acquisition unit 22 acquires the data, the storage control unit 25 updates the teacher data stored in the storage unit 12. The storage control unit 25 updates the DNN model stored in the storage unit 12 every time the learning unit 23 generates or updates the DNN model.

The abnormality monitoring device 30 is arranged, for example, in a motorcycle. In the present embodiment, the abnormality monitoring device 30 will be described as an integrated device having a function of a drive recorder for a motorcycle and a security device. The abnormality monitoring device 30 includes a communication unit 31, a GPS receiving unit 32, a camera 33, a microphone 34, an acceleration sensor 35, a speaker 36, an operation unit 37, a storage unit 39, and a control unit 40. ..

The abnormality monitoring device 30 monitors a security abnormality while the motorcycle is parked. The abnormality monitoring device 30 determines a security abnormality that has occurred in the motorcycle by the server device 10 based on the sensor data detected by the sensor arranged on the motorcycle and the video data obtained by the camera 33 taking a picture of the surroundings. Let me. When it is determined that there is a security abnormality, the abnormality monitoring device 30 outputs an alarm voice or notifies the user. Sensors include, for example, acceleration sensors, motion sensors, temperature sensors, pressure sensors and the like. In the present embodiment, the case where the acceleration sensor is used will be described, but the sensor is not limited to this, and other sensors may be used, or a plurality of types of sensors may be arranged.

The communication unit 31 is a communication unit. The communication unit 31 enables communication with the server device 10. In the present embodiment, the communication unit 31 can always communicate with the server device 10 when the abnormality monitoring system 1 is operating normally.

The GPS receiving unit 32 receives radio waves from GPS satellites (not shown). The GPS receiving unit 32 outputs the signal of the received radio wave to the position information acquisition unit 43 of the control unit 40.

The camera 33 is a camera that captures video data around the motorcycle. The number of cameras 33 may be one or a plurality. The camera 33 functions as a drive recorder camera while the motorcycle engine is on. In the present embodiment, the camera 33 starts shooting when the acceleration sensor 35 detects an acceleration equal to or higher than a predetermined threshold value while the motorcycle is parked, and can store video data for a predetermined period of, for example, about several tens of seconds. Is. The camera 33 outputs the captured video data to the video data acquisition unit 44 of the control unit 40. The video data is, for example, a moving image composed of an image of 30 frames per second. The camera 33 starts shooting based on the determination result of the sensor determination unit 49 of the control unit 40, and outputs the captured video data to the video data acquisition unit 44.

The microphone 34 is a microphone that acquires voice data around a motorcycle. The number of microphones 34 may be one or a plurality. The microphone 34 outputs the acquired voice data to the voice data acquisition unit 45 of the control unit 40.

The acceleration sensor 35 is a sensor that detects the acceleration generated on the motorcycle. The acceleration sensor 35 detects by acceleration that, for example, an impact due to mischief on the motorcycle has been applied, or that the posture has changed due to the motorcycle being knocked down or moved. The acceleration sensor 35 outputs the sensor data, which is the detection result, to the sensor data acquisition unit 48 of the control unit 40. The acceleration sensor 35 is, for example, a sensor that detects acceleration in three axial directions. The three-axis directions are the front-rear direction, the left-right direction, and the up-down direction of the motorcycle.

The speaker 36 is an audio output device. The speaker 36 outputs a warning voice based on the voice signal output from the output control unit 52.

The operation unit 37 can accept various operations on the abnormality monitoring device 30. For example, the operation unit 37 has a function of stopping the output of the alarm sound when the alarm sound is output. For example, the operation unit 37 accepts an operation in which the user determines a false detection, such as when an alarm voice is output. The operation unit 37 outputs the operation information to the operation control unit (user judgment information acquisition unit) 53 of the control unit 40. The operation unit 37 may be arranged in the abnormality monitoring device 30, may be arranged in any of the vehicle bodies of the motorcycle, or may be an operation unit of the electronic device used by the user. ..

The storage unit 39 is used for temporary storage of data in the abnormality monitoring device 30 and the like. The storage unit 39 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage unit such as a memory card. Alternatively, it may be an external storage unit that is wirelessly connected via the communication unit 31. The storage unit 39 records video data based on the control signal output from the video storage control unit 46 of the control unit 40.

The control unit 40 is, for example, an arithmetic processing unit (control device) composed of a CPU or the like. The control unit 40 loads the stored program into the memory and executes the instructions included in the program. The control unit 40 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control unit 40 and the like.

The control unit 40 includes each function of the abnormality monitoring device 30, specifically, a communication unit 31, a GPS reception unit 32, a camera 33, an acceleration sensor 35, a storage unit 39, a speaker 36, an operation unit 37, and a control unit 40. Executes a disconnection monitoring process that constantly monitors the connection status of. For example, when a third party intentionally destroys the abnormality monitoring device 30, the communication unit 31, the GPS receiving unit 32, the camera 33, the acceleration sensor 35, the storage unit 39, the speaker 36, the operating unit 37, and the control unit 40 Is likely to be disconnected. The control unit 40 is disconnected by disconnecting at least one of the wiring of the communication unit 31, the GPS receiving unit 32, the camera 33, the acceleration sensor 35, the storage unit 39, the speaker 36, and the operation unit 37. When it is detected, the alarm sound is output and the user is notified immediately.

The control unit 40 includes a communication control unit 41, a peripheral information acquisition unit 42 including a position information acquisition unit 43, a video data acquisition unit 44, and an audio data acquisition unit 45, a video storage control unit 46, and a bicycle parking detection unit 47. A sensor data acquisition unit 48, a sensor determination unit 49, a determination result acquisition unit 51, an output control unit 52, and an operation control unit 53.

The communication control unit 41 controls communication with the server device 10 via the communication unit 31. In the present embodiment, the communication control unit 41 acquires sensor data acquired by the sensor data acquisition unit 48, position information acquired by the position information acquisition unit 43, video data acquired by the video data acquisition unit 44, and audio data. The voice data acquired by the unit 45 is controlled to be transmitted to the server device 10. The communication control unit 41 constantly monitors the communication status with the server device 10.

The peripheral information acquisition unit 42 acquires peripheral information indicating the surrounding state of the motorcycle by the position information acquisition unit 43, the video data acquisition unit 44, and the audio data acquisition unit 45.

The position information acquisition unit 43 calculates the current position information of the motorcycle based on the radio waves received by the GPS reception unit 32 by a known method. In the present embodiment, the position information acquired by the position information acquisition unit 43 is included in the peripheral information.

The video data acquisition unit 44 acquires video data obtained by photographing the periphery of the motorcycle. More specifically, the video data acquisition unit 44 acquires the video data around the motorcycle output by the camera 33 arranged on the motorcycle and outputs it to the video storage control unit 46. The video data is included in the peripheral information.

The voice data acquisition unit 45 acquires voice data obtained by recording the voice around the motorcycle. More specifically, the voice data acquisition unit 45 acquires voice data around the motorcycle, which is output by the microphone 34 arranged on the motorcycle. Audio data is included in the peripheral information.

The video storage control unit 46 uses, for example, H.A. It converts to any file format encoded by any type of codec such as 264 or MPEG-4 (Moving Picture Experts Group), for example, MP4 format. The period of the video data generated as a file is, for example, 60 seconds, but is not limited to this. Further, the video data may be data including audio in addition to the video captured by the camera 33. The video storage control unit 46 controls the storage unit 39 to record the filed video data.

Based on the information acquired from the motorcycle, the bicycle parking detection unit 47 detects that the motorcycle is in the parked state, in other words, is in the state of starting the monitoring process for monitoring the security abnormality.

The state in which the motorcycle is parked means, for example, a state in which the speed becomes zero for a time of 5 seconds or more, the engine is stopped, or the like. Alternatively, when an arbitrary trigger such as a user operation occurs, it may be determined that the motorcycle is in the parked state.

The sensor data acquisition unit 48 acquires sensor data from a sensor that detects an abnormality in the security of a motorcycle. In the present embodiment, the sensor data acquisition unit 48 acquires the sensor data of the acceleration sensor 35. The sensor data acquisition unit 48 outputs the acquired sensor data to the sensor determination unit 49.

Based on the sensor data of the acceleration sensor 35, the sensor determination unit 49 determines whether or not there is an impact due to mischief on the motorcycle or a change in posture due to the motorcycle being knocked down or moved. More specifically, the sensor determination unit 49 detects whether or not the acceleration sensor 35 has detected an acceleration equal to or greater than the threshold value. The threshold value is a value of acceleration that can be detected when an impact such as mischief is applied to the motorcycle or when the posture of the motorcycle changes. When the sensor determination unit 49 determines that the acceleration equal to or higher than the threshold value is detected, it determines that the impact or the posture has changed due to mischief or the like. When the sensor determination unit 49 does not determine that the acceleration equal to or higher than the threshold value is detected, it determines that there is no impact or change in posture due to mischief or the like.

The threshold value used in the sensor determination unit 49 is updated every time the DNN model is updated in the server device 10.

The determination result acquisition unit 51 acquires the determination result by having the server device 10 determine the presence or absence of a security abnormality according to the determination result of the sensor determination unit 49. More specifically, when the determination result acquisition unit 51 determines that the sensor determination unit 49 has an impact or a change in posture due to mischief or the like, the determination result acquisition unit 51 transmits the determination sensor data and peripheral information to the server device via the communication control unit 41. It is transmitted to 10 to determine whether or not there is a security abnormality. The determination result acquisition unit 51 acquires the determination result from the server device 10.

The output control unit 52 controls the output of the alarm sound based on the determination result acquired by the determination result acquisition unit 51. More specifically, when the determination result indicates that the determination result is not a false detection but a security abnormality, the output control unit 52 outputs a voice signal so as to output an alarm sound from the speaker 36.

Further, when it is determined that the connection between each function of the abnormality monitoring device 30 and the control unit 40 is disconnected, the output control unit 52 outputs a voice signal so as to output an alarm sound from the speaker 36. Further, the output control unit 52 controls to output a notification to the electronic device or the like used by the user when it is determined that the connection between each function of the abnormality monitoring device 30 and the control unit 40 is disconnected. Output a signal.

The operation control unit 53 acquires the operation information of the operation received by the operation unit 37. For example, the operation control unit 53 acquires the alarm stop operation information indicating the operation of stopping the output of the alarm sound when the alarm sound is output. For example, when the user determines that the erroneous detection is made when the alarm voice is output, the operation control unit 53 acquires the determination information indicating the operation for which the user has determined the erroneous detection.

Next, the abnormality diagnosis method and operation of the abnormality monitoring system will be described with reference to FIGS. 4 to 10. FIG. 4 is a flowchart showing a flow of learning processing in the server device according to the first embodiment. FIG. 5 is a flowchart showing a flow of disconnection monitoring processing in the abnormality monitoring device according to the first embodiment. FIG. 6 is a flowchart showing a flow of monitoring processing in the abnormality monitoring device according to the first embodiment. FIG. 7 is a flowchart showing a processing flow in the abnormality monitoring device according to the first embodiment. FIG. 8 is a flowchart showing a processing flow in the abnormality monitoring device according to the first embodiment. FIG. 9 is a flowchart showing a processing flow in the abnormality monitoring device according to the first embodiment. FIG. 10 is a flowchart showing a flow of determination processing in the server device according to the first embodiment.

The learning process in the server device 10 will be described with reference to FIG. In the server device 10, the control unit 20 acquires sensor data and peripheral information from the plurality of abnormality monitoring devices 30 by the data acquisition unit 22 (step S101). The control unit 20 stores the acquired sensor data and peripheral information in the storage unit 12 by the storage control unit 25. The control unit 20 proceeds to step S102.

The control unit 20 uses the sensor data and peripheral information of the plurality of abnormality monitoring devices 30 stored in the storage unit 12 by the learning unit 23 as a DNN as a learned model for determining a security abnormality and a false detection. The weight of is learned by artificial intelligence (step S102). The control unit 20 proceeds to step S103.

The control unit 20 stores the learned model generated by the storage control unit 25 in the storage unit 12 (step S103). The control unit 20 ends the process.

In this way, the trained model is generated in advance in the server device 10 before the processing in the abnormality monitoring device 30 of the abnormality monitoring system 1 is executed. The server device 10 updates the DNN model every time data is acquired from the data acquisition unit 22, at a predetermined period, or at a desired timing.

The disconnection monitoring process in the abnormality monitoring device 30 will be described with reference to FIG. In the abnormality monitoring device 30, the control unit 40 determines whether or not the connection between each function of the abnormality monitoring device 30 and the control unit 40 is disconnected (step S201). When the communication control unit 41 determines that the connection has been disconnected (Yes in step S201), the process proceeds to step S202. When it is determined that the connection between each function of the abnormality monitoring device 30 and the control unit 40 is disconnected, it is highly possible that the abnormality monitoring device 30 has been intentionally destroyed by a third party, so an alarm sound is output immediately. , It is preferable to notify the user. If the communication control unit 41 does not determine that the connection between each function of the abnormality monitoring device 30 and the control unit 40 has been disconnected (No in step S201), the process of step S201 is executed again.

When it is determined that the connection is disconnected (Yes in step S201), the control unit 40 outputs a voice signal by the output control unit 52 so as to output an alarm sound from the speaker 36 (step S202). The control unit 40 proceeds to step S203.

The control unit 40 outputs a control signal by the output control unit 52 so as to output a notification to an electronic device or the like used by the user (step S203). The control unit 40 proceeds to step S204.

The control unit 40 determines whether or not the operation control unit 53 has acquired the operation information indicating that the alarm release operation has been executed (step S204). When the control unit 40 determines that the operation information indicating that the alarm release operation has been executed has been acquired (Yes in step S204), the control unit 40 proceeds to step S205. If the control unit 40 does not determine that the operation information indicating that the alarm release operation has been executed has been acquired (No in step S204), the control unit 40 proceeds to step S206.

When it is determined that the alarm release operation has been executed (Yes in step S204), the control unit 40 stops the output of the voice signal by the output control unit 52 so as to stop the output of the alarm sound from the speaker 36 (step S205). .. The control unit 40 ends the process.

If it is not determined that the alarm release operation has been executed (No in step S204), the control unit 40 determines the connection state between each function of the abnormality monitoring device 30 and the control unit 40 in the state in which the connection is determined to be disconnected in step S201. The sensor data, the position data, and the video data are transmitted to the server device 10 together with the indicated information (step S206). The control unit 40 ends the process.

The monitoring process in the abnormality monitoring device 30 will be described with reference to FIG. In the abnormality monitoring device 30, the control unit 40 acquires acceleration as sensor data from the acceleration sensor 35 by the sensor data acquisition unit 48 (step S211). The control unit 40 proceeds to step S212.

The control unit 40 determines whether or not the acceleration, which is the sensor data, is equal to or greater than the threshold value by the sensor determination unit 49 (step S212). When the sensor determination unit 49 determines that the acceleration is equal to or higher than the threshold value (Yes in step S212), the control unit 40 proceeds to step S213. If the sensor determination unit 49 does not determine that the acceleration is equal to or greater than the threshold value (No in step S212), the control unit 40 re-executes the process of step S211.

The control unit 40 executes the determination result acquisition process (step S213). The control unit 40 ends the process.

The determination result acquisition process in step S213 of the flowchart shown in FIG. 6 will be described with reference to FIG. 7. The control unit 40 activates the camera 33 and starts shooting (step S221). The control unit 40 proceeds to step S222.

The control unit 40 determines that the sensor determination unit 49 has detected an acceleration equal to or higher than the threshold value by the determination result acquisition unit 51, and the sensor data of the acceleration sensor 35 and the position information acquired by the position information acquisition unit 43. The video data that started shooting in step S221 is transmitted to the server device 10 as determination data (step S222). The control unit 40 proceeds to step S223.

The control unit 40 determines whether or not the determination result has been acquired from the server device 10 by the determination result acquisition unit 51 (step S223). When the control unit 40 determines that the determination result has been acquired (Yes in step S223), the control unit 40 proceeds to step S224. If the control unit 40 does not determine that the determination result has been acquired (No in step S223), the control unit 40 re-executes the process of step S223.

When it is determined that the determination result has been acquired (Yes in step S223), the control unit 40 determines whether or not the determination result is a security abnormality (step S224). When the control unit 40 determines that the determination result is a security abnormality (Yes in step S224), the control unit 40 proceeds to step S225. If the control unit 40 does not determine that the determination result is a security abnormality (No in step S224), the control unit 40 proceeds to step S226.

When the control unit 40 determines that the determination result is a security abnormality (Yes in step S224), the control unit 40 executes an abnormality processing (step S225). The control unit 40 ends the process.

When the determination result is not determined to be a security abnormality (No in step S224), the control unit 40 determines whether or not the determination result is an erroneous detection (step S226). When the control unit 40 determines that the determination result is an erroneous detection (Yes in step S226), the control unit 40 proceeds to step S227. If the control unit 40 does not determine that the determination result is an erroneous detection (No in step S226), in other words, if the server device 10 cannot determine, the control unit 40 proceeds to step S228.

When the control unit 40 determines that the determination result is an erroneous detection (Yes in step S226), the output control unit 52 performs a process of notifying the user of the erroneous detection (step S227). The false detection notification may be output from the speaker 36, or may be output to an electronic device or the like used by the user. The control unit 40 ends the process.

If the control unit 40 does not determine that the determination result is an erroneous detection (No in step S226), the control unit 40 executes a determination process by the user (step S228). The control unit 40 ends the process.

The abnormality processing in step S225 of the flowchart shown in FIG. 7 will be described with reference to FIG. The control unit 40 outputs a voice signal by the output control unit 52 so as to output the alarm sound from the speaker 36 (step S231). The control unit 40 proceeds to step S232.

The control unit 40 outputs a control signal by the output control unit 52 so as to output a notification warning that a security abnormality has occurred to an electronic device or the like used by the user (step S232). In this case, the output control unit 52 transmits the captured video data to the electronic device or the like used by the user. The user checks the video data and determines whether the security is abnormal or falsely detected. The control unit 40 proceeds to step S233.

The control unit 40 determines whether or not the operation control unit 53 has acquired the operation information indicating that the alarm release operation has been executed (step S233). When the control unit 40 determines that the operation information indicating that the alarm release operation has been executed has been acquired (Yes in step S233), the control unit 40 proceeds to step S234. If the control unit 40 does not determine that the operation information indicating that the alarm release operation has been executed has been acquired (No in step S233), the control unit 40 proceeds to step S236.

When it is determined that the alarm release operation has been executed (Yes in step S233), the control unit 40 stops the output of the voice signal by the output control unit 52 so as to stop the output of the alarm sound from the speaker 36 (step S234). .. The control unit 40 proceeds to step S235.

The control unit 40 adds information indicating that the alarm has been canceled by the user to the determination data transmitted to the server device 10, and transmits the data to the server device 10 (step S235). The server device 10 updates the teacher data stored in the storage unit 12 based on the acquired determination data. The control unit 40 ends the process.

When it is not determined that the alarm release operation has been executed (No in step S233), the control unit 40 transmits the determination data transmitted to the server device 10 and the information indicating that the alarm has not been released to the server device 10. (Step S236). The server device 10 updates the teacher data stored in the storage unit 12 based on the acquired determination data. The control unit 40 ends the process.

The user judgment process of step S228 of the flowchart shown in FIG. 7 will be described with reference to FIG. The control unit 40 outputs a voice signal by the output control unit 52 so as to output the alarm sound from the speaker 36 (step S241). The control unit 40 proceeds to step S242.

The control unit 40 outputs a control signal so that the output control unit 52 outputs a notification warning that the security abnormality could not be determined to the electronic device or the like used by the user (step S242). ). In this case, the output control unit 52 transmits the captured video data to the electronic device or the like used by the user. The user checks the video data and determines whether the security is abnormal or falsely detected. The control unit 40 proceeds to step S243.

The control unit 40 determines whether or not the operation control unit 53 has acquired the operation information indicating that the erroneous detection determination operation has been executed (step S243). When the control unit 40 determines that the operation information indicating that the false detection determination operation has been executed has been acquired (Yes in step S243), the control unit 40 proceeds to step S244. If the control unit 40 does not determine that the operation information indicating that the false detection determination operation has been executed has been acquired (No in step S243), the control unit 40 proceeds to step S246.

When it is determined that the false detection determination operation has been executed (Yes in step S243), the control unit 40 stops the output of the voice signal by the output control unit 52 so as to stop the output of the alarm sound from the speaker 36 (step S244). ). The control unit 40 proceeds to step S245.

The control unit 40 adds information indicating that the user has determined that the error has been detected to the determination data transmitted to the server device 10, and transmits the data to the server device 10 (step S245). The server device 10 updates the teacher data stored in the storage unit 12 based on the acquired determination data. The control unit 40 ends the process.

When it is not determined that the false detection determination operation has been executed (No in step S243), the control unit 40 includes the determination data transmitted to the server device 10 and information indicating that the user has determined that the security is abnormal. Is transmitted to the server device 10 (step S246). The server device 10 updates the teacher data stored in the storage unit 12 based on the acquired determination data. The control unit 40 ends the process.

The determination process in the server device 10 will be described with reference to FIG. In the server device 10, the control unit 20 acquires the sensor data for determination and the peripheral information from the abnormality monitoring device 30 by the data acquisition unit 22 (step S111). The control unit 20 proceeds to step S112.

The control unit 20 uses the DNN to determine the security abnormality and the false detection based on the sensor data for determination and the peripheral information by the determination unit 24 (step S112). The control unit 20 proceeds to step S113.

The control unit 20 transmits the determination result to the abnormality monitoring device 30 by the communication control unit 21 (step S113). The control unit 20 ends the process.

In this way, in the server device 10, DNN as a learned model for determining a security abnormality and a false detection by machine learning by artificial intelligence using sensor data acquired from a plurality of abnormality monitoring devices 30 and peripheral information. To generate. When the server device 10 acquires the sensor data for determination and the peripheral information from the abnormality monitoring device 30, the server device 10 uses the DNN to determine a security abnormality and an erroneous detection.

In the abnormality monitoring device 30, based on the sensor data detected by the sensor placed on the motorcycle and the video data obtained by the camera 33 taking a picture of the surroundings, the server device 10 causes the server device 10 to detect a security abnormality generated in the motorcycle by DNN. To make a judgment using.

As described above, in the present embodiment, in the server device 10, a security abnormality and a false detection are determined by machine learning by artificial intelligence using sensor data acquired from a plurality of abnormality monitoring devices 30 and peripheral information. DNN can be generated as a trained model. In the present embodiment, when the server device 10 acquires the sensor data for determination and the peripheral information from the abnormality monitoring device 30, the DNN can be used to determine a security abnormality and an erroneous detection.

According to the present embodiment, by machine learning by artificial intelligence, a DNN model can be generated to appropriately determine a state in which a security is abnormal and a state in which a false detection is made. As a result, the present embodiment can reduce erroneous detection according to the peripheral information of the motorcycle.

In the present embodiment, the server device 10 generates the abnormality for the motorcycle based on the sensor data detected by the sensor arranged on the motorcycle and the video data obtained by the camera 33 in the periphery of the abnormality monitoring device 30. Security anomalies can be determined using DNN. In this way, according to the present embodiment, it is possible to monitor the security abnormality with high accuracy while the motorcycle is parked.

[Second Embodiment]
The abnormality monitoring system 1 according to the present embodiment will be described with reference to FIGS. 11 and 12. FIG. 11 is a schematic view showing a configuration example of an imaging device for abnormality monitoring according to the second embodiment. FIG. 12 is a schematic view showing a configuration example of the abnormality monitoring device according to the second embodiment. The present embodiment is different from the first embodiment in that the functions of the abnormality monitoring device 30 of the first embodiment are provided separately for the photographing device 30A for abnormality monitoring and the abnormality monitoring device 30B. The basic configuration of each function of the photographing device 30A and the abnormality monitoring device 30B is the same as that of the abnormality monitoring device 30 of the first embodiment. In the following description, components similar to the abnormality monitoring device 30 are designated by the same reference numerals or corresponding reference numerals, and detailed description thereof will be omitted.

The photographing device 30A for abnormality monitoring has a function of a drive recorder for a motorcycle. The photographing device 30A includes a communication unit 31A, a GPS receiving unit 32A, a camera 33A, a storage unit 39A, and a control unit 40A. The photographing device 30A transmits the photographed video data to the server device 10.

The control unit 40A includes a communication control unit 41A, a peripheral information acquisition unit 42A including a position information acquisition unit 43A, a video data acquisition unit 44A, and an audio data acquisition unit 45A, and a video storage control unit 46A.

The abnormality monitoring device 30B has a function of a security device for a motorcycle. The abnormality monitoring device 30B includes a communication unit 31B, an acceleration sensor 35B, a speaker 36B, an operation unit 37B, and a control unit 40B. The abnormality monitoring device 30B transmits the sensor data to the server device 10.

The control unit 40B includes a communication control unit 41B, a bicycle parking detection unit 47B, a sensor data acquisition unit 48B, a sensor determination unit 49B, a determination result acquisition unit 51B, an output control unit 52B, and an operation control unit 53B. Have.

The server device 10 executes processing by associating video data, position information, and audio data, which are peripheral information received from the photographing device 30A, with sensor data received from the abnormality monitoring device 30B.

As described above, the present embodiment can be divided into the photographing device 30A and the abnormality monitoring device 30B. According to the present embodiment, the abnormality monitoring device 30B can be arranged at a position where it is difficult to see from the outside or inside the vehicle body of the motorcycle. According to the present embodiment, it is possible to make it difficult for a third party to notice that the security abnormality is being monitored while the motorcycle is parked.

[Third Embodiment]
The abnormality monitoring system 1 according to the present embodiment will be described with reference to FIG. FIG. 13 is a schematic view showing a configuration example of the abnormality monitoring device according to the third embodiment. This embodiment is different from the first embodiment in that the camera function is removed from the abnormality monitoring device 30 of the first embodiment. The basic configuration of each function of the abnormality monitoring device 30C is the same as that of the abnormality monitoring device 30 of the first embodiment.

The abnormality monitoring device 30C includes a communication unit 31C, a GPS receiving unit 32C, an acceleration sensor 35C, a storage unit 39C, a speaker 36C, an operation unit 37C, and a control unit 40C.

The control unit 40C includes a communication control unit 41C, a peripheral information acquisition unit 42C including a position information acquisition unit 43C, a video data acquisition unit 44C, and an audio data acquisition unit 45C, a video storage control unit 46C, and a bicycle parking detection unit 47C. It has a sensor data acquisition unit 48C, a sensor determination unit 49C, a determination result acquisition unit 51C, an output control unit 52C, and an operation control unit 53C. The video data acquisition unit 44C acquires video data from an external camera. The voice data acquisition unit 45C acquires voice data from an external microphone.

As described above, the present embodiment can be configured by removing the camera function from the abnormality monitoring device 30C. According to the present embodiment, the abnormality monitoring device 30C can be arranged at a position where it is difficult to see from the outside or inside the vehicle body of the motorcycle.

[Fourth Embodiment]
The abnormality monitoring system 1 according to the present embodiment will be described with reference to FIGS. 14 and 15. FIG. 14 is a schematic view showing a configuration example of an imaging device for abnormality monitoring according to a fourth embodiment. FIG. 15 is a schematic view showing a configuration example of the abnormality monitoring device according to the fourth embodiment. This embodiment is different from the second embodiment in that the photographing device 30D for abnormality monitoring does not have a camera function.

The photographing device 30D includes a communication unit 31D, a GPS receiving unit 32D, a storage unit 39D, and a control unit 40D.

The control unit 40D includes a communication control unit 41D, a peripheral information acquisition unit 42D including a position information acquisition unit 43D, a video data acquisition unit 44D, and an audio data acquisition unit 45D, and a video storage control unit 46D. The video data acquisition unit 44D acquires video data from an external camera. The voice data acquisition unit 45D acquires voice data from an external microphone.

The abnormality monitoring device 30E includes a communication unit 31E, an acceleration sensor 35E, a speaker 36E, an operation unit 37E, and a control unit 40E.

The control unit 40E includes a communication control unit 41E, a bicycle parking detection unit 47E, a sensor data acquisition unit 48E, a sensor determination unit 49E, a determination result acquisition unit 51E, an output control unit 52E, and an operation control unit 53E. Have.

As described above, the present embodiment can be configured separately by the photographing device 30D and the abnormality monitoring device 30E, and further, the abnormality monitoring device 30C can be configured by removing the camera function. According to the present embodiment, the photographing device 30D and the abnormality monitoring device 30E can be arranged at a position where it is difficult to see from the outside or inside the vehicle body of the motorcycle.

By the way, although the abnormality monitoring system 1 according to the present invention has been described so far, it may be implemented in various different forms other than the above-described embodiment.

Each component of the abnormality monitoring system 1 shown in the figure is a functional concept and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of each device is not limited to the one shown in the figure, and all or part of each device is functionally or physically dispersed or integrated in an arbitrary unit according to the processing load and usage status of each device. You may.

The configuration of the abnormality monitoring system 1 is realized, for example, by a program loaded in a memory as software. In the above embodiment, it has been described as a functional block realized by linking these hardware or software. That is, these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The above-mentioned components include those that can be easily assumed by those skilled in the art and those that are substantially the same. Further, the above configurations can be combined as appropriate. Further, various omissions, substitutions or changes of the configuration can be made without departing from the gist of the present invention.

In the above, also in step S227, the captured video data may be transmitted when the notification is output to the electronic device or the like used by the user. As a result, the user may be able to check the video data and determine whether it is a security abnormality or a false positive.

Further, when the motorcycle is damaged by theft, the detected sensor data and peripheral information may be stored in an external server device so that the motorcycle can be used for identifying the stolen vehicle.

1 Abnormality monitoring system 10 Server device 20 Control unit 23 Learning unit 24 Judgment unit 30 Abnormality monitoring device 33 Camera 35 Accelerometer (sensor)
40 Control unit 42 Peripheral information acquisition unit 43 Position information acquisition unit 44 Video data acquisition unit 47 Bicycle parking detection unit 48 Sensor data acquisition unit 49 Sensor judgment unit 51 Judgment result acquisition unit 53 Operation control unit (user judgment information acquisition unit)

Claims (5)

A sensor data acquisition unit that acquires sensor data from a sensor that detects security abnormalities on a moving object,
Peripheral information acquisition unit that acquires peripheral information indicating the surrounding state of the moving body,
A trained model for determining a security abnormality and a false detection by learning by artificial intelligence based on the sensor data acquired by the sensor data acquisition unit and the peripheral information acquired by the peripheral information acquisition unit. With the learning department that generates
Using the learned model learned by the learning unit, a security abnormality and erroneous detection are made based on the sensor data acquired by the sensor data acquisition unit and the peripheral information acquired by the peripheral information acquisition unit. Judgment unit that determines
An abnormality monitoring system characterized by being equipped with. The peripheral information includes meteorological information indicating the weather state around the moving body, object detection information indicating the detection of an object in the vicinity of the moving body, audio information indicating the sound around the moving body, and the moving body. Contains at least one of the location information that indicates the current location of
The abnormality monitoring system according to claim 1. The peripheral information acquisition unit includes a video data acquisition unit that acquires video data around the moving body, which is captured by a photographing unit arranged on the moving body.
The abnormality monitoring system according to claim 1 or 2. User judgment information acquisition unit that acquires judgment information indicating the judgment result that the user has judged to be a false positive
With
The learning unit uses artificial intelligence based on the sensor data acquired by the sensor data acquisition unit, the peripheral information acquired by the peripheral information acquisition unit, and the determination information acquired by the user judgment information acquisition unit. Train to generate a trained model for determining security anomalies and false positives,
The abnormality monitoring system according to any one of claims 1 to 3. The sensor data acquisition step to acquire sensor data from the sensor that detects the security abnormality of the moving object,
Peripheral information acquisition step to acquire peripheral information indicating the surrounding state of the moving body,
A trained model for determining a security abnormality and a false detection by learning by artificial intelligence based on the sensor data acquired in the sensor data acquisition step and the peripheral information acquired in the peripheral information acquisition step. With learning steps to generate
Using the trained model learned in the learning step, a security abnormality and erroneous detection are made based on the sensor data acquired in the sensor data acquisition step and the peripheral information acquired in the peripheral information acquisition step. And the judgment step to judge
Abnormality monitoring method including.

Images