CN111626905A - Passenger safety monitoring method and device and computer readable storage medium - Google Patents
Passenger safety monitoring method and device and computer readable storage medium Download PDFInfo
- Publication number
- CN111626905A CN111626905A CN202010453315.7A CN202010453315A CN111626905A CN 111626905 A CN111626905 A CN 111626905A CN 202010453315 A CN202010453315 A CN 202010453315A CN 111626905 A CN111626905 A CN 111626905A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- passenger
- safety monitoring
- safety
- environment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012544 monitoring process Methods 0.000 title claims abstract description 95
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000004927 fusion Effects 0.000 claims abstract description 49
- 238000012806 monitoring device Methods 0.000 claims abstract description 30
- 238000004891 communication Methods 0.000 claims description 36
- 238000011156 evaluation Methods 0.000 claims description 27
- 230000008447 perception Effects 0.000 claims description 24
- 238000004590 computer program Methods 0.000 claims description 21
- 230000007613 environmental effect Effects 0.000 claims description 18
- 238000010586 diagram Methods 0.000 claims description 10
- 230000004044 response Effects 0.000 claims description 7
- 238000013507 mapping Methods 0.000 claims description 3
- 238000001514 detection method Methods 0.000 abstract description 19
- 230000000694 effects Effects 0.000 abstract description 10
- 230000000875 corresponding effect Effects 0.000 description 21
- 238000005516 engineering process Methods 0.000 description 7
- 238000012545 processing Methods 0.000 description 5
- 230000001276 controlling effect Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000012549 training Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- CURLTUGMZLYLDI-UHFFFAOYSA-N Carbon dioxide Chemical compound O=C=O CURLTUGMZLYLDI-UHFFFAOYSA-N 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000011217 control strategy Methods 0.000 description 2
- 206010003497 Asphyxia Diseases 0.000 description 1
- 206010019345 Heat stroke Diseases 0.000 description 1
- 206010021143 Hypoxia Diseases 0.000 description 1
- 208000007180 Sunstroke Diseases 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 229910002092 carbon dioxide Inorganic materials 0.000 description 1
- 239000001569 carbon dioxide Substances 0.000 description 1
- 230000019771 cognition Effects 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001902 propagating effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
- G06Q50/10—Services
- G06Q50/26—Government or public services
- G06Q50/265—Personal security, identity or safety
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/251—Fusion techniques of input or preprocessed data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/02—Knowledge representation; Symbolic representation
- G06N5/022—Knowledge engineering; Knowledge acquisition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N5/00—Computing arrangements using knowledge-based models
- G06N5/04—Inference or reasoning models
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computational Linguistics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Tourism & Hospitality (AREA)
- Evolutionary Biology (AREA)
- Primary Health Care (AREA)
- Educational Administration (AREA)
- Computer Security & Cryptography (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Health & Medical Sciences (AREA)
- Human Resources & Organizations (AREA)
- Marketing (AREA)
- Development Economics (AREA)
- Strategic Management (AREA)
- General Business, Economics & Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Alarm Systems (AREA)
Abstract
According to the passenger safety monitoring method, the passenger safety monitoring device and the computer readable storage medium, multi-physical-domain information acquired by a distributed sensor is acquired through vehicle-mounted monitoring equipment, and safety detection fusion data are generated based on the multi-physical-domain information and uploaded to a server; and inputting the safety detection fusion data into an expert system through a server, and outputting a corresponding safety rescue decision. By implementing the invention, the multi-physical domain information is detected and fused, the acquired monitoring data is more comprehensive and accurate, the passenger safety detection effect is improved, and the independent decision is made by the expert system, so that effective indication can be provided for the rescue of the trapped passenger.
Description
Technical Field
The invention relates to the technical field of safety monitoring, in particular to a passenger safety monitoring method, a passenger safety monitoring device and a computer readable storage medium.
Background
With the continuous progress of society, the conservation quantity of automobiles in China is increased year by year, and the automobiles can bring great convenience for users to go out, but in recent years, the cases that children, old people, disabled people and other vulnerable groups are forgotten in the automobiles by car owners frequently occur, so that safety accidents of sunstroke and death caused by suffocation are caused, and the safety problem in the using process of the automobiles is not ignored.
At present, in order to avoid the condition that passengers are forgotten in the car by the car owner to take place, prior art proposes to be equipped with safety monitoring system on the car, also carries out the perception to stranded passenger through simple and easy sensor promptly, then reminds to the car owner, however, because data accuracy and the data comprehensiveness through the data that simple and easy sensor detected are comparatively limited to lead to passenger safety monitoring effect not good.
Disclosure of Invention
The present invention provides a passenger safety monitoring method, a passenger safety monitoring device, and a computer-readable storage medium, which can at least solve the problem of poor passenger safety monitoring effect of a passenger safety monitoring system provided for vehicle false locking in the prior art.
In order to achieve the above object, a first aspect of the embodiments of the present invention provides a passenger safety monitoring method applied to a vehicle-mounted monitoring device, the method including:
acquiring multi-physical-domain information which is acquired by a distributed sensor and comprises vehicle use information, vehicle environment parameters and in-vehicle passenger perception data;
determining a type of a trapped passenger based on the in-vehicle passenger perception data and determining an in-vehicle environment assessment result based on the vehicle environment parameter when it is determined that the vehicle is in a locked state based on the vehicle usage information;
generating safety monitoring fusion data based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
and uploading the safety monitoring fusion data to a server based on communication connection with the server.
In order to achieve the above object, a second aspect of the embodiments of the present invention provides a passenger safety monitoring method, applied to a server, the method including:
receiving safety monitoring fusion data uploaded by vehicle-mounted monitoring equipment; wherein the safety monitoring fusion data is generated based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
inputting the safety monitoring fusion data into a preset expert system; wherein the expert system comprises a knowledge base and an inference engine;
and controlling the inference engine to call the knowledge base and plan a safety rescue decision corresponding to the safety monitoring fusion data.
In order to achieve the above object, a third aspect of the embodiments of the present invention provides an occupant safety monitoring apparatus applied to a vehicle-mounted monitoring device, the apparatus including:
the acquisition module is used for acquiring multi-physical-domain information which is acquired by the distributed sensors and comprises vehicle use information, vehicle environment parameters and in-vehicle passenger perception data;
the determining module is used for determining the type of the trapped passenger based on the in-vehicle passenger perception data and determining the in-vehicle environment evaluation result based on the vehicle environment parameter when the vehicle is determined to be in the locked state based on the vehicle use information;
the generating module is used for generating safety monitoring fusion data based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
and the uploading module is used for uploading the safety monitoring fusion data to the server based on the communication connection with the server.
In order to achieve the above object, a fourth aspect of the embodiments of the present invention provides a passenger safety monitoring device applied to a server and applied to a vehicle-mounted monitoring apparatus, the device including:
the receiving module is used for receiving the safety monitoring fusion data uploaded by the vehicle-mounted monitoring equipment; wherein the safety monitoring fusion data is generated based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
the input module is used for inputting the safety monitoring fusion data to a preset expert system; wherein the expert system comprises a knowledge base and an inference engine;
and the planning module is used for controlling the inference machine to call the knowledge base and planning a safety rescue decision corresponding to the safety monitoring fusion data.
To achieve the above object, a fifth aspect of embodiments of the present invention provides an electronic apparatus, including: a processor, a memory, and a communication bus;
the communication bus is used for realizing connection communication between the processor and the memory;
the processor is configured to execute a first computer program and a second computer program stored in the memory, the processor implementing the steps of the method according to the first aspect when executing the first computer program, and implementing the steps of the method according to the second aspect when executing the second computer program.
In order to achieve the above object, a sixth aspect of the embodiments of the present invention provides a computer-readable storage medium, which stores a first computer program that, when executed by a processor, implements the steps of the method according to the first aspect, and a second computer program that, when executed by a processor, implements the steps of the method according to the second aspect.
According to the passenger safety monitoring method, the passenger safety monitoring device and the computer readable storage medium, multi-physical-domain information acquired by a distributed sensor is acquired through vehicle-mounted monitoring equipment, and safety detection fusion data are generated based on the multi-physical-domain information and uploaded to a server; and inputting the safety detection fusion data into an expert system through a server, and outputting a corresponding safety rescue decision. By implementing the invention, the multi-physical domain information is detected and fused, the acquired monitoring data is more comprehensive and accurate, the passenger safety detection effect is improved, and the independent decision is made by the expert system, so that effective indication can be provided for the rescue of the trapped passenger.
Other features and corresponding effects of the present invention are set forth in the following portions of the specification, and it should be understood that at least some of the effects are apparent from the description of the present invention.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a passenger safety monitoring method applied to a vehicle-mounted monitoring device according to a first embodiment of the present invention;
fig. 2 is a schematic flowchart of a method for determining a type of a trapped passenger according to a first embodiment of the present invention;
fig. 3 is a schematic flow chart of a passenger safety monitoring method applied to a server side according to a first embodiment of the present invention;
fig. 4 is a schematic diagram of program modules of a passenger safety monitoring device applied to a vehicle-mounted monitoring device according to a second embodiment of the invention;
fig. 5 is a schematic diagram of program modules applied to a server-side passenger safety monitoring device according to a second embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to a third embodiment of the invention.
Detailed Description
In order to make the objects, features and advantages of the present invention more obvious and understandable, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The first embodiment:
in order to solve the technical problem in the prior art that the passenger safety monitoring effect of a passenger safety monitoring system proposed for automobile false locking is not good, the present embodiment proposes a passenger safety monitoring method, which is applied to a vehicle-mounted monitoring device installed on a vehicle, as shown in fig. 1, which is a schematic flow diagram of the passenger safety monitoring method provided by the present embodiment and applied to a vehicle-mounted monitoring device side, and the passenger safety monitoring method proposed by the present embodiment includes the following steps:
Specifically, in this embodiment, a distributed multi-sensor based multi-physical domain data acquisition technology is provided, and a corresponding acquisition system is constructed to acquire vehicle usage information (whether the vehicle is a usage scene in which the vehicle is stopped and the door of the vehicle is locked), and vehicle environment parameters (position, in-vehicle temperature, humidity, and CO)2Parameters such as concentration and CO concentration), passenger perception data (video data, image data, infrared detection data and the like) in the vehicle and the like.
It should be noted that, for the collection of the vehicle use information, an ignition switch ACC gear detection module, a door switch state detection module, a door lock state detection module and a pressure sensor module can be adopted, which can detect whether the vehicle is in a flameout state, whether the door is closed and locked, whether a person is in a main driving position, etc.; for the collection of vehicle environment parameters, a Beidou/GPS double positioning module, a temperature sensor and a CO sensor module are adopted to detect the vehicle environment and acquire vehicle positioning information and vehicle internal environment information in time; for the acquisition of the sensing data of passengers in the vehicle, a camera, an infrared sensor and the like can be adopted, so that the monitoring and the identification of the passengers trapped in the vehicle can be realized.
And 102, when the vehicle is determined to be in a locked state based on the vehicle use information, determining the type of the trapped passenger based on the in-vehicle passenger perception data, and determining an in-vehicle environment evaluation result based on the vehicle environment parameter.
Specifically, in the embodiment, when it is detected that the vehicle is in a flameout state, the vehicle door is in a closed state, and no person is in the main driving position, it is determined that the vehicle door is locked and the vehicle owner leaves, and the vehicle is currently in a vehicle locking state; then, whether passengers exist in a copilot or a rear row in the vehicle is determined based on the sensing data of the passengers in the vehicle, and if so, the types of the trapped passengers (such as old people, children, handicapped people and the like) are identified; in addition, the environment in the vehicle where the trapped passenger is located is evaluated according to the vehicle environment parameters, the environment evaluation result is used for indicating the environment danger level, and different environment danger levels correspond to different life threat degrees.
And 103, generating safety monitoring fusion data based on the type of the trapped passenger and the evaluation result of the environment in the vehicle.
Specifically, the embodiment performs feature fusion on the type of trapped passengers and the evaluation result of the environment in the vehicle, performs complementation by using information from different sensors, eliminates the working blind area of a single sensor, can describe the environment object in all directions and all dimensions, overcomes the one-sidedness of perception, and improves the comprehensiveness and the correctness of system cognition. And the multisource information acquisition and parallel processing mechanism based on the fusion system can obviously improve the transmission and processing speed of information, increase the real-time performance of the system and effectively solve the problem of information flooding.
It should be noted that, in this embodiment, when a single sensor fails, the system can still work normally based on a perfect topology and a good self-organizing capability, and the complementation, redundancy and correlation characteristics of the multiple sensors to the environment description capability can ensure that the fusion result has good fault tolerance and reliability, thereby improving the robustness and robustness of the system.
And 104, uploading the safety monitoring fusion data to a server based on the communication connection with the server.
Specifically, in the embodiment, after monitoring of vehicle use information, vehicle environment parameters, in-vehicle passenger perception data and the like is realized by using a distributed multi-sensor-based multi-physical-domain data acquisition method and technology, a vehicle-mounted monitoring device based on multi-system digital mobile communication protocols such as Wifi and 3G/4G is provided, fused data is remotely and wirelessly transmitted to a server in real time, and trapped people in a vehicle are remotely monitored in a background.
As shown in fig. 2, which is a schematic flow chart of the method for determining a type of a trapped passenger provided in this embodiment, in an optional implementation manner of this embodiment, when the in-vehicle passenger perception data is an in-vehicle passenger image, determining the type of the trapped passenger based on the in-vehicle passenger perception data specifically includes the following steps:
Specifically, in this embodiment, based on the classification and recognition technology of the deep convolutional neural network, classification and recognition of people trapped in the vehicle (especially for the old, children, and handicapped people) can be realized by collecting a specific person sample and training the ssd _ mobile network model. The technology is used for classifying and identifying the old, children and disabled people by training an ssd _ mobilenet neural network model through tensoflow. In this embodiment, first, a development environment is built, a tensrflow-GPU version is installed, a tensrflow object detection API is downloaded, Protobuf is installed and configured, and environment variable settings are performed. And then creating a data set, and performing image annotation on the data set by using LabelImg software, wherein because Tensorflow needs to input a special TFRecords Format Format, the data set information is recorded into the csv table, and then the TFRecords Format is created from the csv table. Next, a configuration file ssd _ mobilenet _ v1_ coco.config is set, and a tab set text. And finally, model training is carried out, a better recognition effect is achieved after the loss is close to 0.5, and then the character recognition model after training is generated. The embodiment realizes more detailed understanding of the trapped people in the vehicle through the person identification so as to provide guidance for rescuing the trapped passengers.
It should be noted that, in this embodiment, the in-vehicle passenger sensing may be performed through multiple sensors, that is, the in-vehicle passenger sensing data is collected by combining the camera and the infrared sensor, then the type of the trapped passenger is identified based on the image data and the infrared detection data, and then the final trapped passenger identification result is output by combining the identification results of the two, so as to avoid misjudgment caused by a single detection mode.
It should be further noted that, in other embodiments, it may also be considered that the performance of different types of sensors in different usage scenarios is different, for example, the imaging level of the camera in a dark environment (e.g., at night, in a warehouse) is low, so that the accuracy of the type identification of the trapped passenger using the image of the passenger in the vehicle is poor, so that this embodiment may obtain valid in-vehicle passenger perception data from a plurality of in-vehicle passenger perception data according to the vehicle environment parameters, and then use the corresponding identification technology to identify the type of the trapped passenger.
In an optional implementation manner of this embodiment, the determining the evaluation result of the environment inside the vehicle based on the vehicle environment parameter includes: determining a corresponding environment risk index based on the vehicle environment parameter and a preset mapping relation between the vehicle environment parameter and the environment risk index; and generating an in-vehicle environment evaluation result based on the environmental risk index.
Specifically, the environmental risk indicator is configured for different vehicle environmental parameters, and the expression form of the environmental risk indicator may be a score value (for example, the score interval is 0 to 100, and the higher the score value is, the higher the risk is). In the embodiment, a plurality of vehicle environmental parameters are collected, and life threatening factors considered by different parameters are different, so that a final vehicle environmental evaluation result needs to be generated by combining the environmental risk indexes corresponding to the plurality of vehicle environmental parameters, specifically, a weighted average method can be adopted, that is, corresponding weighting coefficients are determined according to life threatening degrees corresponding to different vehicle environmental parameter types, in practical application, if the fatality probability of the oxygen deficiency ratio of passengers is higher than a high temperature, a relatively higher weighting coefficient can be set for the environmental risk index corresponding to the carbon dioxide concentration, and the weighting coefficient of the environmental risk index corresponding to the vehicle environmental temperature is relatively lower, then the environmental risk indexes of different types of vehicle environmental parameters are weighted by the corresponding weighting coefficients, and then, an average value is obtained, so that a comprehensive evaluation environmental risk index can be obtained and used as a final in-vehicle environmental evaluation result.
Correspondingly, the first embodiment of the present invention further provides another passenger safety monitoring method, which is applied to a server, and as shown in fig. 3, which is a basic flowchart of the passenger safety monitoring method applied to the server side provided in this embodiment, the passenger safety monitoring method includes the following steps:
and step 303, controlling the inference engine to call the knowledge base and planning a safety rescue decision corresponding to the safety monitoring fusion data.
Specifically, in this embodiment, the safety monitoring fusion data is generated based on the trapped passenger type and the in-vehicle environment evaluation result, the safety monitoring fusion data is contrasted and analyzed through the expert system, then an autonomous decision is made according to the analysis result, the alarm processing is timely performed on the trapped passenger in the vehicle, and an optimal rescue scheme is planned, so that the problems that a traditional monitoring system only stays on the level of collecting and displaying relevant information in the vehicle and remotely controlling, and a manual 24-hour watch and manual intervention decision making are needed are solved.
It should be noted that the Expert System (ES) is defined as: a computer model of human expert reasoning is used to deal with complex problems in the real world that require an expert to make an explanation and draw the same conclusions as the expert. In short, the expert system can be viewed as a combination of "knowledge base" and "inference machine". The main work of the knowledge base system is to collect human knowledge, express it systematically or modularize it so that the computer can make inference and solve problems. The inference engine deduces each special knowledge in the knowledge base by an algorithm or a control strategy, and derives a correct answer according to the question of the user. The knowledge base stores the memory of expert knowledge, experience, book knowledge, vehicle environment, personnel state and safety optimal solution, and how to plan the rescue scheme. The structural form of the knowledge base adopts a production rule expression knowledge method, which not only can express facts, but also can be attached with confidence factors to express the credibility of the facts, thereby realizing the accurate reasoning capability of an expert system. In addition, the embodiment may adopt an inexact reasoning method, and define a set of functions to find the uncertainty measure of the conclusion based on the rule strength given by the expert and the uncertainty of the original evidence given by the user. And (3) giving a certain certainty factor to uncertain knowledge at each position, calculating the certainty factor of the intermediate result according to an intelligent algorithm in the inference process, and propagating the uncertainty along the inference until a conclusion is reached. It should also be understood that the control strategy mainly refers to a selection strategy of a control and inference rule of an inference direction, and the embodiment can adopt a forward and reverse mixed inference method superior to a one-way inference method such as forward inference and reverse inference, and the method can obtain an assumption through forward inference according to important symptoms, then perform reverse inference by the assumption, and repeatedly search necessary conditions, thereby realizing condition analysis of people in the vehicle and obtaining an optimal autonomous decision support such as planning rescue and alarm processing.
In an optional implementation manner of this embodiment, after planning a safety rescue decision corresponding to the safety monitoring fused data, the method further includes: and sending the safety rescue decision to the vehicle owner communication terminal based on the communication connection with the vehicle owner communication terminal.
Specifically, in this embodiment, the content included in the safety rescue decision may be: the safety rescue decision is sent to the communication terminal of the vehicle owner (such as a mobile phone carried by the vehicle owner) so as to carry out rescue warning and indicate the rescue scheme to the vehicle owner, thereby helping the vehicle owner to carry out rescue timely and effectively.
Further, in an optional implementation manner of this embodiment, after sending the safety rescue decision to the owner communication terminal, the method further includes: judging whether a decision acquisition response sent by the vehicle owner communication terminal is received within a preset time length; when a decision-making learning response is not received, generating alarm information based on a safety rescue decision; and sending the alarm information to an alarm platform.
Specifically, in practical applications, after the server sends the safety rescue decision to the owner communication terminal, the owner may not know the safety rescue decision sent by the server in time due to objective factors (for example, the owner is in a meeting, etc.), thereby missing the opportunity to rescue the trapped passenger autonomously. In order to protect the life safety of passengers trapped in the vehicle practically, when a decision-making learning response fed back by the vehicle owner communication terminal is not received within a preset time period, a safety rescue decision is sent to the alarm platform to inform and assist police officers to timely and effectively rescue the passengers trapped in the vehicle.
In addition, it should be further noted that the server of this embodiment may also send a control instruction to the vehicle-mounted monitoring device, control the vehicle-mounted monitoring device to perform whistling alarm, or perform dialogue with a trapped passenger in the vehicle through a voice module of the vehicle-mounted monitoring device, so as to know the state and demand of the trapped passenger in real time, so as to further assist the trapped passenger in rescue.
According to the passenger safety monitoring method provided by the embodiment of the invention, the vehicle-mounted monitoring equipment is used for acquiring the multi-physical-domain information acquired by the distributed sensor, and then safety detection fusion data are generated based on the multi-physical-domain information and uploaded to the server; and inputting the safety detection fusion data into an expert system through a server, and outputting a corresponding safety rescue decision. By implementing the invention, the multi-physical domain information is detected and fused, the acquired monitoring data is more comprehensive and accurate, the passenger safety detection effect is improved, and the independent decision is made by the expert system, so that effective indication can be provided for the rescue of the trapped passenger.
Second embodiment:
in order to solve the technical problem that the passenger safety monitoring effect of the passenger safety monitoring system proposed for the automobile false lock in the prior art is not good, the present embodiment shows a passenger safety monitoring device, which is applied to an on-vehicle monitoring device, and specifically please refer to fig. 4, the passenger safety monitoring device of the present embodiment includes:
the acquisition module 401 is configured to acquire multi-physical-domain information including vehicle usage information, vehicle environment parameters, and in-vehicle passenger perception data acquired by a distributed sensor;
a determining module 402, configured to determine a type of a trapped passenger based on the in-vehicle passenger perception data and determine an in-vehicle environment evaluation result based on the vehicle environment parameter when the vehicle is determined to be in a locked state based on the vehicle usage information;
a generating module 403, configured to generate safety monitoring fusion data based on the type of the trapped passenger and the evaluation result of the environment inside the vehicle;
and an uploading module 404, configured to upload the safety monitoring fusion data to the server based on the communication connection with the server.
In some embodiments of this embodiment, if the in-vehicle passenger perception data is an in-vehicle passenger image, the determining module 402 is specifically configured to, when determining the type of the trapped passenger based on the in-vehicle passenger perception data: extracting an interested region of the passenger image in the vehicle to obtain an interested region characteristic diagram; inputting the characteristic diagram of the region of interest into the trained character recognition model; the type of trapped passenger is determined based on the classification label output by the character recognition model.
In some embodiments of the present embodiment, the determining module 402, when determining the evaluation result of the environment in the vehicle based on the vehicle environment parameter, is specifically configured to: determining a corresponding environment risk index based on the vehicle environment parameter and a preset mapping relation between the vehicle environment parameter and the environment risk index; and generating an in-vehicle environment evaluation result based on the environmental risk index.
Correspondingly, the present embodiment further provides a passenger safety monitoring device applied to the server side, as shown in fig. 5, the passenger safety monitoring device mainly includes:
the receiving module 501 is configured to receive safety monitoring fusion data uploaded by the vehicle-mounted monitoring device based on communication connection with the vehicle-mounted monitoring device; the safety monitoring fusion data is generated based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
an input module 502, configured to input the safety monitoring fusion data to a preset expert system; the expert system comprises a knowledge base and an inference machine;
and a planning module 503, configured to control the inference engine to invoke the knowledge base and plan a safety rescue decision corresponding to the safety monitoring fusion data.
In some embodiments of this embodiment, the passenger safety monitoring device further comprises: and the sending module is used for sending the safety rescue decision to the vehicle owner communication terminal based on the communication connection with the vehicle owner communication terminal after planning the safety rescue decision corresponding to the safety monitoring fusion data.
Further, in some embodiments of this embodiment, the sending module is further configured to: after the safety rescue decision is sent to the vehicle owner communication terminal, judging whether a decision acquisition response sent by the vehicle owner communication terminal is received within a preset time length; when a decision-making learning response is not received, generating alarm information based on a safety rescue decision; and sending the alarm information to an alarm platform.
It should be noted that, the passenger safety monitoring methods in the foregoing embodiments can be implemented based on the passenger safety monitoring device provided in this embodiment, and it can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the passenger safety monitoring device described in this embodiment may refer to the corresponding process in the foregoing method embodiments, and details are not described herein again.
By adopting the passenger safety monitoring device provided by the embodiment, the multi-physical-domain information acquired by the distributed sensor is acquired through the vehicle-mounted monitoring equipment, and then safety detection fusion data are generated based on the multi-physical-domain information and uploaded to the server; and inputting the safety detection fusion data into an expert system through a server, and outputting a corresponding safety rescue decision. By implementing the invention, the multi-physical domain information is detected and fused, the acquired monitoring data is more comprehensive and accurate, the passenger safety detection effect is improved, and the independent decision is made by the expert system, so that effective indication can be provided for the rescue of the trapped passenger.
The third embodiment:
the present embodiment provides an electronic device, as shown in fig. 6, which includes a processor 601, a memory 602, and a communication bus 603, wherein: the communication bus 603 is used for realizing connection communication between the processor 601 and the memory 602; the processor 601 is configured to execute one or more computer programs stored in the memory 602 to implement at least one step of the passenger safety monitoring method in the first embodiment.
The present embodiments also provide a computer-readable storage medium including volatile or non-volatile, removable or non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, computer program modules or other data. Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash Memory or other Memory technology, CD-ROM (Compact disk Read-Only Memory), Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer.
The computer-readable storage medium in this embodiment may be used for storing one or more computer programs, and the stored one or more computer programs may be executed by a processor to implement at least one step of the method in the first embodiment.
The present embodiment also provides a computer program, which can be distributed on a computer readable medium and executed by a computing device to implement at least one step of the method in the first embodiment; and in some cases at least one of the steps shown or described may be performed in an order different than that described in the embodiments above.
The present embodiments also provide a computer program product comprising a computer readable means on which a computer program as shown above is stored. The computer readable means in this embodiment may include a computer readable storage medium as shown above.
It will be apparent to those skilled in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software (which may be implemented in computer program code executable by a computing device), firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit.
In addition, communication media typically embodies computer readable instructions, data structures, computer program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to one of ordinary skill in the art. Thus, the present invention is not limited to any specific combination of hardware and software.
The foregoing is a more detailed description of embodiments of the present invention, and the present invention is not to be considered limited to such descriptions. For those skilled in the art to which the invention pertains, several simple deductions or substitutions can be made without departing from the spirit of the invention, and all shall be considered as belonging to the protection scope of the invention.
Claims (10)
1. A passenger safety monitoring method is applied to vehicle-mounted monitoring equipment and is characterized by comprising the following steps:
acquiring multi-physical-domain information which is acquired by a distributed sensor and comprises vehicle use information, vehicle environment parameters and in-vehicle passenger perception data;
determining a type of a trapped passenger based on the in-vehicle passenger perception data and determining an in-vehicle environment assessment result based on the vehicle environment parameter when it is determined that the vehicle is in a locked state based on the vehicle usage information;
generating safety monitoring fusion data based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
and uploading the safety monitoring fusion data to a server based on communication connection with the server.
2. The passenger safety monitoring method according to claim 1, wherein the in-vehicle passenger perception data is an in-vehicle passenger image, and the determining the type of the trapped passenger based on the in-vehicle passenger perception data includes:
extracting an interested region of the passenger image in the vehicle to obtain an interested region characteristic diagram;
inputting the characteristic diagram of the region of interest into a trained character recognition model;
determining a type of trapped passenger based on the classification label output by the character recognition model.
3. The passenger safety monitoring method according to claim 1 or 2, wherein the determining of the in-vehicle environment evaluation result based on the vehicle environment parameter includes:
determining a corresponding environment risk index based on the vehicle environment parameter and a preset mapping relation between the vehicle environment parameter and the environment risk index;
and generating an in-vehicle environment evaluation result based on the environmental risk index.
4. A passenger safety monitoring method is applied to a server and is characterized by comprising the following steps:
receiving safety monitoring fusion data uploaded by vehicle-mounted monitoring equipment; wherein the safety monitoring fusion data is generated based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
inputting the safety monitoring fusion data into a preset expert system; wherein the expert system comprises a knowledge base and an inference engine;
and controlling the inference engine to call the knowledge base and plan a safety rescue decision corresponding to the safety monitoring fusion data.
5. The passenger safety monitoring method according to claim 4, wherein after planning a safety rescue decision corresponding to the safety monitoring fused data, further comprising:
and sending the safety rescue decision to the vehicle owner communication terminal based on the communication connection with the vehicle owner communication terminal.
6. The passenger safety monitoring method according to claim 5, wherein after the sending the safety rescue decision to the owner communication terminal, further comprising:
judging whether a decision acquisition response sent by the vehicle owner communication terminal is received within a preset time length;
when the decision-making learning response is not received, generating alarm information based on the safety rescue decision;
and sending the alarm information to an alarm platform.
7. The utility model provides a passenger safety monitoring device, is applied to on-vehicle monitoring facilities, its characterized in that includes:
the acquisition module is used for acquiring multi-physical-domain information which is acquired by the distributed sensors and comprises vehicle use information, vehicle environment parameters and in-vehicle passenger perception data;
the determining module is used for determining the type of the trapped passenger based on the in-vehicle passenger perception data and determining the in-vehicle environment evaluation result based on the vehicle environment parameter when the vehicle is determined to be in the locked state based on the vehicle use information;
the generating module is used for generating safety monitoring fusion data based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
and the uploading module is used for uploading the safety monitoring fusion data to the server based on the communication connection with the server.
8. A passenger safety monitoring device is applied to a server and is characterized by comprising:
the receiving module is used for receiving the safety monitoring fusion data uploaded by the vehicle-mounted monitoring equipment; wherein the safety monitoring fusion data is generated based on the type of the trapped passenger and the evaluation result of the environment in the vehicle;
the input module is used for inputting the safety monitoring fusion data to a preset expert system; wherein the expert system comprises a knowledge base and an inference engine;
and the planning module is used for controlling the inference machine to call the knowledge base and planning a safety rescue decision corresponding to the safety monitoring fusion data.
9. An electronic device, comprising: a processor, a memory, and a communication bus;
the communication bus is used for realizing connection communication between the processor and the memory;
the processor is configured to execute a first computer program and a second computer program stored in the memory, the processor implementing the steps of the method according to any one of claims 1 to 3 when executing the first computer program, and implementing the steps of the method according to any one of claims 4 to 6 when executing the second computer program.
10. A computer-readable storage medium, characterized in that it stores a first computer program which, when executed by a processor, carries out the steps of the method according to any one of claims 1 to 3, and a second computer program which, when executed by a processor, carries out the steps of the method according to any one of claims 4 to 6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010453315.7A CN111626905A (en) | 2020-05-26 | 2020-05-26 | Passenger safety monitoring method and device and computer readable storage medium |
| PCT/CN2020/124375 WO2021238046A1 (en) | 2020-05-26 | 2020-10-28 | Passenger safety monitoring method and apparatus, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010453315.7A CN111626905A (en) | 2020-05-26 | 2020-05-26 | Passenger safety monitoring method and device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111626905A true CN111626905A (en) | 2020-09-04 |
Family
ID=72260710
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010453315.7A Pending CN111626905A (en) | 2020-05-26 | 2020-05-26 | Passenger safety monitoring method and device and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111626905A (en) |
| WO (1) | WO2021238046A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021238046A1 (en) * | 2020-05-26 | 2021-12-02 | 深圳技术大学 | Passenger safety monitoring method and apparatus, and computer-readable storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115996236B (en) * | 2023-03-23 | 2023-09-01 | 广东海新智能厨房股份有限公司 | Safety monitoring method, device, equipment and medium for gas stove |
| CN116321524B (en) * | 2023-04-11 | 2023-10-24 | 广州爱浦路网络技术有限公司 | Environment monitoring data processing method and device |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101083019A (en) * | 2006-12-31 | 2007-12-05 | 中国人民解放军63791部队 | Rapid evaluating system based on roomage state sensing |
| US7667609B1 (en) * | 2005-06-02 | 2010-02-23 | Kevin Roe | Expert system rescue of impaired equipment operators |
| CN201689455U (en) * | 2009-12-01 | 2010-12-29 | 北京市市政工程研究院 | Tunnel construction disaster planning and preventing response system based on network platform |
| US8044772B1 (en) * | 2005-06-10 | 2011-10-25 | Kevin Roe | Expert system assistance for persons in danger |
| CN102572156A (en) * | 2012-01-11 | 2012-07-11 | 南京航空航天大学 | Wireless sensor network-based onboard system and method for giving alarm for child |
| CN202472841U (en) * | 2011-12-19 | 2012-10-03 | 南京农业大学 | Forest fire monitoring and early warning system based on IOT |
| CN103330554A (en) * | 2013-07-18 | 2013-10-02 | 梁亚楠 | Wearable artificial intelligence wireless Internet of Things security system |
| CN105389944A (en) * | 2015-12-03 | 2016-03-09 | 长春工业大学 | Early warning and automatic rescue system for kid detained in car |
| CN105550824A (en) * | 2016-01-13 | 2016-05-04 | 天津中科智能识别产业技术研究院有限公司 | Intelligent disaster situation evaluation system |
| CN108257249A (en) * | 2017-12-29 | 2018-07-06 | 广州视声光电有限公司 | A kind of assessment of risks method and automobile data recorder |
| CN109292566A (en) * | 2017-07-24 | 2019-02-01 | 上海峰景移动科技有限公司 | A kind of automatic detecting system that elevator carriage is oppressive and logic |
| CN109345770A (en) * | 2018-11-14 | 2019-02-15 | 深圳市尼欧科技有限公司 | A kind of child leaves in-vehicle alarm system and child leaves interior alarm method |
| CN109955818A (en) * | 2017-12-22 | 2019-07-02 | 意法半导体股份有限公司 | Safe electronic device for the presence detection in vehicle |
| CN110329869A (en) * | 2019-08-08 | 2019-10-15 | 湖南高福星智能科技有限公司 | A kind of elevator safety monitoring system based on artificial intelligence |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104228768B (en) * | 2014-07-24 | 2016-10-05 | 华南理工大学 | Antitheft and error preventing lock warning system and method for work thereof in the car of multiple features fusion |
| US10077582B2 (en) * | 2016-06-22 | 2018-09-18 | Mark Ring | Automatic child safety lock release |
| CN109017658B (en) * | 2017-06-08 | 2021-09-03 | 浙江师范大学 | System for preventing children from being locked in vehicle |
| CN109300276A (en) * | 2018-07-27 | 2019-02-01 | 昆明理工大学 | A kind of car inside abnormity early warning method based on Fusion |
| CN111626905A (en) * | 2020-05-26 | 2020-09-04 | 深圳技术大学 | Passenger safety monitoring method and device and computer readable storage medium |
-
2020
- 2020-05-26 CN CN202010453315.7A patent/CN111626905A/en active Pending
- 2020-10-28 WO PCT/CN2020/124375 patent/WO2021238046A1/en active Application Filing
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7667609B1 (en) * | 2005-06-02 | 2010-02-23 | Kevin Roe | Expert system rescue of impaired equipment operators |
| US8044772B1 (en) * | 2005-06-10 | 2011-10-25 | Kevin Roe | Expert system assistance for persons in danger |
| CN101083019A (en) * | 2006-12-31 | 2007-12-05 | 中国人民解放军63791部队 | Rapid evaluating system based on roomage state sensing |
| CN201689455U (en) * | 2009-12-01 | 2010-12-29 | 北京市市政工程研究院 | Tunnel construction disaster planning and preventing response system based on network platform |
| CN202472841U (en) * | 2011-12-19 | 2012-10-03 | 南京农业大学 | Forest fire monitoring and early warning system based on IOT |
| CN102572156A (en) * | 2012-01-11 | 2012-07-11 | 南京航空航天大学 | Wireless sensor network-based onboard system and method for giving alarm for child |
| CN103330554A (en) * | 2013-07-18 | 2013-10-02 | 梁亚楠 | Wearable artificial intelligence wireless Internet of Things security system |
| CN105389944A (en) * | 2015-12-03 | 2016-03-09 | 长春工业大学 | Early warning and automatic rescue system for kid detained in car |
| CN105550824A (en) * | 2016-01-13 | 2016-05-04 | 天津中科智能识别产业技术研究院有限公司 | Intelligent disaster situation evaluation system |
| CN109292566A (en) * | 2017-07-24 | 2019-02-01 | 上海峰景移动科技有限公司 | A kind of automatic detecting system that elevator carriage is oppressive and logic |
| CN109955818A (en) * | 2017-12-22 | 2019-07-02 | 意法半导体股份有限公司 | Safe electronic device for the presence detection in vehicle |
| CN108257249A (en) * | 2017-12-29 | 2018-07-06 | 广州视声光电有限公司 | A kind of assessment of risks method and automobile data recorder |
| CN109345770A (en) * | 2018-11-14 | 2019-02-15 | 深圳市尼欧科技有限公司 | A kind of child leaves in-vehicle alarm system and child leaves interior alarm method |
| CN110329869A (en) * | 2019-08-08 | 2019-10-15 | 湖南高福星智能科技有限公司 | A kind of elevator safety monitoring system based on artificial intelligence |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021238046A1 (en) * | 2020-05-26 | 2021-12-02 | 深圳技术大学 | Passenger safety monitoring method and apparatus, and computer-readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2021238046A1 (en) | 2021-12-02 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11375338B2 (en) | Method for smartphone-based accident detection | |
| Aljaafreh et al. | Driving style recognition using fuzzy logic | |
| CN104859662B (en) | Troubleshooting in autonomous vehicle | |
| CN111626905A (en) | Passenger safety monitoring method and device and computer readable storage medium | |
| Chan et al. | A comprehensive review of driver behavior analysis utilizing smartphones | |
| CN110753934B (en) | System and method for actively selecting and tagging images for semantic segmentation | |
| US9346400B2 (en) | Affective user interface in an autonomous vehicle | |
| CN106114515B (en) | Car steering behavior based reminding method and system | |
| US9406177B2 (en) | Fault handling in an autonomous vehicle | |
| CN110371132B (en) | Driver takeover evaluation method and device | |
| US20150066284A1 (en) | Autonomous vehicle control for impaired driver | |
| US10997430B1 (en) | Dangerous driver detection and response system | |
| KR20200078274A (en) | Method and system for evaluating safety operation index using vehicle driving information collection device | |
| US20170028991A1 (en) | System And Method For Estimating The Driving Style Of A Vehicle | |
| CN109360417B (en) | Dangerous driving behavior identification and pushing method and system based on block chain | |
| US20180222494A1 (en) | Enhanced curve negotiation | |
| WO2019177511A1 (en) | Method, control arrangement and machine learning based system for autonomous vehicles for proactively acting on situations involving an increased traffic accident risk | |
| CN110782200A (en) | Intelligent management system and method for logistics vehicles | |
| CN114103988B (en) | Safety monitoring device, vehicle comprising same, and corresponding method, device and medium | |
| CN113415292A (en) | Driving takeover capability evaluation method and electronic device | |
| CN113450567A (en) | Artificial intelligence early warning system | |
| FR3073191A1 (en) | METHOD AND DEVICE FOR PREVENTING ROAD RISKS | |
| Aljaafreh | Web driving performance monitoring system | |
| Muthumanickam et al. | Vehicle health monitoring and accident avoidance system based on IoT model | |
| CN106909075A (en) | A kind of method that utilization gps data analyzes driving behavior |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Cheng Tao Inventor after: Cai Yongwei Inventor before: Cai Yongwei Inventor before: Cheng Tao |