CN115805956A

CN115805956A - Danger prompting method and device, vehicle and storage medium

Info

Publication number: CN115805956A
Application number: CN202211260092.8A
Authority: CN
Inventors: 侯莹
Original assignee: Great Wall Motor Co Ltd
Current assignee: Great Wall Motor Co Ltd
Priority date: 2022-10-14
Filing date: 2022-10-14
Publication date: 2023-03-17

Abstract

The application discloses a danger prompting method and device, a vehicle and a storage medium, and belongs to the technical field of vehicles. Through the technical scheme provided by the embodiment of the application, the vehicle-mounted terminal can directly identify the passengers in the intelligent cabin to obtain the passenger information of the passengers. The vehicle-mounted terminal can obtain the risk state of the vehicle riding on the vehicle based on the running information of the vehicle, the passenger information and the vehicle information of the vehicle. The vehicle-mounted terminal can timely prompt danger based on the risk state so that the risk of a passenger or a driver can be timely eliminated, and the safety of the passenger taking the vehicle is improved.

Description

Danger prompting method and device, vehicle and storage medium

Technical Field

The present application relates to the field of vehicle technologies, and in particular, to a method and an apparatus for danger notification, a vehicle, and a storage medium.

Background

With the development of vehicle technology, the design and development of intelligent cabins are receiving more and more attention from automobile manufacturers. The intelligent cabin is a digital platform which is updated and modified in the vehicle, and is an intelligent product matched with various sensors.

In the related art, attention is often paid to smart cabins to improve driving feeling of drivers, riding comfort of passengers, entertainment functions, and the like. However, it is more important how to improve the safety of passengers riding in the vehicle in the smart cabin than the above improvements.

Disclosure of Invention

The embodiment of the application provides a danger prompting method, a danger prompting device, a vehicle and a storage medium, which can improve the safety of passengers taking the vehicle, and the technical scheme is as follows:

in one aspect, a danger prompting method is provided, and the method includes:

identifying passengers in the intelligent cabin to obtain passenger information of the passengers;

determining a risk state of the passenger riding in the vehicle based on traveling information of the vehicle, the traveling information representing a traveling state of the vehicle, and at least one of passenger information of the passenger and vehicle information of the vehicle, the vehicle information representing states of a plurality of functional components in the vehicle;

and carrying out danger prompt based on the risk state of the passenger, wherein the danger prompt is used for prompting that the passenger possibly has danger.

In a possible implementation, the identifying, by the motion recognition model, based on image features of the passenger image, to obtain the motion of the passenger in the intelligent cabin includes:

fully connecting and normalizing the image characteristics of the passenger image through the action recognition model, and outputting the probability of the passenger image corresponding to a plurality of candidate actions;

determining a candidate action with a highest probability among the plurality of candidate actions as the action of the passenger.

In a possible embodiment, the identifying the position of the passenger by the sensor on the seat of the intelligent cabin, and the obtaining the position of the passenger in the intelligent cabin includes any one of the following:

in the case that any sensor on the seat of the intelligent cabin detects a passenger, determining the position of the passenger as the position of the sensor on the seat of the intelligent cabin;

determining that a passenger is not seated in a seat of the smart cabin in a situation where a sensor on the seat does not detect the passenger.

In one aspect, a danger prompting device is provided, the device comprising:

the identification module is used for identifying passengers in the intelligent cabin to obtain passenger information of the passengers;

a risk state determination module configured to determine a risk state of the passenger riding the vehicle based on travel information of the vehicle, the travel information being indicative of a travel state of the vehicle, and at least one of passenger information of the passenger and vehicle information of the vehicle, the vehicle information being indicative of states of a plurality of functional components in the vehicle;

and the danger prompt module is used for carrying out danger prompt based on the risk state of the passenger, and the danger prompt is used for prompting that the passenger possibly has danger.

In a possible implementation manner, the identification module is used for performing action identification on passengers in the intelligent cabin to obtain actions of the passengers in the intelligent cabin; carrying out position identification on passengers in the intelligent cabin to obtain the positions of the passengers in the intelligent cabin; the passenger information includes an action and a position of the passenger.

In a possible implementation manner, the recognition module is configured to input the passenger images collected in the smart cabin into a motion recognition model, and perform feature extraction on the passenger images through the motion recognition model to obtain image features of the passenger images; and identifying based on the image characteristics of the passenger image through the action identification model to obtain the action of the passenger in the intelligent cabin.

In a possible implementation manner, the recognition module is configured to perform full connection and normalization on image features of the passenger image through the motion recognition model, and output probabilities that the passenger image corresponds to a plurality of candidate motions; determining a candidate action with a highest probability among the plurality of candidate actions as the action of the passenger.

In a possible implementation, the identification module is configured to perform any one of:

carrying out position identification on the passenger through a sensor on a seat of the intelligent cabin to obtain the position of the passenger in the intelligent cabin;

and identifying based on the passenger image through a position identification model to obtain the position of the passenger in the intelligent cabin.

In a possible implementation, the identification module is configured to perform any one of the following:

In a possible implementation, the risk status determination module is configured to perform any one of:

determining a risk state of the passenger riding the vehicle as a first candidate risk state when the driving information indicates that the vehicle is running straight or turning, and the passenger information indicates that the passenger moves to any one of the positions of extending a hand out of a vehicle window, detecting a body out of the vehicle window and throwing a thing out of the vehicle window, wherein the first candidate risk state is used for representing that the passenger executes a dangerous motion;

determining a risk state of the passenger riding the vehicle as a second candidate risk state, wherein the second candidate risk state is used for representing that the passenger is out of seat, and the passenger information of the passenger indicates that the passenger is located in the intelligent cabin and is not seated on a seat of the intelligent cabin;

determining that a risk state of the passenger taking the vehicle is a third candidate risk state, wherein the third candidate risk state is used for representing that the passenger possibly leaves the vehicle, under the condition that the running information of the vehicle is turning, the vehicle information of the vehicle indicates that the opening degree of the window of the vehicle is larger than a preset opening degree threshold value, and the passenger information of the passenger indicates that the passenger is positioned in the intelligent cabin;

and under the condition that the running information of the vehicle is straight running or turning, the vehicle information of the vehicle indicates that the door of the vehicle is not opened in the current running period, and the passenger information of the passenger indicates that the passenger is not positioned in the intelligent cabin, determining that the risk state corresponding to taking the vehicle is a fourth candidate risk state, wherein the fourth candidate risk state is used for indicating that the passenger is separated from the vehicle.

In one possible embodiment, the risk state determination module is configured to input the driving information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle into a risk state determination model, predict based on the driving information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle by the risk state determination model, and output the risk state of the passenger riding the vehicle.

In a possible embodiment, the risk state determining module is configured to predict, by the risk state determination model, driving information of the vehicle and at least one of passenger information of the passenger and vehicle information of the vehicle, and obtain probabilities of a plurality of candidate risk states corresponding to the passenger taking the vehicle; determining a candidate risk state with a highest probability among the plurality of candidate risk states as a risk state of the passenger riding the vehicle.

In one possible embodiment, the danger prompting module is configured to perform at least one of:

playing a danger prompting voice based on the type of the risk state of the passenger;

displaying a danger prompt text based on the type of risk status of the passenger;

and executing a danger prompting action based on the type of the risk state of the passenger, wherein the danger prompting action is any one of seat vibration, steering wheel vibration and instrument panel flicker.

In one aspect, a vehicle is provided, the vehicle including a vehicle-mounted terminal, the vehicle-mounted terminal including one or more processors and one or more memories, at least one computer program being stored in the one or more memories, the computer program being loaded and executed by the one or more processors to implement the hazard prompting method.

In one aspect, a computer-readable storage medium is provided, in which at least one computer program is stored, the computer program being loaded and executed by a processor to implement the hazard prompting method.

In one aspect, a computer program product or a computer program is provided, the computer program product or the computer program comprising program code stored in a computer-readable storage medium, the program code being read by a processor of a computer device from the computer-readable storage medium, the program code being executed by the processor such that the computer device performs the above-mentioned hazard notification method.

Through the technical scheme provided by the embodiment of the application, the vehicle-mounted terminal can directly identify the passengers in the intelligent cabin to obtain the passenger information of the passengers. The vehicle-mounted terminal can obtain the risk state of the vehicle riding on the vehicle based on the running information of the vehicle, the passenger information and the vehicle information of the vehicle. The vehicle-mounted terminal can timely prompt danger based on the risk state so that the risk of a passenger or a driver can be timely eliminated, and the safety of the passenger taking the vehicle is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic diagram of an implementation environment of a danger indicating method according to an embodiment of the present application;

fig. 2 is a flowchart of a danger indication method according to an embodiment of the present application;

fig. 3 is a flowchart of another danger indicating method provided in the embodiment of the present application;

fig. 4 is a schematic structural diagram of a danger indicating device according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of an in-vehicle terminal according to an embodiment of the present application.

Detailed Description

To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.

The terms "first," "second," and the like in this application are used for distinguishing between similar items and items that have substantially the same function or similar functionality, and it should be understood that "first," "second," and "nth" do not have any logical or temporal dependency or limitation on the number or order of execution.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the implementation method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

Cloud Computing (Cloud Computing) refers to a mode of delivery and use of IT (Internet Technology) infrastructure, and refers to obtaining required resources through a network in an on-demand, easily extensible manner; the generalized cloud computing refers to a delivery and use mode of a service, and refers to obtaining a required service in an on-demand and easily-extensible manner through a network. Such services may be IT and software, internet related, or other services. Cloud Computing is a product of development and fusion of traditional computers and Network Technologies, such as Grid Computing (Grid Computing), distributed Computing (Distributed Computing), parallel Computing (Parallel Computing), utility Computing (Utility Computing), network Storage (Network Storage Technologies), virtualization (Virtualization), load balancing (Load Balance), and the like.

With the development of diversification of internet, real-time data stream and connecting equipment and the promotion of demands of search service, social network, mobile commerce, open collaboration and the like, cloud computing is rapidly developed. Different from the prior parallel distributed computing, the generation of cloud computing can promote the revolutionary change of the whole internet mode and the enterprise management mode in concept.

The intelligent cabin: the intelligent cabin is an updated and modified digital platform in the vehicle, the traditional vehicle cabin can be only used for marking various driving working conditions, and the key characteristics of the intelligent cabin are reflected on an intelligent two-character. In the cockpit, a plurality of displays are modified, and the operation steps are changed from the traditional button actual operation to touch or voice chat actual operation. Meanwhile, various sensors and AI intelligent products are also matched, so that more comfortable driving feeling can be given in consideration of habitual and comfortable feelings of drivers. The intelligent cabin has the characteristic that the intelligent cabin is closely related to the daily life game and entertainment of the driver. A plurality of design schemes for relieving stuffiness in game and entertainment are arranged in the cockpit, so that a driver can watch TV plays, KTVs, play games, chat videos and the like in the automobile. In addition, various external information contents such as air temperature, arrival conditions, automobile charging pile positions, driving time and traffic road conditions can be ensured, the physical and mental health conditions of a driver can be recognized according to facilities, and the extracellular fluid of the automobile can be adjusted.

Normalization: and the arrays with different value ranges are mapped to the (0, 1) interval, so that the data processing is facilitated. In some cases, the normalized values may be directly implemented as probabilities.

Embedded Coding (Embedded Coding): the embedded code mathematically represents a correspondence,that is, mapping the data in X space to Y space by a function F, wherein the function F is a single-shot function, the mapping result is structure preservation, the single-shot function represents that the mapped data is uniquely corresponding to the data before mapping, and the structure preservation represents that the size relationship of the data before mapping is the same after the size relationship of the data after mapping, for example, the data X exists before mapping ₁ And X ₂ Mapping to obtain X ₁ Corresponding Y ₁ And X ₂ Corresponding Y ₂ . If data X before mapping ₁ ＞X ₂ Then correspondingly, the mapped data Y ₁ Greater than Y ₂ . For words, the words are mapped to another space, so that subsequent machine learning and processing are facilitated.

It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals referred to in this application are authorized by the user or fully authorized by various parties, and the collection, use and processing of the relevant data are subject to relevant laws and regulations and standards in relevant countries and regions.

Fig. 1 is a schematic diagram of an implementation environment of a danger indication method according to an embodiment of the present application, and referring to fig. 1, the implementation environment may include a vehicle-mounted terminal 110 and a server 140.

The in-vehicle terminal 110 is connected to the server 140 through a wireless network, and the in-vehicle terminal 110 includes a seat on which a passenger can sit. The in-vehicle terminal 110 includes various types of sensors through which various information inside and outside the in-vehicle terminal 110 can be obtained, and accordingly, the in-vehicle terminal 110 further includes a processor for storing information collected by the sensors and a memory for processing the information stored in the memory. The in-vehicle terminal 110 is installed and operated with a supported danger prompting application.

The server 140 is an independent physical server, or a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, middleware service, a domain name service, a security service, a Delivery Network (CDN), a big data and artificial intelligence platform, and the like.

After the implementation environment provided by the embodiment of the present application is introduced, an application scenario of the embodiment of the present application is described below, and the technical solution provided by the embodiment of the present application can be applied to any vehicle with a vehicle-mounted terminal. The in-vehicle terminal determines a risk state of the passenger riding in the vehicle based on the form information of the vehicle and at least one of the passenger information and the vehicle information. The vehicle-mounted terminal can prompt dangers based on the risk state, so that passengers and drivers are reminded of the danger possibly existing, the passengers and the drivers can intervene in time, and the danger is avoided.

After the implementation environment and the application scenario of the embodiment of the present application are introduced, a technical solution provided by the embodiment of the present application is described below, referring to fig. 2, taking a vehicle-mounted terminal whose execution subject is a vehicle as an example, and the method includes the following steps.

201. The vehicle-mounted terminal identifies passengers in the intelligent cabin to obtain passenger information of the passengers.

The passenger identification comprises action identification and position identification, and the action identification result is the action performed by the passenger in the intelligent cabin, for example, the action performed by the passenger in the intelligent cabin is the action performed by the passenger in the intelligent cabin when the passenger extends the hand out of the window; the result of the position recognition is the position of the passenger in the intelligent cabin, such as whether the passenger is in the intelligent cabin, or whether the passenger is sitting on a seat of the intelligent cabin and on which seat of the intelligent cabin the passenger is sitting. The passenger information of the passenger can be the state of the passenger in the intelligent cabin.

202. The in-vehicle terminal determines a risk state of the passenger riding in the vehicle based on traveling information of the vehicle, the traveling information indicating a traveling state of the vehicle, and at least one of passenger information of the passenger and vehicle information of the vehicle, the vehicle information indicating states of a plurality of functional components in the vehicle.

The driving state of the vehicle includes, among others, stationary, straight traveling, turning, climbing, and descending. The functional components in the vehicle include components such as a window, a door, and a sunroof, and accordingly, the states of the functional components include an open/close state of the window, an open/close state of the door, a locked state of the door, an open/close state of the sunroof, and the like. The risk status of the passenger riding the vehicle is used to indicate the risk of the passenger riding the vehicle.

203. And the vehicle-mounted terminal carries out danger prompt based on the risk state of the passenger, and the danger prompt is used for prompting that the passenger possibly has danger.

The above steps 201 to 203 are brief descriptions of the technical solutions provided in the embodiments of the present application, and the technical solutions provided in the embodiments of the present application will be more clearly described below with reference to some examples, and referring to fig. 3, taking an example of an in-vehicle terminal whose executing body is a vehicle, the method includes the following steps.

301. And the vehicle-mounted terminal identifies the actions of the passengers in the intelligent passenger cabin to obtain the actions of the passengers.

Wherein the result of the action recognition is an action performed by the passenger in the smart cabin, such as an action performed by the passenger in the smart cabin to extend the hand out of the window is an action performed by the passenger in the smart cabin, an action performed by the passenger in the smart cabin to extend the head out of the window is an action performed by the passenger in the smart cabin, and an action performed by the passenger in the smart cabin to throw the head out of the window is an action performed by the passenger in the smart cabin. After the action of the passenger in the intelligent cabin is identified, the risk state of the passenger taking the vehicle can be identified in a follow-up mode by combining with other information, so that the risk is eliminated in time, and the riding safety of the passenger is ensured.

In one possible implementation mode, the vehicle-mounted terminal inputs passenger images collected in the intelligent cabin into a motion recognition model, and feature extraction is carried out on the passenger images through the motion recognition model to obtain image features of the passenger images. And the vehicle-mounted terminal identifies based on the image characteristics of the passenger image through the action identification model to obtain the action of the passenger in the intelligent cabin.

Wherein the passenger images are collected in the intelligent cabin, such as through a camera installed in the intelligent cabin. The motion recognition model is a multi-classification model for recognizing the motion of an object in an image based on an input image, in the embodiment of the present application, the image is an image of a passenger, and the object is a passenger. In the process of training the motion recognition model, a sample passenger image is input into the motion recognition model, the motion recognition model recognizes the passenger image based on the sample passenger image, and a predicted motion corresponding to the sample passenger image is output. And performing a round of iterative training on the motion recognition model based on the difference between the corresponding labeled motion and the predicted motion of the sample passenger image. Through multiple rounds of iterative training, a trained motion recognition model is obtained, and in the following description, the motion recognition model refers to the trained motion recognition model.

The motion recognition model may be trained by the in-vehicle terminal or by the server, which is not limited in the embodiment of the present application.

Through the embodiment, the vehicle-mounted terminal can recognize the action of the passenger image by utilizing the action recognition model, so that the action of the passenger in the intelligent cabin is obtained, and the efficiency and the accuracy are high.

In order to more clearly describe the above embodiments, the above embodiments will be described in several parts.

The first part is that the vehicle-mounted terminal inputs passenger images collected in the intelligent cabin into an action recognition model.

The number of the passenger images may be one or multiple, which is not limited in the embodiment of the present application. When the number of passenger images is plural, the motion recognition model can recognize the motion by using the correlation between the plural images, and the accuracy of the motion recognition is high. When the number of the passenger images is one, the motion recognition model can quickly recognize the motion of the passenger, and the efficiency is high. In the embodiment of the present application, the number of the passenger images is described as one example.

And the second part is that the vehicle-mounted terminal performs feature extraction on the passenger image through the action recognition model to obtain the image features of the passenger image.

In one possible implementation, the vehicle-mounted terminal performs at least one convolution on the passenger image through the convolution layer of the motion recognition model to obtain the image characteristics of the passenger image.

For example, the in-vehicle terminal slides on the pixel matrix of the passenger image by using a plurality of convolution kernels through the convolution layer of the motion recognition model, and performs convolution operation in the sliding process to obtain the feature map corresponding to each convolution kernel. And the vehicle-mounted terminal fuses a plurality of feature maps corresponding to a plurality of convolution kernels through the action recognition model to obtain the image features of the passenger image. The size and the moving step size of the convolution kernel are set by a technician according to an actual situation, which is not limited in the embodiment of the present application.

In one possible implementation, the vehicle-mounted terminal performs at least one full connection on the passenger image through a full connection layer of the motion recognition model to obtain the image characteristics of the passenger image.

For example, the vehicle-mounted terminal multiplies a pixel matrix of the passenger image by at least one fully-connected matrix through a fully-connected layer of the motion recognition model, and adds the multiplied pixel matrix and at least one offset matrix to obtain the image characteristics of the passenger image.

In one possible embodiment, the in-vehicle terminal performs attention coding on the passenger image through an attention coding layer of the motion recognition model to obtain an image feature of the passenger image.

For example, the in-vehicle terminal performs Embedding (Embedding) encoding on a plurality of portions of a pixel matrix of the passenger image through an attention encoding layer of the motion recognition model, and obtains an Embedding characteristic of each portion of the pixel matrix. And the vehicle-mounted terminal encodes the embedded features of the plurality of parts based on the attention mechanism through the attention encoding layer of the action recognition model to obtain a query vector, a key vector and a value vector of each part. And the vehicle-mounted terminal fuses the query vectors, the key vectors and the value vectors of the plurality of parts through the attention coding layer of the action recognition model to obtain the image characteristics of the passenger image.

It should be noted that different feature extraction methods correspond to motion recognition models with different structures, and in this embodiment of the present application, any structure of motion recognition model may be used, for example, a neural network model, a deep learning model, a convolutional neural network, a residual neural network, and the like, which is not limited in this embodiment of the present application.

And the third part is that the vehicle-mounted terminal carries out recognition based on the image characteristics of the passenger image through the motion recognition model to obtain the motion of the passenger in the vehicle-mounted terminal.

In one possible implementation mode, the vehicle-mounted terminal performs full connection and normalization on the image characteristics of the passenger image through the action recognition model, and outputs the probability that the passenger image corresponds to a plurality of candidate actions. The vehicle-mounted terminal determines the candidate action with the highest probability in the plurality of candidate actions as the action of the passenger.

In some embodiments, the action recognition model output is in the form of a set of probabilities, the set of probabilities including a plurality of probabilities, each probability corresponding to a candidate action.

302. And the vehicle-mounted terminal carries out position identification on the passenger in the intelligent cabin to obtain the position of the passenger.

The result of the position recognition is the position of the passenger in the intelligent cabin, such as whether the passenger is in the intelligent cabin, or whether the passenger sits on a seat of the intelligent cabin and which seat of the intelligent cabin the passenger sits on.

In one possible implementation mode, the vehicle-mounted terminal conducts position recognition on the passenger through a sensor on a seat of the intelligent cabin, and the position of the passenger in the intelligent cabin is obtained.

For example, in the case where any one of the sensors on the seats of the smart cabin detects a passenger, the in-vehicle terminal determines the position of the passenger as the position of the sensor on the seat of the smart cabin. In the case where the sensor on the seat of the smart cabin does not detect a passenger, the in-vehicle terminal determines that the passenger is not seated on the seat.

In the embodiment, the vehicle-mounted terminal can identify the position of the passenger through the sensor on the seat, and the sensor has high identification efficiency and low cost.

In some embodiments, the sensor is a pressure sensor mounted in the smart cabin below a seat. In the case where the pressure value detected by any one of the pressure sensors is greater than a preset pressure threshold value, it is determined that the pressure sensor detects the passenger. The vehicle-mounted terminal determines the seat above the pressure sensor as the position of the passenger. And under the condition that the pressure values detected by the pressure sensors below the seats in the intelligent cabin are all smaller than or equal to the preset pressure threshold value, determining that the pressure sensors do not detect passengers. The in-vehicle terminal determines that the passenger is not in the seat.

In some embodiments, the sensor is a laser sensor installed above a seat in the intelligent cabin, and the laser sensor identifies the position of the passenger by sending laser to the position of the seat, that is, by detecting a time difference between a time point when the laser is sent and a time point when the reflected laser is received. And determining that the laser sensor detects the passenger in the case that the time difference detected by any one of the laser sensors is less than or equal to a preset time difference threshold value. The vehicle-mounted terminal determines a seat below the laser sensor as a position of the passenger. And under the condition that the time difference values detected by the laser sensors above the seats in the intelligent cabin are all larger than the preset time difference value threshold value, determining that the laser sensors do not detect passengers. The in-vehicle terminal determines that the passenger is not in the seat.

In one possible implementation mode, the vehicle-mounted terminal conducts recognition based on the passenger image through a position recognition model, and the position of the passenger in the intelligent cabin is obtained.

The position recognition model is used for recognizing the position of an object in an image based on an input image, in this embodiment, the image is an image of a passenger, and the object is a passenger. In the process of training the position recognition model, the sample passenger image is input into the position recognition model, the position recognition model carries out recognition based on the sample passenger image, and the predicted position corresponding to the sample passenger image is output. And performing a round of iterative training on the position recognition model based on the difference between the corresponding labeled position of the sample passenger image and the predicted position. Through multiple rounds of iterative training, a trained position recognition model is obtained, and in the following description, the position recognition model refers to the trained position recognition model.

Through the embodiment, the vehicle-mounted terminal can perform position recognition on the passenger image by using the position as the recognition model, so that the position of the passenger in the intelligent cabin is obtained, and the efficiency and the accuracy are high.

The position recognition model may be trained by the vehicle-mounted terminal or by the server, which is not limited in the embodiment of the present application. In addition, any model structure of the position recognition model in the above embodiments may be adopted, for example, a neural network model, a deep learning model, a convolutional neural network, a residual neural network, and the like, which is not limited in the examples of the present application.

For example, the vehicle-mounted terminal inputs passenger images collected in the intelligent cabin into the position recognition model. And the vehicle-mounted terminal performs feature extraction on the passenger image through the position identification model to obtain the image features of the passenger image. And the vehicle-mounted terminal carries out recognition based on the image characteristics of the passenger image through the position recognition model to obtain the position of the passenger in the intelligent cabin.

It should be noted that, the

steps

301 and 302 may be executed successively or simultaneously, and in a case that the

steps

301 and 302 are executed successively, the step 301 may be executed first and then the step 302 is executed, or the step 302 may be executed first and then the step 301 is executed, and the execution sequence and the execution timing of the

steps

301 and 302 are not limited in this embodiment of the application.

Optionally, before step 301, the vehicle-mounted terminal may further perform age identification on the passenger in the intelligent cabin, and when the age of the passenger is identified to be smaller than an age threshold, that is, when the passenger is a child, the vehicle-mounted terminal performs step 301, and the vehicle-mounted terminal may perform age identification in any manner, which is not limited in this embodiment of the present application.

303. The in-vehicle terminal determines a risk state of the passenger riding in the vehicle based on traveling information of the vehicle, the traveling information indicating a traveling state of the vehicle, and at least one of passenger information of the passenger and vehicle information of the vehicle, the vehicle information indicating states of a plurality of functional components in the vehicle, the passenger information including an action and a position of the passenger.

The driving state of the vehicle includes, among others, stationary, straight traveling, turning, climbing, and descending. The functional components in the vehicle include components such as a window, a door, and a sunroof, and accordingly, the states of the functional components include an open/close state of the window, an open/close state of the door, a locked state of the door, an open/close state of the sunroof, and the like. The risk status of the passenger riding the vehicle is used to indicate the risk of the passenger riding the vehicle. In some embodiments, the driving information of the vehicle is acquired by the vehicle-mounted terminal through a driving sensor of the vehicle, such as a tire sensor or a steering wheel sensor of the vehicle, wherein the tire sensor can detect a rotation angle of a tire and the steering wheel sensor can detect a rotation angle of a steering wheel. Accordingly, the vehicle information of the vehicle is acquired by the state sensors of the plurality of functional components in the vehicle, for example, the opening degree of the window can be determined by the window sensor for the window of the plurality of functional components, and the opening degree of the sunroof can be determined by the sunroof sensor for the sunroof of the plurality of functional components.

The above step 303 is explained below by four examples.

In example 1, when the travel information indicates that the vehicle is traveling straight or turning, and the passenger information indicates that the passenger's motion is one of a hand extending out of the window, a body extending out of the window, and a parabola extending out of the window, the in-vehicle terminal determines that the risk state of the passenger riding the vehicle is a first candidate risk state indicating that the passenger has performed a dangerous motion.

The first candidate risk state belongs to the plurality of candidate risk states, and the plurality of candidate risk states are set by a technician according to an actual situation, which is not limited in the embodiment of the present application.

Under the embodiment, the vehicle-mounted terminal can quickly identify that the passenger executes the dangerous action in the intelligent cabin, and then prompt is carried out on the basis of the first candidate risk state, so that the passenger is reminded to finish the dangerous action, and the safety of the passenger is guaranteed.

Example 2, in a case where the traveling information of the vehicle is straight or turning, and the passenger information of the passenger indicates that the passenger is located inside the smart cabin and is not seated in the seat of the smart cabin, the in-vehicle terminal determines that the risk state of the passenger riding the vehicle is a second candidate risk state indicating that the passenger is out of seat.

Wherein the second candidate risk state belongs to the plurality of candidate risk states. The passenger is out of the seat means that the passenger is not in the seat during the driving of the vehicle.

Under the embodiment, the vehicle-mounted terminal can quickly identify that the passenger leaves the seat in the intelligent cabin, so that the prompt is carried out based on the second candidate risk state subsequently, the passenger is reminded to keep on the seat, and the safety of the passenger is guaranteed.

Example 3, in a case where the traveling information of the vehicle is turning, the vehicle information of the vehicle indicates that the opening degree of the window of the vehicle is greater than a preset opening degree threshold, and the passenger information of the passenger indicates that the passenger is located inside the smart cabin, the in-vehicle terminal determines that the risk state of the passenger riding in the vehicle is a third candidate risk state indicating that the passenger may leave the vehicle.

The third candidate risk state belongs to the plurality of candidate risk states, and the preset opening degree threshold is set by a technician according to an actual situation, which is not limited in the embodiment of the present application. The passenger may be out of the vehicle indicating that the passenger may be out of the vehicle from the windows of the vehicle during a turn of the vehicle.

In the embodiment, the vehicle-mounted terminal can determine the opening degree of the window when the vehicle turns, and when the opening degree of the window is larger than the preset opening degree threshold value, the risk state of the passenger is determined to be the third risk state, which indicates that the passenger has the risk of leaving the vehicle, especially when the passenger is a child, the vehicle-mounted terminal can remind the driver to close the window or close the window to reduce the window, so that the child is prevented from leaving the window from the window, and the safety of the child when the child takes the vehicle is ensured.

Example 4, in the case that the driving information of the vehicle is straight or turning, the vehicle information of the vehicle indicates that the doors of the vehicle are not opened in the current driving cycle, and the passenger information of the passenger indicates that the passenger is not located inside the smart cabin, the in-vehicle terminal cabin determines that the risk state corresponding to the riding of the vehicle is a fourth candidate risk state, which is used to indicate that the passenger has departed from the vehicle.

Wherein the fourth candidate risk state belongs to the plurality of candidate risk states. The passenger having left the vehicle indicates that the passenger is not already inside the smart cabin.

Under the embodiment, the vehicle-mounted terminal can quickly identify that the passenger is not in the intelligent cabin, so that prompt is carried out subsequently based on the fourth candidate risk state to remind the passenger that the passenger is not in the vehicle, and the driver can take remedial measures in time.

In one possible embodiment, the in-vehicle terminal inputs the travel information of the vehicle and at least one of the occupant information and the vehicle information of the vehicle into a risk state determination model, predicts based on the travel information of the vehicle and at least one of the occupant information and the vehicle information of the vehicle by the risk state determination model, and outputs the risk state of the occupant taking the vehicle.

For example, the risk state determination model predicts the driving information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle to obtain the probabilities of the candidate risk states corresponding to the passenger taking the vehicle. And the vehicle-mounted terminal determines the candidate risk state with the highest probability in the candidate risk states as the risk state of the passenger taking the vehicle.

It should be noted that any model structure of the risk state determination model in the above embodiments may be adopted, for example, a neural network model, a deep learning model, a convolutional neural network, a residual neural network, and the like, which is not limited in the examples of the present application.

In some embodiments, in the case where the in-vehicle terminal determines the risk state of the passenger, the in-vehicle terminal can first make a determination based on the type of the risk state to determine whether to perform step 304 described below. For example, if the determined type of the risk state is the target type, the vehicle-mounted terminal performs the following step 304, and if the determined type of the risk state is not the target type, the vehicle-mounted terminal does not perform the following step 304, wherein the target type is set by a technician according to an actual situation, which is not limited in the embodiment of the present application.

304. And the vehicle-mounted terminal carries out danger prompt based on the risk state of the passenger, and the danger prompt is used for prompting that the passenger possibly has danger.

In one possible implementation, the vehicle-mounted terminal plays a danger prompting voice based on the type of the risk state of the passenger.

Wherein, this intelligence passenger cabin includes the speaker, and this intelligence passenger cabin can broadcast danger through this speaker and indicate pronunciation. The target type is set by a technician according to actual conditions, and the embodiment of the application is not limited to this.

Under this kind of embodiment, vehicle-mounted terminal can carry out danger suggestion through the mode of broadcast pronunciation, and passenger and driver all can receive danger suggestion fast to in time take corresponding measure and avoid dangerous the emergence, improve the security that the passenger took the vehicle.

For example, when the risk state is the first candidate risk state, the in-vehicle terminal plays a danger prompting voice through the speaker, where the danger prompting voice is used to prompt that execution of a dangerous action is stopped, that is, the danger prompting voice is used to prompt a passenger. For example, the danger-prompting voice is "body is extended out of the window, which is dangerous behavior", "stop performing dangerous action", and "body is not extended out of the window".

And under the condition that the risk state is a second candidate risk state, the vehicle-mounted terminal plays a danger prompting voice through a loudspeaker, wherein the danger prompting voice is used for prompting to be kept on the seat, namely the danger prompting voice is used for prompting the passenger. For example, the danger prompting voice is "you do not sit on the seat, which is dangerous behavior" and "please keep on the seat", and the like.

Under the condition that the risk state is the third candidate risk state, the vehicle-mounted terminal plays a danger prompting voice through the loudspeaker, the danger prompting voice is used for prompting to reduce the opening degree of the vehicle window, namely the danger prompting voice is used for prompting the driver, for example, the danger prompting voice is 'please drive carefully, the vehicle window moves upwards, and the rear row personnel are prevented from encountering danger'. Alternatively, the danger sound may be used to prompt the user to keep a sitting posture, that is, the danger sound may be used to prompt the user, for example, the danger sound is "please do you to avoid falling outside the vehicle".

And under the condition that the risk state is a fourth candidate risk state, the vehicle-mounted terminal plays a danger prompting voice through a loudspeaker, wherein the danger prompting voice is used for prompting that the passenger is separated from the vehicle, namely the danger prompting voice is used for prompting the driver, and for example, the danger prompting voice is that the passenger does not exist at the rear row in the current cabin.

In one possible implementation manner, in the case that the risk state is the risk state of the target type, the vehicle-mounted terminal displays a danger prompt text based on the risk state.

The intelligent cabin comprises a first display and a second display, the first display is used for reminding a driver, the second display is used for reminding passengers, and the intelligent cabin can display danger prompt texts through the first display or the second display.

Under the embodiment, the vehicle-mounted terminal can carry out danger prompt in a text display mode, and both passengers and drivers can quickly receive the danger prompt, so that corresponding measures can be taken timely to avoid danger, and the safety of the passengers taking the vehicle is improved.

For example, when the risk state is the first candidate risk state, the in-vehicle terminal displays a danger prompt text through the second display, and the danger prompt text is used for prompting that the execution of the dangerous action is stopped. For example, the danger prompt text is "stop performing dangerous action", "do not extend the body out of the window", and the like.

And under the condition that the risk state is a second candidate risk state, the vehicle-mounted terminal displays a danger prompt text through the second display, wherein the danger prompt text is used for prompting that the vehicle-mounted terminal is kept on the seat, namely the danger prompt text is used for prompting passengers. For example, the danger prompt text is "you do not sit on the seat, which is dangerous behavior", and "please keep on the seat", and the like.

And under the condition that the risk state is a third candidate risk state, the vehicle-mounted terminal displays a danger prompt text through the first display, wherein the danger prompt text is used for prompting the reduction of the opening degree of the window, namely the danger prompt text is used for prompting the driver, for example, the danger prompt text is' please take care of driving, the window moves upwards, and the danger of the rear row personnel is avoided. Alternatively, the danger prompting text is displayed on the second display for prompting to keep the sitting posture, that is, the danger prompting text is used for prompting the passenger, for example, the danger prompting text is "please do you to avoid falling outside the vehicle".

In the case that the risk state is the fourth candidate risk state, the in-vehicle terminal displays a danger prompt text through the first display, the danger prompt text is used for prompting that the passenger is separated from the vehicle, namely the danger prompt text is used for prompting the driver, for example, the danger prompt text is that the current passenger compartment rear row has no passenger.

In one possible embodiment, in a case where the risk state is a risk state of the target type, the in-vehicle terminal performs a danger prompting action based on the risk state, the danger prompting action being any one of seat shake, steering wheel shake, and dashboard flickering.

The seat vibration, the steering wheel vibration and the instrument panel flicker are used for reminding a driver, and the driver can remind passengers in time.

Under the embodiment, the vehicle-mounted terminal can remind a driver of timely reminding a passenger by triggering a danger prompting action, so that corresponding measures are timely taken to avoid danger, and the safety of the passenger taking a vehicle is improved.

The vehicle-mounted terminal can provide the danger indication by using any one of the above-described modes or a combination of any two or three of the above-described modes, which is not limited in the embodiment of the present application.

All the above optional technical solutions may be combined arbitrarily to form optional embodiments of the present application, and are not described herein again.

Through the technical scheme provided by the embodiment of the application, the vehicle-mounted terminal can directly identify the passengers in the intelligent cabin to obtain the passenger information of the passengers. The vehicle-mounted terminal can obtain the risk state of the vehicle riding on the vehicle based on the running information of the vehicle, the passenger information and the vehicle information of the vehicle. The vehicle-mounted terminal can timely prompt danger based on the risk state, so that the risk of a passenger or a driver can be timely eliminated, and the safety of the passenger taking a vehicle is improved.

Fig. 4 is a schematic structural diagram of a danger indicating device according to an embodiment of the present application, and referring to fig. 4, the device includes: an identification module 401, a risk status determination module 402, and a danger prompt module 403.

And the identification module 401 is configured to identify a passenger in the intelligent cabin to obtain passenger information of the passenger.

A risk state determination module 402 configured to determine a risk state of the passenger riding the vehicle based on traveling information of the vehicle, the traveling information being indicative of a traveling state of the vehicle, and at least one of passenger information of the passenger and vehicle information of the vehicle, the vehicle information being indicative of states of a plurality of functional components in the vehicle.

And a danger prompt module 403, configured to perform a danger prompt based on the risk status of the passenger, where the danger prompt is used to prompt the passenger that danger may exist.

In a possible embodiment, the identification module 401 is configured to perform motion identification on the passengers in the intelligent cabin, so as to obtain the motion of the passengers in the intelligent cabin. And carrying out position identification on the passengers in the intelligent cabin to obtain the positions of the passengers in the intelligent cabin. The passenger information includes the action and location of the passenger.

In a possible implementation manner, the recognition module 401 is configured to input the passenger image collected in the smart car into a motion recognition model, and perform feature extraction on the passenger image through the motion recognition model to obtain an image feature of the passenger image. And identifying based on the image characteristics of the passenger image through the action identification model to obtain the action of the passenger in the intelligent cabin.

In a possible implementation, the recognition module 401 is configured to perform full connection and normalization on the image features of the passenger image through the motion recognition model, and output probabilities that the passenger image corresponds to multiple candidate motions. And determining the candidate action with the highest probability in the plurality of candidate actions as the action of the passenger.

In a possible implementation, the identifying module 401 is configured to perform any one of the following:

and carrying out position identification on the passenger through a sensor on a seat of the intelligent cabin to obtain the position of the passenger in the intelligent cabin.

in the case where any one of the sensors on the seat of the smart cabin detects a passenger, the position of the passenger is determined as the position of the sensor on the seat of the smart cabin.

In the event that a passenger is not detected by a sensor on a seat of the smart cabin, the passenger is determined to be positioned as not sitting on the seat.

In one possible implementation, the risk state determination module 402 is configured to perform any one of:

when the travel information indicates that the vehicle is traveling straight or turning, and the passenger information indicates that the passenger's motion is any one of a hand extending out of the window, a body extending out of the window, and a parabolic motion outward of the window, a risk state of the passenger riding on the vehicle is determined as a first candidate risk state indicating that the passenger has performed a dangerous motion.

And determining the risk state of the passenger taking the vehicle as a second candidate risk state, wherein the second candidate risk state is used for indicating that the passenger leaves the seat, under the condition that the running information of the vehicle is straight running or turning and the passenger information of the passenger indicates that the passenger is positioned in the intelligent cabin and is not seated on the seat of the intelligent cabin.

And under the condition that the running information of the vehicle is turning, the vehicle information of the vehicle indicates that the opening degree of the window of the vehicle is greater than a preset opening degree threshold value, and the passenger information of the passenger indicates that the passenger is positioned in the intelligent cabin, determining that the risk state of the passenger taking the vehicle is a third candidate risk state, wherein the third candidate risk state is used for indicating that the passenger possibly leaves the vehicle.

And under the conditions that the running information of the vehicle is straight running or turning, the vehicle information of the vehicle indicates that the doors of the vehicle are not opened in the current running period, and the passenger information of the passenger indicates that the passenger is not positioned in the intelligent cabin, determining that the risk state corresponding to taking the vehicle is a fourth candidate risk state, wherein the fourth candidate risk state is used for indicating that the passenger has departed from the vehicle.

In one possible embodiment, the risk state determining module 402 is configured to input the driving information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle into a risk state determining model, predict by the risk state determining model based on the driving information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle, and output the risk state of the passenger riding in the vehicle.

In a possible embodiment, the risk status determining module 402 is configured to predict the driving information of the vehicle, and at least one of the passenger information of the passenger and the vehicle information of the vehicle by the risk status determining module 402, so as to obtain probabilities of a plurality of candidate risk statuses corresponding to the passenger taking the vehicle. And determining the candidate risk state with the highest probability in the candidate risk states as the risk state of the passenger taking the vehicle.

In one possible embodiment, the danger prompting module 403 is configured to perform at least one of the following:

and playing danger prompt voice based on the type of the risk state of the passenger.

And displaying danger prompt text based on the type of the risk state of the passenger.

It should be noted that: in the danger prompting device provided in the above embodiment, only the division of the above functional modules is used for illustration when performing danger prompting, and in practical applications, the above functions may be distributed by different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the above described functions. In addition, the danger prompting device and the danger prompting method provided by the above embodiments belong to the same concept, and specific implementation processes thereof are detailed in the method embodiments and are not described herein again.

The embodiment of the application provides a vehicle, which comprises a vehicle-mounted terminal and is used for executing the method, and the structure of the vehicle-mounted terminal is described as follows:

fig. 5 is a schematic structural diagram of an in-vehicle terminal according to an embodiment of the present application. Generally, the in-vehicle terminal 500 includes: one or more processors 501 and one or more memories 502.

The processor 501 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and so on. The processor 501 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 501 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 501 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content required to be displayed on a display screen. In some embodiments, the processor 501 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in the memory 502 is used to store at least one computer program for execution by the processor 501 to implement the method for hazard notification in a vehicle terminal provided by the method embodiments of the present application.

In some embodiments, the vehicle-mounted terminal 500 may further include: a peripheral interface 503 and at least one peripheral. The processor 501, memory 502, and peripheral interface 503 may be connected by buses or signal lines. Various peripheral devices may be connected to the peripheral interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 504, display screen 505, camera assembly 506, audio circuitry 507, and power supply 508.

The peripheral interface 503 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 501 and the memory 502. In some embodiments, the processor 501, memory 502, and peripheral interface 503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 501, the memory 502, and the peripheral interface 503 may be implemented on separate chips or circuit boards, which is not limited by the present embodiment.

The Radio Frequency circuit 504 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 504 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 504 converts an electrical signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth.

The display screen 505 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 505 is a touch display screen, the display screen 505 also has the ability to capture touch signals on or over the surface of the display screen 505. The touch signal may be input to the processor 501 as a control signal for processing. At this point, the display screen 505 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard.

The camera assembly 506 is used to capture images or video. Optionally, camera assembly 506 includes a front camera and a rear camera. Generally, a front camera is provided at a front panel of the in-vehicle terminal, and a rear camera is provided at a rear surface of the in-vehicle terminal.

Audio circuitry 507 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 501 for processing, or inputting the electric signals to the radio frequency circuit 504 to realize voice communication.

The power supply 508 is used to supply power to the respective components in the in-vehicle terminal 500. The power source 508 may be alternating current, direct current, disposable or rechargeable.

In some embodiments, the vehicle terminal 500 further includes one or more sensors 509. The one or more sensors 509 include, but are not limited to: acceleration sensor 510, gyro sensor 511, pressure sensor 512, optical sensor 513, and proximity sensor 514.

The acceleration sensor 510 may detect the magnitude of acceleration on three coordinate axes of the coordinate system established with the in-vehicle terminal 500.

The gyro sensor 511 may acquire a 3D motion of the user with respect to the in-vehicle terminal 500 in cooperation with the acceleration sensor 510.

The pressure sensor 512 may be disposed on a side frame of the in-vehicle terminal 500 and/or a lower layer of the display screen 505. When the pressure sensor 512 is disposed on the side frame of the in-vehicle terminal 500, a user's holding signal of the in-vehicle terminal 500 may be detected, and the processor 501 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 512. When the pressure sensor 512 is disposed at the lower layer of the display screen 505, the processor 501 controls the operability control on the UI interface according to the pressure operation of the user on the display screen 505.

The optical sensor 513 is used to collect the ambient light intensity. In one embodiment, the processor 501 may control the display brightness of the display screen 505 based on the ambient light intensity collected by the optical sensor 513.

The proximity sensor 514 is used to collect a distance between the user and the front surface of the in-vehicle terminal 500.

Those skilled in the art will appreciate that the configuration shown in fig. 5 is not limiting to the in-vehicle terminal 500, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components may be employed.

In an exemplary embodiment, a computer-readable storage medium, such as a memory including a computer program, is also provided, the computer program being executable by a processor to perform the hazard prompting method in the above embodiments. For example, the computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

In an exemplary embodiment, a computer program product or a computer program is also provided, which includes program code stored in a computer-readable storage medium, which is read by a processor of a computer device from the computer-readable storage medium, and which is executed by the processor to cause the computer device to execute the above-mentioned hazard notification method.

In some embodiments, the computer program according to the embodiments of the present application may be deployed to be executed on one computer device or on multiple computer devices located at one site, or may be executed on multiple computer devices distributed at multiple sites and interconnected by a communication network, and the multiple computer devices distributed at the multiple sites and interconnected by the communication network may constitute a block chain system.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is intended only to illustrate the alternative embodiments of the present application, and should not be construed as limiting the present application, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A method for danger alerting, the method comprising:

2. The method of claim 1, wherein identifying passengers in the smart car and obtaining passenger information for the passengers comprises:

carrying out action recognition on passengers in the intelligent cabin to obtain actions of the passengers in the intelligent cabin;

carrying out position identification on passengers in the intelligent cabin to obtain the positions of the passengers in the intelligent cabin; the passenger information includes an action and a position of the passenger.

3. The method of claim 2, wherein the performing action recognition on the passenger in the smart car and obtaining the action of the passenger in the smart car comprises:

inputting passenger images collected in the intelligent cabin into an action recognition model, and performing feature extraction on the passenger images through the action recognition model to obtain image features of the passenger images;

and identifying based on the image characteristics of the passenger image through the action identification model to obtain the action of the passenger in the intelligent cabin.

4. The method of claim 2, wherein the identifying the location of the passenger in the smart car comprises any one of:

carrying out position identification on the passengers through a sensor on a seat of the intelligent cabin to obtain the positions of the passengers in the intelligent cabin;

5. The method according to claim 1, wherein the determining the risk state of the passenger riding the vehicle based on the traveling information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle includes any one of:

determining a risk state of the passenger riding the vehicle as a second candidate risk state in the case that the traveling information of the vehicle is straight traveling or turning and the passenger information of the passenger indicates that the passenger is located inside the smart cabin and is not seated on a seat of the smart cabin, the second candidate risk state being used for representing the passenger is out of seat;

6. The method of claim 1, wherein the determining the risk status of the passenger riding the vehicle based on the travel information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle comprises:

inputting the traveling information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle into a risk state determination model, predicting by the risk state determination model based on the traveling information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle, and outputting a risk state of the passenger riding the vehicle.

7. The method of claim 6, wherein the predicting by the risk state determination model based on the traveling information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle, the outputting the risk state of the passenger riding in the vehicle comprises:

predicting the running information of the vehicle and at least one of the passenger information of the passenger and the vehicle information of the vehicle by the risk state determination model to obtain the probability of a plurality of candidate risk states corresponding to the passenger taking the vehicle;

determining a candidate risk state with a highest probability among the plurality of candidate risk states as a risk state of the passenger riding the vehicle.

8. The method of claim 1, wherein the danger prompting based on the risk status of the passenger comprises at least one of:

9. A hazard prompting device, the device comprising:

10. A vehicle comprising a vehicle terminal, the vehicle terminal comprising one or more processors and one or more memories having stored therein at least one computer program, the computer program being loaded and executed by the one or more processors to implement the hazard prompting method of any one of claims 1 to 8.

11. A computer-readable storage medium, in which at least one computer program is stored, which is loaded and executed by a processor to implement the hazard suggesting method of any one of claims 1 to 8.