CN115840911A

CN115840911A - Detection method and device for pedal mistaken touch, computer equipment and storage medium

Info

Publication number: CN115840911A
Application number: CN202211520307.5A
Authority: CN
Inventors: 许梦竹; 刘强
Original assignee: Weilai Automobile Technology Anhui Co Ltd
Current assignee: Weilai Automobile Technology Anhui Co Ltd
Priority date: 2022-11-30
Filing date: 2022-11-30
Publication date: 2023-03-24

Abstract

The application relates to a method and a device for detecting pedal mis-touch, a computer device, a storage medium and a computer program product. The method comprises the steps of monitoring the triggering condition of the vehicle accelerator pedal, acquiring multi-modal data inside the vehicle and environmental data outside the vehicle when the condition that the vehicle accelerator pedal is triggered is monitored, extracting features to obtain driving posture features, driving state features and driving environment features, determining the probability that the accelerator pedal is touched by mistake according to the extracted features, and further judging whether the accelerator pedal is touched by mistake. Because this embodiment is when confirming whether accelerator pedal is touched by mistake, is based on and is confirmed after carrying out the feature extraction to the inside multimode data of vehicle and the outside environmental data of vehicle, consequently, compare in traditional art and can improve the accuracy of judging whether accelerator pedal is touched by mistake, and then can improve driving safety, avoid the emergence of accident.

Description

Detection method and device for pedal mistaken touch, computer equipment and storage medium

Technical Field

The present application relates to the field of automotive technologies, and in particular, to a method and an apparatus for detecting a pedal mis-touch, a computer device, a storage medium, and a computer program product.

Background

With the improvement of living standard of people, automobiles have been popularized. During the driving process of the automobile, the condition that the accelerator pedal is touched by mistake is a common condition. On the traditional oil-burning motor vehicle, because of factors such as slow acceleration, small horsepower and matching with a manual gearbox, the mistouch of the accelerator pedal can not bring about great disasters. However, for a new electric vehicle, since the vehicle is equipped with high-power dual motors, acceleration of hundreds of kilometers can be completed within several seconds, and the accelerator pedal is extremely sensitive, and thus, a catastrophic accident may be caused by mistaken touch of the accelerator pedal.

In the conventional art, the current focus for solving the related problems is mainly on recognition of a scene outside the vehicle and recognition of a false touch on a sensor mounted on an accelerator pedal. Whether the surrounding environment is complex or not is identified, whether the accelerator pedal adopts a larger rotating angular speed or not is identified, and whether the mistaken touch is detected or not is judged according to a threshold value.

However, the current method for determining whether the accelerator pedal is touched by mistake depends heavily on the sensitivity of the external sensor and the camera of the vehicle, and the determination means for determining whether the accelerator pedal is touched by mistake is single. If a radar or a camera on the front side of the vehicle is interfered, or a driver habitually starts and steps on an accelerator suddenly, a 'mistaken touch system' is triggered to work, and the vehicle cannot be started normally. Therefore, the accuracy of the conventional method for judging whether the accelerator pedal is touched by mistake is low.

Disclosure of Invention

In view of the above, it is necessary to provide a detection method, an apparatus, a computer device, a computer readable storage medium, and a computer program product capable of accurately determining whether an accelerator pedal is touched by mistake, in order to solve the technical problem of low accuracy of the manner for determining whether the accelerator pedal is touched by mistake in the conventional technology.

In a first aspect, the present application provides a method for detecting a pedal mis-touch. The method comprises the following steps:

when it is monitored that an accelerator pedal of a vehicle is triggered, acquiring multi-modal data inside the vehicle and environmental data outside the vehicle;

performing feature extraction on the multi-modal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features and driving environment features;

and determining the probability of mistakenly touching the accelerator pedal according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the multimodal data of the vehicle interior includes audio information of the vehicle interior, facial images, pose images, and physiological indicator information of a target user; the feature extraction of the multi-modal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features and driving environment features includes: extracting the facial image and the attitude image of the target user in the vehicle by adopting a preset attitude feature extraction network to obtain corresponding driving attitude features; performing state feature extraction on the facial image and the physiological index information of the target user in the vehicle by adopting a preset state feature extraction network to obtain corresponding driving state features; and extracting the environmental characteristics of the audio information in the vehicle and the environmental data outside the vehicle by adopting a preset environmental characteristic extraction network to obtain corresponding driving environmental characteristics.

In one embodiment, the determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic comprises: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; and calculating the product of the user weight and the driving posture characteristic, the driving state characteristic and the driving environment characteristic to obtain the probability of mistakenly touching the accelerator pedal.

In one embodiment, the determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic comprises: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; and determining the probability of the accelerator pedal being touched by mistake according to the user weight, the first characteristic weight, the second characteristic weight, the third characteristic weight, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic comprises: determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; and determining the probability of mistakenly touching the accelerator pedal according to the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature and the driving environment feature.

In one embodiment, the determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic comprises: acquiring the angular speed of the accelerator pedal when triggered; and determining the probability of mistakenly touching the accelerator pedal according to the angular speed, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In a second aspect, the application further provides a detection device for detecting the pedal mistaken touch. The device comprises:

the data acquisition module is used for acquiring multi-modal data inside the vehicle and environmental data outside the vehicle when the condition that an accelerator pedal of the vehicle is triggered is monitored;

the characteristic extraction module is used for carrying out characteristic extraction on the multi-modal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture characteristics, driving state characteristics and driving environment characteristics;

and the false touch recognition module is used for determining the probability of false touch of the accelerator pedal according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In a third aspect, the application also provides a computer device. The computer device comprises a memory storing a computer program and a processor implementing the steps of the method according to the first aspect as described above when executing the computer program.

In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method according to the first aspect.

In a fifth aspect, the present application further provides a computer program product. The computer program product comprising a computer program which, when executed by a processor, performs the steps of the method according to the first aspect.

According to the pedal mis-touch detection method, the pedal mis-touch detection device, the computer equipment, the storage medium and the computer program product, the triggering condition of the vehicle accelerator pedal is monitored, when the accelerator pedal of the vehicle is triggered, multi-mode data inside the vehicle and environment data outside the vehicle are obtained, feature extraction is carried out to obtain driving posture features, driving state features and driving environment features, the probability that the accelerator pedal is touched by mistake is determined according to the extracted features, and whether the accelerator pedal is touched by mistake is judged. Because this embodiment is when confirming whether accelerator pedal is touched by mistake, is based on and is confirmed after carrying out the feature extraction to the inside multimode data of vehicle and the outside environmental data of vehicle, consequently, compare in traditional art and can improve the accuracy of judging whether accelerator pedal is touched by mistake, and then can improve driving safety, avoid the emergence of accident.

Drawings

FIG. 1 is a schematic flow chart illustrating a method for detecting a false touch of a pedal according to an embodiment;

FIG. 2 is a schematic flow chart of the feature extraction step in one embodiment;

FIG. 3 is a flowchart illustrating steps for determining a probability of an accelerator pedal being touched by mistake in one embodiment;

FIG. 4 is a flowchart illustrating steps for determining a probability of a mis-touch of an accelerator pedal in another embodiment;

FIG. 5 is a flowchart illustrating steps for determining a probability of an accelerator pedal being touched by mistake in accordance with yet another embodiment;

FIG. 6 is a flowchart illustrating steps for determining a probability of an accelerator pedal being touched by mistake in accordance with yet another embodiment;

FIG. 7 is a block diagram of a detection device for detecting a false touch of a pedal according to an embodiment;

FIG. 8 is a diagram illustrating an internal structure of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

In one embodiment, as shown in fig. 1, a method for detecting a pedal mis-touch is provided, and this embodiment is exemplified by applying the method to an in-vehicle device, and it is understood that the method may also be applied to a server, and may also be applied to a system including the in-vehicle device and the server, and is implemented by interaction between the in-vehicle device and the server. In this embodiment, the method may include the steps of:

step 102, when it is monitored that an accelerator pedal of a vehicle is triggered, multi-modal data inside the vehicle and environmental data outside the vehicle are acquired.

The multimodal data includes, but is not limited to, image data, audio data, and related user data, which can be acquired by an image acquisition module, an audio acquisition module, and a user data acquisition module disposed inside the vehicle. And the user may be a user currently driving the vehicle. The environment data may be data related to the current environment of the vehicle, for example, data related to the outside of the vehicle collected by a sensor and a camera outside the vehicle. An accelerator pedal being triggered refers to a situation in which the accelerator pedal is activated (e.g., depressed) to initiate an acceleration command.

In this embodiment, the vehicle-mounted device may monitor the triggering condition of the accelerator pedal of the vehicle in real time, and when it is monitored that the accelerator pedal of the vehicle is triggered, for example, when it is monitored that the accelerator pedal is stepped on, may acquire multimodal data inside the vehicle and environmental data outside the vehicle at the current time.

And 104, performing feature extraction on the multi-modal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features and driving environment features.

Among them, feature extraction is a process of extracting useful information (i.e., features) from initial measurement data in machine learning, pattern recognition, and image processing. Specifically, the driving posture feature may be related feature information reflecting whether the driving posture of the user is appropriate. The driving state feature may be related feature information reflecting whether the driving state of the user is normal or not. The driving environment characteristic may be related characteristic information reflecting whether the current driving environment is normal.

In the present embodiment, the in-vehicle device performs feature extraction based on the multimodal data inside the vehicle and the environmental data outside the vehicle acquired in the above steps, thereby obtaining extracted driving posture features, driving state features, and driving environment features.

And 106, determining the probability of mistakenly touching the accelerator pedal according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

The probability of the accelerator pedal being touched by mistake can reflect the possibility of the accelerator pedal being touched by mistake. For example, if the probability of the accelerator pedal being touched by mistake is higher, that is, the current trigger to the accelerator pedal is a false touch rather than a true meaning that the user wants to accelerate. If the probability of the accelerator pedal being touched by mistake is smaller, namely the current trigger to the accelerator pedal is not touched by mistake, but is the true meaning that the user wants to accelerate.

In this embodiment, the vehicle-mounted device may determine the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic, and the driving environment characteristic, and further determine whether the accelerator pedal is touched by mistake.

In the method for detecting the pedal mistaken touch, the vehicle-mounted equipment monitors the triggering condition of the vehicle accelerator pedal, acquires multi-mode data inside the vehicle and environment data outside the vehicle when the accelerator pedal of the vehicle is triggered, extracts the characteristics to obtain driving posture characteristics, driving state characteristics and driving environment characteristics, determines the probability of mistakenly touching the accelerator pedal according to the extracted characteristics, and further judges whether the accelerator pedal is mistakenly touched. Because this embodiment is when confirming whether accelerator pedal is touched by mistake, is based on and is confirmed after carrying out the feature extraction to the inside multimode data of vehicle and the outside environmental data of vehicle, consequently, compare in traditional art and can improve the accuracy of judging whether accelerator pedal is touched by mistake, and then can improve driving safety, avoid the emergence of accident.

In one embodiment, the multimodal data of the vehicle interior includes audio information of the vehicle interior, facial images, pose images, and physiological metric information of the target user. The target user may be a user located in a driver's seat, i.e. a driver driving a vehicle. The pose image is an image that reflects the overall pose of the target user, and may be, for example, a whole-body image of the target user. The physiological index information includes, but is not limited to, heart rate, body surface temperature, humidity, etc. of the target user.

Then, as shown in fig. 2, performing feature extraction on the multi-modal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features, and driving environment features, which may specifically include:

step 202, extracting the facial image and the posture image of the target user in the vehicle by adopting a preset posture characteristic extraction network to obtain the corresponding driving posture characteristic.

Wherein, the posture feature extraction network can be a pre-trained machine learning network or a neural network for extracting the driving posture feature of the target user. The driving posture feature may reflect whether the driving posture of the target user is appropriate. Therefore, the network is trained by adopting training data such as facial images and posture images of the user in normal driving postures and abnormal driving postures until the network converges, and the posture characteristic extraction network can be obtained. For example, when the driver speaks toward the rear row while the vehicle is in motion, the acquired face image and posture image are training data in an abnormal driving posture. That is, all face images or posture images violating the normal driving requirement are training data of abnormal driving postures. And the face image or the posture image which complies with the normal driving requirement is training data of the normal driving posture.

In this embodiment, the vehicle-mounted device performs attitude feature extraction on the face image and the attitude image of the target user inside the vehicle by using a preset attitude feature extraction network, so as to obtain corresponding driving attitude features. For example, the vehicle-mounted device may input a face image and a posture image of a target user inside the vehicle into a preset posture feature extraction network, that is, may obtain a feature of a driving posture output by the network, where the feature may specifically be a probability or a score of an abnormal driving posture.

And 204, performing state feature extraction on the facial image and the physiological index information of the target user in the vehicle by adopting a preset state feature extraction network to obtain corresponding driving state features.

Similarly, the state feature extraction network may be a machine learning network or a neural network trained in advance for extracting the driving state feature of the target user. The driving state feature may reflect whether the driving state of the target user is normal. Therefore, the network is trained by adopting training data such as facial images and physiological index information of the user in a normal driving state and an abnormal driving state, and the state feature extraction network can be obtained until the network converges. When an accident happens, the temperature, the humidity and the heart rate of a human body are obviously improved, and the face state changes, so that the face images and the physiological index information of the user in a normal driving state and an abnormal driving state can be collected as training data based on the face images and the physiological index information.

In this embodiment, the vehicle-mounted device performs state feature extraction on the facial image and the physiological index information of the target user inside the vehicle by using a preset state feature extraction network, so as to obtain corresponding driving state features. For example, the vehicle-mounted device may input the facial image and the physiological index information of the target user inside the vehicle into a preset state feature extraction network, that is, a feature of the driving state output by the network may be obtained, where the feature may specifically be a probability or a score of an abnormal driving state.

And step 206, extracting the environmental features of the audio information inside the vehicle and the environmental data outside the vehicle by using a preset environmental feature extraction network to obtain corresponding driving environmental features.

The environmental feature extraction network may be a machine learning network or a neural network trained in advance for extracting driving environmental features of an environment in which the vehicle is located. The driving environment characteristic may reflect whether the current driving environment is normal. Therefore, the network is trained by adopting the audio information inside the vehicle and the environmental data outside the vehicle under the normal driving environment and the abnormal driving environment, and the environmental feature extraction network can be obtained until the network converges.

In this embodiment, the vehicle-mounted device performs the environmental feature extraction on the audio information inside the vehicle and the environmental data outside the vehicle by using the preset environmental feature extraction network, so as to obtain the corresponding driving environmental feature. For example, the vehicle-mounted device may input the audio information inside the vehicle and the environment data outside the vehicle into a preset environment feature extraction network, that is, may obtain the driving environment feature output by the network, where the feature may specifically be a probability or a score of an abnormal driving environment.

In the above embodiment, the pre-trained feature extraction network is adopted to perform feature extraction on the multi-modal data inside the vehicle and the environmental data outside the vehicle, so as to obtain the corresponding driving posture feature, driving state feature and driving environmental feature, thereby improving the accuracy of feature extraction.

In one embodiment, the probability of the accelerator pedal being touched by mistake may be a product of the above-described extracted driving posture characteristic, driving state characteristic, and driving environment characteristic.

In one embodiment, as shown in fig. 3, in step 106, determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic may specifically include:

step 302, obtaining the user attribute information of the target user, and determining the matched user weight according to the user attribute information.

The user attribute information may be characteristic information of the user, for example, the user attribute information may include, but is not limited to, a gender, an age, a height, and the like of the user. The user weight is the importance degree of the user in the evaluation process of the mistaken touch of the accelerator pedal, which is determined based on the user attribute information. Specifically, different user weights may be set in advance based on different user attribute information.

Therefore, in this embodiment, the vehicle-mounted device may acquire the user attribute information of the target user, and may further determine the matched user weight according to the user attribute information.

And step 304, calculating the probability of mistakenly touching the accelerator pedal according to the user weight.

Specifically, the in-vehicle apparatus may calculate a product of the user weight and the driving posture characteristic, the driving state characteristic, and the driving environment characteristic, and take the product as the probability that the accelerator pedal is erroneously touched.

In the above embodiment, the vehicle-mounted device determines the matched user weight according to the user attribute information by acquiring the user attribute information of the target user, and calculates the probability of the accelerator pedal being touched by mistake according to the user weight. Since the user weight corresponding to the user attribute information is referred to when calculating the probability that the accelerator pedal is touched by mistake in the embodiment, the accuracy of identifying whether the accelerator pedal is touched by mistake can be further improved.

In one embodiment, as shown in fig. 4, in step 106, determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic may specifically include:

step 402, determining feature weights.

The characteristic weight can comprise a first characteristic weight corresponding to the driving posture characteristic, a second characteristic weight corresponding to the driving state characteristic and a third characteristic weight corresponding to the driving environment characteristic, and the characteristic weight is used for representing the importance degree of the corresponding characteristic in the evaluation process that the accelerator pedal is touched by mistake. Specifically, the feature weight corresponding to each feature may be set in advance based on the training data.

Therefore, in the present embodiment, according to the preset feature weights, a first feature weight corresponding to the driving posture feature, a second feature weight corresponding to the driving state feature, and a third feature weight corresponding to the driving environment feature may be determined.

And step 404, calculating the probability of mistakenly touching the accelerator pedal according to the characteristic weight.

Specifically, the vehicle-mounted device may determine the probability of the accelerator pedal being touched by mistake based on the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature. For example, a first product of the first feature weight and the driving posture feature, a second product of the second feature weight and the driving state feature, and a third product of the third feature weight and the driving environment feature may be calculated, and a sum of the first product, the second product, and the third product may be taken as the probability that the accelerator pedal is erroneously touched.

In the above embodiment, the in-vehicle device may calculate the probability that the accelerator pedal is touched by mistake according to the preset feature weight. Since the present embodiment refers to the weights of the different features, that is, the importance levels of the different features, when calculating the probability that the accelerator pedal is touched by mistake, the accuracy of identifying whether the accelerator pedal is touched by mistake can be further improved.

In one embodiment, as shown in fig. 5, in step 106, determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic may specifically include:

step 502, obtaining user attribute information of a target user, and determining a matched user weight according to the user attribute information.

The determining manner of the user weight in this embodiment may refer to step 302 in fig. 3, which is not described in detail in this embodiment.

Step 504, determine feature weights.

The determining manner of the feature weight in this embodiment may refer to step 402 in fig. 4, which is not described in detail in this embodiment.

And step 506, calculating the probability of mistakenly touching the accelerator pedal according to the user weight and the characteristic weight.

Specifically, the vehicle-mounted device may determine the probability of the accelerator pedal being touched by mistake according to the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature, and the driving environment feature.

For example, a first product of the first feature weight and the driving posture feature, a second product of the second feature weight and the driving state feature, and a third product of the third feature weight and the driving environment feature may be calculated, and a sum of the first product, the second product, and the third product may be calculated, and the product of the sum and the user weight may be taken as the probability that the accelerator pedal is touched by mistake.

In the above embodiment, the vehicle-mounted device calculates the probability of the accelerator pedal being touched by mistake by acquiring the user attribute information of the target user, determining the matched user weight according to the user attribute information, and according to the preset feature weight. In the embodiment, when the probability that the accelerator pedal is touched by mistake is calculated, not only the user weight corresponding to the user attribute information is referred to, but also the weights of different characteristics, namely the importance degrees of the different characteristics are referred to, so that whether the accelerator pedal is touched by mistake can be more accurately identified, and the driving safety is further improved.

In one embodiment, as shown in fig. 6, in step 106, determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic may specifically include:

in step 602, the angular velocity at which the accelerator pedal is triggered is obtained.

The angular velocity may be a normalized angular velocity when the accelerator pedal is depressed. Specifically, the vehicle-mounted device may monitor the angular velocity of the accelerator pedal in real time, so that the corresponding angular velocity may be acquired when the accelerator pedal is triggered.

And step 604, determining the probability of mistakenly touching the accelerator pedal according to the angular speed, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

Specifically, the vehicle-mounted device can calculate the product of the driving posture characteristic, the driving state characteristic and the driving environment characteristic, and multiply the product by the acquired angular velocity, and the obtained result can be used as the probability that the accelerator pedal is touched by mistake, so that the accuracy of identifying the accelerator pedal by mistake by the vehicle-mounted device is improved.

In one scenario, the vehicle-mounted device can also calculate the product of the user weight and the driving posture characteristic, the driving state characteristic and the driving environment characteristic, and multiply the product and the acquired angular velocity to obtain a result, wherein the result can be used as the probability that the accelerator pedal is touched by mistake, so that the accuracy of identifying the accelerator pedal by mistake by the vehicle-mounted device is further improved.

In one scenario, the vehicle-mounted device may further calculate a first product of the first feature weight and the driving posture feature, calculate a second product of the second feature weight and the driving state feature, calculate a third product of the third feature weight and the driving environment feature, calculate a sum of the first product, the second product and the third product, and multiply the sum with the obtained angular velocity, and an obtained result may be used as a probability that the accelerator pedal is touched by mistake, so as to further improve accuracy of the vehicle-mounted device in recognizing that the accelerator pedal is touched by mistake.

In one scenario, the vehicle-mounted device may further calculate a first product of the first feature weight and the driving posture feature, calculate a second product of the second feature weight and the driving state feature, calculate a third product of the third feature weight and the driving environment feature, calculate a sum of the first product, the second product and the third product, multiply the sum with the user weight and the acquired angular velocity, and obtain a result, which may be used as a probability that the accelerator pedal is touched by mistake, so as to further improve accuracy of the vehicle-mounted device in recognizing that the accelerator pedal is touched by mistake.

It should be understood that, although the steps in the flowcharts related to the embodiments as described above are sequentially displayed as indicated by arrows, the steps are not necessarily performed sequentially as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the flowcharts related to the embodiments described above may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the execution order of the steps or stages is not necessarily sequential, but may be rotated or alternated with other steps or at least a part of the steps or stages in other steps.

Based on the same inventive concept, the embodiment of the application also provides a pedal mis-touch detection device for realizing the pedal mis-touch detection method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the method, so that specific limitations in the embodiment of the detection device for detecting one or more pedal false touches provided below can be referred to the limitations on the detection method for detecting the pedal false touches, and details are not repeated herein.

In one embodiment, as shown in fig. 7, there is provided a pedal mis-touch detection device, including: a data acquisition module 702, a feature extraction module 704, and a false touch recognition module 706, wherein:

a data acquisition module 702, configured to acquire multimodal data inside a vehicle and environmental data outside the vehicle when it is monitored that an accelerator pedal of the vehicle is triggered;

a feature extraction module 704, configured to perform feature extraction on the multimodal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features, and driving environment features;

and the false touch recognition module 706 is configured to determine the probability that the accelerator pedal is touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the multimodal data of the vehicle interior includes audio information of the vehicle interior, facial images, pose images, and physiological indicator information of a target user; the feature extraction module is specifically configured to: extracting the facial image and the attitude image of the target user in the vehicle by adopting a preset attitude feature extraction network to obtain corresponding driving attitude features; performing state feature extraction on the facial image and the physiological index information of the target user in the vehicle by adopting a preset state feature extraction network to obtain corresponding driving state features; and extracting the environmental characteristics of the audio information in the vehicle and the environmental data outside the vehicle by adopting a preset environmental characteristic extraction network to obtain corresponding driving environmental characteristics.

In one embodiment, the false touch recognition module is specifically configured to: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; and calculating the product of the user weight and the driving posture characteristic, the driving state characteristic and the driving environment characteristic to obtain the probability of mistakenly touching the accelerator pedal.

In one embodiment, the false touch recognition module is further configured to: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; and determining the probability of the accelerator pedal being touched by mistake according to the user weight, the first characteristic weight, the second characteristic weight, the third characteristic weight, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the false touch recognition module is further configured to: determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; and determining the probability of the accelerator pedal being touched by mistake according to the first characteristic weight, the second characteristic weight, the third characteristic weight, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the false touch recognition module is further configured to: acquiring the angular speed of the accelerator pedal when triggered; and determining the probability of mistakenly touching the accelerator pedal according to the angular speed, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

All or part of each module in the device for detecting the pedal mistaken touch can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a vehicle-mounted device, and the internal structure thereof may be as shown in fig. 8. The computer apparatus includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input device. The processor, the memory and the input/output interface are connected by a system bus, and the communication interface, the display unit and the input device are connected by the input/output interface to the system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The input/output interface of the computer device is used for exchanging information between the processor and an external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a method of detecting a false pedal touch. The display unit of the computer device is used for forming a visual visible picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 8 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having a computer program stored therein, the processor implementing the following steps when executing the computer program:

In one embodiment, the multimodal data of the vehicle interior includes audio information of the vehicle interior, facial images, pose images, and physiological indicator information of a target user; the processor, when executing the computer program, further performs the steps of: extracting the facial image and the attitude image of the target user in the vehicle by adopting a preset attitude feature extraction network to obtain corresponding driving attitude features; performing state feature extraction on the facial image and the physiological index information of the target user in the vehicle by adopting a preset state feature extraction network to obtain corresponding driving state features; and extracting the environmental characteristics of the audio information in the vehicle and the environmental data outside the vehicle by adopting a preset environmental characteristic extraction network to obtain corresponding driving environmental characteristics.

In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; and calculating the product of the user weight and the driving posture characteristic, the driving state characteristic and the driving environment characteristic to obtain the probability of mistakenly touching the accelerator pedal.

In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; determining the probability of mistakenly touching the accelerator pedal according to the user weight, the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature and the driving environment feature.

In one embodiment, the processor, when executing the computer program, further performs the steps of: determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; and determining the probability of the accelerator pedal being touched by mistake according to the first characteristic weight, the second characteristic weight, the third characteristic weight, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the processor, when executing the computer program, further performs the steps of: acquiring the angular speed of the accelerator pedal when triggered; and determining the probability of mistakenly touching the accelerator pedal according to the angular speed, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:

In one embodiment, the multimodal data of the vehicle interior includes audio information of the vehicle interior, facial images, pose images, and physiological indicator information of a target user; the computer program when executed by the processor further realizes the steps of: extracting the facial image and the attitude image of the target user in the vehicle by adopting a preset attitude feature extraction network to obtain corresponding driving attitude features; performing state feature extraction on the facial image and the physiological index information of the target user in the vehicle by adopting a preset state feature extraction network to obtain corresponding driving state features; and extracting the environmental characteristics of the audio information inside the vehicle and the environmental data outside the vehicle by adopting a preset environmental characteristic extraction network to obtain corresponding driving environmental characteristics.

In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; and calculating the product of the user weight and the driving posture characteristic, the driving state characteristic and the driving environment characteristic to obtain the probability of mistakenly touching the accelerator pedal.

In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information; determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; and determining the probability of the accelerator pedal being touched by mistake according to the user weight, the first characteristic weight, the second characteristic weight, the third characteristic weight, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the computer program when executed by the processor further performs the steps of: determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature; and determining the probability of the accelerator pedal being touched by mistake according to the first characteristic weight, the second characteristic weight, the third characteristic weight, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, the computer program when executed by the processor further performs the steps of: acquiring the angular speed of the accelerator pedal when triggered; and determining the probability of mistakenly touching the accelerator pedal according to the angular speed, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

In one embodiment, a computer program product is provided, comprising a computer program which when executed by a processor performs the steps of:

In one embodiment, the multimodal data of the vehicle interior includes audio information of the vehicle interior, facial images, pose images, and physiological indicator information of a target user; the computer program when executed by the processor further realizes the steps of: extracting the facial image and the attitude image of the target user in the vehicle by adopting a preset attitude feature extraction network to obtain corresponding driving attitude features; performing state feature extraction on the facial image and the physiological index information of the target user in the vehicle by adopting a preset state feature extraction network to obtain corresponding driving state features; and extracting the environmental characteristics of the audio information in the vehicle and the environmental data outside the vehicle by adopting a preset environmental characteristic extraction network to obtain corresponding driving environmental characteristics.

It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) referred to in the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the relevant laws and regulations and standards of the relevant countries and regions.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high-density embedded nonvolatile Memory, resistive Random Access Memory (ReRAM), magnetic Random Access Memory (MRAM), ferroelectric Random Access Memory (FRAM), phase Change Memory (PCM), graphene Memory, and the like. Volatile Memory can include Random Access Memory (RAM), external cache Memory, and the like. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), for example. The databases involved in the embodiments provided herein may include at least one of relational and non-relational databases. The non-relational database may include, but is not limited to, a block chain based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, quantum computing based data processing logic devices, etc., without limitation.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for detecting pedal mis-touch, the method comprising:

2. The method of claim 1, wherein the multimodal data of the vehicle interior comprises audio information of the vehicle interior, facial images, pose images, and physiological metric information of a target user; the performing feature extraction on the multi-modal data inside the vehicle and the environmental data outside the vehicle to obtain corresponding driving posture features, driving state features and driving environment features includes:

extracting the facial image and the attitude image of the target user in the vehicle by adopting a preset attitude feature extraction network to obtain corresponding driving attitude features;

performing state feature extraction on the facial image and the physiological index information of the target user in the vehicle by adopting a preset state feature extraction network to obtain corresponding driving state features;

and extracting the environmental characteristics of the audio information in the vehicle and the environmental data outside the vehicle by adopting a preset environmental characteristic extraction network to obtain corresponding driving environmental characteristics.

3. The method of claim 2, wherein determining the probability of the accelerator pedal being touched by mistake based on the driving posture characteristic, the driving state characteristic, and the driving environment characteristic comprises:

acquiring user attribute information of the target user, and determining matched user weight according to the user attribute information;

and calculating the product of the user weight and the driving posture characteristic, the driving state characteristic and the driving environment characteristic to obtain the probability of mistakenly touching the accelerator pedal.

4. The method of claim 2, wherein determining the probability of the accelerator pedal being touched by mistake based on the driving posture characteristic, the driving state characteristic, and the driving environment characteristic comprises:

determining a first feature weight corresponding to the driving posture feature according to a preset feature weight, determining a second feature weight corresponding to the driving state feature, and determining a third feature weight corresponding to the driving environment feature;

and determining the probability of the accelerator pedal being touched by mistake according to the user weight, the first characteristic weight, the second characteristic weight, the third characteristic weight, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

5. The method according to claim 1 or 2, wherein the determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic comprises:

and determining the probability of mistakenly touching the accelerator pedal according to the first feature weight, the second feature weight, the third feature weight, the driving posture feature, the driving state feature and the driving environment feature.

6. The method according to any one of claims 1 to 4, wherein the determining the probability of the accelerator pedal being touched by mistake according to the driving posture characteristic, the driving state characteristic and the driving environment characteristic comprises:

acquiring the angular speed of the accelerator pedal when triggered;

and determining the probability of mistakenly touching the accelerator pedal according to the angular speed, the driving posture characteristic, the driving state characteristic and the driving environment characteristic.

7. A detection device for pedal mis-touch, the device comprising:

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor, when executing the computer program, implements the steps of the method of any of claims 1 to 6.

9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 6 when executed by a processor.