CN115965946A - Method, device and equipment for detecting violation of foot behaviors of driver and storage medium - Google Patents
Method, device and equipment for detecting violation of foot behaviors of driver and storage medium Download PDFInfo
- Publication number
- CN115965946A CN115965946A CN202211525611.9A CN202211525611A CN115965946A CN 115965946 A CN115965946 A CN 115965946A CN 202211525611 A CN202211525611 A CN 202211525611A CN 115965946 A CN115965946 A CN 115965946A
- Authority
- CN
- China
- Prior art keywords
- foot
- right foot
- outline
- area
- coincidence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Auxiliary Drives, Propulsion Controls, And Safety Devices (AREA)
Abstract
The invention provides a method, a device, equipment and a storage medium for detecting violation of foot behaviors of a driver, which can detect the violation of foot behaviors of the driver in real time, fill up the technical blank and help to avoid accidents in time.
Description
Technical Field
The invention belongs to the technical field of driving safety detection, and particularly relates to a method, a device, equipment and a storage medium for detecting foot behavior violation of a driver.
Background
The detection of the driver's violations is usually focused on whether the driver is driving fatigue and driving dangerously, such as smoking, making a call, etc. during driving, but a very important part is often ignored, namely, the driver's foot behaviors that the heel is still on the brake pedal after the foot is moved from the brake pedal to the accelerator pedal, or the heel is still on the accelerator pedal after the foot is moved from the accelerator pedal to the brake pedal, which are dangerous and easily cause accidents. Currently, no targeted violation detection technology exists.
Disclosure of Invention
In view of the above, the present invention provides a method, an apparatus, a device and a storage medium for detecting violation of foot behaviors of a driver.
The technical scheme adopted by the invention is as follows:
as a first aspect of the present invention, there is provided a method for detecting a foot behavior violation of a driver, including:
acquiring a foot video of a driver in real time;
in each foot video frame, identifying a right foot target, and determining a first coincidence ratio of the right foot target and an accelerator pedal and a second coincidence ratio of the right foot target and a brake pedal;
and judging whether the first coincidence proportion and the second coincidence proportion are both larger than a threshold value in a plurality of continuous foot video frames with a preset number, if so, judging that the current foot behavior is illegal.
As a second aspect of the present invention, there is provided a foot behavior violation detecting device for a driver, including:
the video acquisition module is used for acquiring a foot video of a driver in real time;
the coincidence proportion determining module is used for identifying a right foot target in each foot video frame, and determining a first coincidence proportion of the right foot target and an accelerator pedal and a second coincidence proportion of the right foot target and a brake pedal;
and the violation judging module is used for judging whether the first coincidence ratio and the second coincidence ratio are both larger than a threshold value in a plurality of continuous foot video frames with preset number, and if so, the current foot behavior is violated.
As a third aspect of the present invention, there is provided an electronic device comprising a storage module including instructions loaded and executed by a processor, the instructions, when executed, causing the processor to execute the method for detecting a violation of a driver's foot behavior of the first aspect.
As a fourth aspect of the present invention, there is provided a computer-readable storage medium storing one or more programs which, when executed by a processor, implement a driver foot behavior violation detection method of the first aspect described above.
The invention can detect the foot behavior of the driver in real time, fills the technical blank and is beneficial to avoiding accidents in time.
Drawings
The invention is described in detail below with reference to the following figures and embodiments:
fig. 1 is a flowchart of a method for detecting violation of foot behaviors of a driver according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a right foot target frame of the present invention;
FIG. 3 is a schematic view of a right foot target of the present invention;
FIG. 4 is a schematic view of the right foot profile of the present invention;
FIG. 5 is a schematic diagram of a denoised right foot outline of the present invention;
fig. 6 is a schematic diagram of a device for detecting foot behavior violation of a driver according to an embodiment of the present invention;
fig. 7 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The embodiments of the present invention will be described below with reference to the drawings attached to the specification. It should be noted that the embodiments mentioned in the present description are not exhaustive and do not represent the only embodiments of the present invention. The following examples are provided for clarity of the inventive concepts of the present invention and are not intended to limit the scope of the invention. It will be apparent to those skilled in the art that various changes and modifications can be made in the embodiment without departing from the spirit and scope of the invention, and it is intended to cover all modifications and variations of the present invention as fall within the true spirit and scope of the invention.
As shown in fig. 1, an embodiment of the present invention provides a method for detecting violation of foot behaviors of a driver, which includes the following specific steps:
and S101, acquiring a foot video of the driver in real time from the camera.
The camera is arranged obliquely above the accelerator pedal and the brake pedal.
S102, in each foot video frame, identifying a right foot target, and determining a first coincidence proportion of the right foot target and an accelerator pedal and a second coincidence proportion of the right foot target and a brake pedal.
When the right foot target is identified, the current foot video frame is input into a deep learning target detection model, such as a yolov5 model, so as to obtain a right foot target frame, which is shown in fig. 2. The model is trained and optimized through a large number of right foot sample pictures in advance.
The first coincidence ratio and the second coincidence ratio are determined as follows:
1. since the interference of extracting the contour directly from the image shown in fig. 2 is high, the right foot target frame is taken as a boundary, and the right foot target image is captured from the current foot video frame, as shown in fig. 3.
2. And (3) performing edge detection on the right foot target image through a Sobel operator to determine the right foot outline, referring to FIG. 4, in order to eliminate interference information, denoising the right foot outline further, and referring to FIG. 5 for the denoised right foot outline.
3. The positions of the accelerator pedal and the brake pedal in the camera picture are fixed, so that the outline of the accelerator pedal and the outline of the brake pedal can be predetermined and stored, and when in actual detection, according to the outline of the right foot and the outline of the predetermined accelerator pedal and the outline of the brake pedal, the area of the outline of the right foot, the first area of the overlapped area of the outline of the right foot and the outline of the accelerator pedal and the second area of the overlapped area of the outline of the right foot and the outline of the brake pedal are calculated, and the specific calculation process is as follows:
a. and respectively generating a right foot outline binary image, an accelerator pedal outline binary image and a brake pedal outline binary image according to the right foot outline, the accelerator pedal outline and the brake pedal outline, wherein in the binary images, the pixel values of pixel points positioned in the outlines are 1, and the pixel values of the other pixel points are 0.
b. And calculating the area of the area formed by all pixel points with the pixel value of 1 in the right foot outline binary image to obtain the area of the right foot outline.
c. And performing AND calculation on the right foot outline binary image and the accelerator pedal outline binary image to obtain a first coincidence image, calculating the area of an area formed by all pixel points with the pixel values of 1 in the first coincidence image to obtain a first area, performing AND calculation on the right foot outline binary image and the brake pedal outline binary image to obtain a second coincidence image, and calculating the area of an area formed by all pixel points with the pixel values of 1 in the second coincidence image to obtain a second area.
4. And dividing the first area by the right foot outline area to obtain a first coincidence ratio, and dividing the second area by the right foot outline area to obtain a second coincidence ratio.
S103, judging whether the first coincidence ratio and the second coincidence ratio are both larger than a threshold value in a plurality of continuous foot video frames (such as continuous 5 frames) with a preset number, if so, judging that the current foot behavior is illegal.
Because the foot action is a continuous action and is paused for a period of time when the foot action is finished, when the first coincidence proportion and the second coincidence proportion are both larger than the threshold value in a plurality of continuous foot video frames, the current foot action is considered to be illegal, and corresponding warning treatment can be carried out at the moment to remind a driver of correcting the illegal action in time, so that the occurrence of accidents is avoided in time.
In the present embodiment, the above threshold may be set to 50%.
Therefore, the method can detect the foot behaviors of the driver in real time, fills up the technical blank, and is beneficial to avoiding accidents in time.
The driver foot behavior violation detection apparatus according to one or more embodiments of the present invention will be described in detail below. Those skilled in the art will appreciate that these detection means can be constructed using commercially available hardware components and configured by the steps taught in the present scheme. Fig. 6 shows a device for detecting foot behavior violation of a driver according to an embodiment of the present invention, and as shown in fig. 6, the device includes a video acquisition module 11, a coincidence ratio determination module 12, and a violation judgment module 13.
And the video acquisition module 11 is used for acquiring the foot video of the driver from the camera in real time.
And the coincidence ratio determining module 12 is used for identifying a right foot target in each foot video frame, and determining a first coincidence ratio of the right foot target and an accelerator pedal and a second coincidence ratio of the right foot target and a brake pedal.
When the right foot target is identified, the current foot video frame is input into a deep learning target detection model, such as a yolov5 model, so as to obtain a right foot target frame, which is shown in fig. 2. The model is trained and optimized in advance through a large number of right foot sample pictures.
The first coincidence proportion and the second coincidence proportion are determined as follows:
1. since the interference of extracting the contour directly from the image shown in fig. 2 is high, the right foot target image is captured from the current foot video frame by taking the right foot target frame as a boundary, see fig. 3.
2. And (3) performing edge detection on the right foot target image through a Sobel operator to determine a right foot outline, referring to fig. 4, in order to eliminate interference information, denoising the right foot outline further, and referring to fig. 5 for the denoised right foot outline.
3. The positions of the accelerator pedal and the brake pedal in the camera picture are fixed, so that the outline of the accelerator pedal and the outline of the brake pedal can be predetermined and stored, and when in actual detection, according to the outline of the right foot and the outline of the predetermined accelerator pedal and the outline of the brake pedal, the area of the outline of the right foot, the first area of the overlapped area of the outline of the right foot and the outline of the accelerator pedal and the second area of the overlapped area of the outline of the right foot and the outline of the brake pedal are calculated, and the specific calculation process is as follows:
a. and respectively generating a right foot outline binary image, an accelerator pedal outline binary image and a brake pedal outline binary image according to the right foot outline, the accelerator pedal outline and the brake pedal outline, wherein in the binary images, the pixel values of pixel points positioned in the outlines are 1, and the pixel values of the other pixel points are 0.
b. And calculating the area of the area formed by all pixel points with the pixel value of 1 in the right foot outline binary image to obtain the area of the right foot outline.
c. And performing phase-and-sum calculation on the right foot outline binary image and the accelerator pedal outline binary image to obtain a first coincidence image, calculating the area of an area formed by all pixel points with the pixel values of 1 in the first coincidence image to obtain a first area, performing phase-and-sum calculation on the right foot outline binary image and the brake pedal outline binary image to obtain a second coincidence image, and calculating the area of an area formed by all pixel points with the pixel values of 1 in the second coincidence image to obtain a second area.
4. And dividing the first area by the right foot outline area to obtain a first coincidence ratio, and dividing the second area by the right foot outline area to obtain a second coincidence ratio.
And the violation judging module 13 is configured to judge whether the first coincidence ratio and the second coincidence ratio are both greater than a threshold value in a plurality of consecutive foot video frames (for example, consecutive 5 frames) of a preset number, and if so, determine that the current foot behavior is violated.
Because the foot action is a continuous action and has a pause of a period of time when the foot action is finished, when the first coincidence ratio and the second coincidence ratio are both larger than the threshold value in a plurality of continuous foot video frames, the current foot action is considered to be illegal, and corresponding warning processing can be carried out at the moment to remind a driver to correct the illegal action in time, so that accidents are avoided in time.
In the present embodiment, the above threshold may be set to 50%.
In summary, the device for detecting violation of foot behavior of driver provided in the foregoing embodiments may execute the method for detecting violation of foot behavior of driver provided in the foregoing embodiments.
Similar to the above concept, the structure of the device for detecting foot behavior violation of a driver shown in fig. 6 can be implemented as an electronic device, and fig. 7 is a block diagram schematically illustrating the structure of an electronic device according to an embodiment of the present invention.
Illustratively, the electronic device comprises a storage module 21 and a processor 22, the storage module 21 comprising instructions that are loaded and executed by the processor 22, the instructions, when executed, causing the processor 22 to perform the steps according to various exemplary embodiments of the present invention described in the above-mentioned driver foot behavior violation detecting method section of this specification.
It should be understood that the processor 22 may be a Central Processing Unit (CPU), and that the processor 22 may also be other general purpose processors, digital signal processors
(digital signal processor, DSP), application specific integrated circuit
An Application Specific Integrated Circuit (ASIC), a Field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
Also provided in an embodiment of the present invention is a computer-readable storage medium storing one or more programs which, when executed by a processor, implement the steps according to various exemplary embodiments of the present invention described in the above section for a driver foot behavior violation detection method.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, or suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable storage media, which may include computer readable storage media (or non-transitory media) and communication media (or transitory media).
The term computer readable storage medium includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those skilled in the art. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
For example, the computer readable storage medium may be an internal storage unit of the electronic device of the foregoing embodiment, such as a hard disk or a memory of the electronic device. The computer readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk provided on the electronic device, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card (FlashCard), and the like.
The electronic device and the computer-readable storage medium provided by the foregoing embodiments can perform violation detection in real time for the foot behavior of the driver, fill up the technical gap, and contribute to avoiding an accident in time.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
Claims (10)
1. A method for detecting violation of foot behavior of a driver is characterized by comprising the following steps:
acquiring a foot video of a driver in real time;
in each foot video frame, identifying a right foot target, and determining a first coincidence proportion of the right foot target and an accelerator pedal and a second coincidence proportion of the right foot target and a brake pedal;
and judging whether the first coincidence proportion and the second coincidence proportion are both larger than a threshold value in a plurality of continuous foot video frames with a preset number, if so, judging that the current foot behavior is illegal.
2. The method for detecting the violation of the foot behavior of the driver according to claim 1, wherein the identifying a right foot target further comprises:
and inputting the current foot video frame into the deep learning target detection model to obtain a right foot target frame.
3. The method for detecting the violation of driver foot behavior according to claim 2, wherein the deep learning target detection model adopts yolov5 model.
4. The method of claim 3, wherein determining a first coincidence ratio of the right foot target to an accelerator pedal and a second coincidence ratio of the right foot target to a brake pedal further comprises:
taking the right foot target frame as a boundary, and intercepting a right foot target image from the current foot video frame;
performing edge detection on the right foot target graph to determine a right foot outline;
calculating the area of the right foot outline, a first area of a superposition area of the right foot outline and the accelerator pedal outline and a second area of a superposition area of the right foot outline and the brake pedal outline according to the right foot outline and the predetermined accelerator pedal outline and brake pedal outline;
and dividing the first area by the outline area of the right foot to obtain a first superposition ratio, and dividing the second area by the outline area of the right foot to obtain a second superposition ratio.
5. The method for detecting the violation of driver's foot behavior according to claim 4, wherein the edge detection of the right foot target map and the determination of the right foot contour further comprise:
and carrying out edge detection on the right foot target graph through a Sobel operator to determine the outline of the right foot.
6. The method for detecting violation of driver foot behavior according to claim 5, wherein calculating a first area of a right foot profile area, a first area of a region of coincidence of the right foot profile and the accelerator pedal profile, and a second area of a region of coincidence of the right foot profile and the brake pedal profile according to the right foot profile and predetermined accelerator pedal profile and brake pedal profile, further comprises:
respectively generating a right foot outline binary image, an accelerator pedal outline binary image and a brake pedal outline binary image according to the right foot outline, the accelerator pedal outline and the brake pedal outline, wherein in the binary images, the pixel values of pixel points positioned in the outlines are 1, and the pixel values of the other pixel points are 0;
calculating the area of an area formed by all pixel points with the pixel value of 1 in the right foot outline binary image to obtain the area of a right foot outline;
and performing phase and calculation on the right foot outline binary image and the accelerator pedal outline binary image to obtain a first coincidence image, calculating the area of a region formed by all pixel points with pixel values of 1 in the first coincidence image to obtain the first area, performing phase and calculation on the right foot outline binary image and the brake pedal outline binary image to obtain a second coincidence image, and calculating the area of a region formed by all pixel points with pixel values of 1 in the second coincidence image to obtain the second area.
7. The method of claim 6, wherein the threshold is 50%.
8. A device for detecting violation of foot behavior of a driver, comprising:
the video acquisition module is used for acquiring a foot video of a driver in real time;
the coincidence proportion determining module is used for identifying a right foot target in each foot video frame, and determining a first coincidence proportion of the right foot target and an accelerator pedal and a second coincidence proportion of the right foot target and a brake pedal;
and the violation judging module is used for judging whether the first coincidence proportion and the second coincidence proportion are both larger than a threshold value in a plurality of continuous foot video frames with preset number, and if so, judging that the current foot behaviors are violated.
9. An electronic device comprising a memory module including instructions loaded and executed by a processor, the instructions when executed cause the processor to perform a method of driver foot behavior violation detection according to any of claims 1-7.
10. A computer readable storage medium storing one or more programs, wherein the one or more programs, when executed by a processor, implement a driver foot behavior violation detection method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211525611.9A CN115965946A (en) | 2022-12-01 | 2022-12-01 | Method, device and equipment for detecting violation of foot behaviors of driver and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211525611.9A CN115965946A (en) | 2022-12-01 | 2022-12-01 | Method, device and equipment for detecting violation of foot behaviors of driver and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115965946A true CN115965946A (en) | 2023-04-14 |
Family
ID=87354004
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211525611.9A Pending CN115965946A (en) | 2022-12-01 | 2022-12-01 | Method, device and equipment for detecting violation of foot behaviors of driver and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115965946A (en) |
-
2022
- 2022-12-01 CN CN202211525611.9A patent/CN115965946A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110569699B (en) | Method and device for carrying out target sampling on picture | |
| CN108875534B (en) | Face recognition method, device, system and computer storage medium | |
| CN107622501B (en) | Boundary detection method for medical image | |
| US20170230652A1 (en) | Determination method and determination apparatus | |
| US10445868B2 (en) | Method for detecting a defect on a surface of a tire | |
| CN111368587B (en) | Scene detection method, device, terminal equipment and computer readable storage medium | |
| JP6260112B2 (en) | Image processing apparatus, image processing method, scanner, and storage medium | |
| CN110969046B (en) | Face recognition method, face recognition device and computer-readable storage medium | |
| CN109344824B (en) | Text line region detection method, device, medium and electronic equipment | |
| CN109509166B (en) | Printed circuit board image detection method and device | |
| CN109102026B (en) | Vehicle image detection method, device and system | |
| CN105976570A (en) | Driver smoking behavior real-time monitoring method based on vehicle video monitoring | |
| CN104573680A (en) | Image detection method, image detection device and traffic violation detection system | |
| CN112597846A (en) | Lane line detection method, lane line detection device, computer device, and storage medium | |
| CN113869137A (en) | Event detection method and device, terminal equipment and storage medium | |
| CN113808135B (en) | Image brightness abnormality detection method, electronic device, and storage medium | |
| CN111191482B (en) | Brake lamp identification method and device and electronic equipment | |
| CN113963149A (en) | Medical bill picture fuzzy judgment method, system, equipment and medium | |
| CN112837384A (en) | Vehicle marking method and device and electronic equipment | |
| CN115965946A (en) | Method, device and equipment for detecting violation of foot behaviors of driver and storage medium | |
| CN113239738B (en) | Image blurring detection method and blurring detection device | |
| CN111340677B (en) | Video watermark detection method, apparatus, electronic device, and computer readable medium | |
| CN113642521A (en) | Traffic light identification quality evaluation method and device and electronic equipment | |
| CN113313189A (en) | Shielding detection method and device and electronic equipment | |
| CN112150373B (en) | Image processing method, image processing apparatus, and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |