CN111169483A - Driving assisting method, electronic equipment and device with storage function - Google Patents
Driving assisting method, electronic equipment and device with storage function Download PDFInfo
- Publication number
- CN111169483A CN111169483A CN201811340499.5A CN201811340499A CN111169483A CN 111169483 A CN111169483 A CN 111169483A CN 201811340499 A CN201811340499 A CN 201811340499A CN 111169483 A CN111169483 A CN 111169483A
- Authority
- CN
- China
- Prior art keywords
- driver
- image
- road condition
- current road
- driving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
Abstract
The invention discloses a driving assistance method, electronic equipment and a device with a storage function, wherein the driving assistance method comprises the following steps: the electronic equipment acquires image information of a driver and current road condition information; judging the driving state of the driver by combining the driver image information and the current road condition information; and if the driver is in an abnormal driving state, driving reminding is carried out on the driver. By the method, the driver can be reminded in time when the driver is in the abnormal driving state, so that the driving safety is improved.
Description
Technical Field
The present invention relates to the field of safe driving technologies, and in particular, to a driving assistance method, an electronic device, and a device having a storage function.
Background
With the rapid increase of the number of automobiles and the increasing complexity of the driving road conditions, people have higher and higher requirements on automobile safety systems.
In the prior art, only the driving road condition of an automobile is generally monitored, and a driver can judge the road condition in or out of a sight line range by watching the monitored image. In the process of high-speed driving of the vehicle, the driver cannot be reminded timely in case of emergency such as dozing or obstacles appearing in a distant place, and the safety is low.
Disclosure of Invention
The invention mainly solves the technical problem of how to remind a driver in time when the driver is in an abnormal driving state so as to improve the driving safety.
In order to solve the technical problem, one technical solution adopted by the present invention is to provide a driving assistance method, including: the electronic equipment acquires image information of a driver and current road condition information; judging the driving state of the driver by combining the driver image information and the current road condition information; and if the driver is in an abnormal driving state, driving reminding is carried out on the driver.
The image information of the driver comprises an eyeball image of the driver, and the current road condition information comprises a current road condition image; the determining the driving state of the driver by combining the driver image information and the current road condition information includes: identifying a driver sight line in the eyeball image of the driver; mapping the identified sight line of the driver to the current road condition image; judging whether the sight of the driver is in the range of the barrier in the current road condition image; if not, judging that the driver is in an abnormal driving state.
Wherein the eyeball image of the driver comprises a pupil image of the driver; the identifying the driver sight line in the eyeball image of the driver comprises: acquiring the central position of an eye socket and the central position of a pupil from the pupil image of the driver; and obtaining the sight line of the driver according to the central position of the eye socket and the central position of the pupil.
The electronic equipment acquires image information of a driver and current road condition information, and the method comprises the following steps: the electronic equipment acquires an eyeball image of the driver by using a front camera and acquires the current road condition image by using a rear camera; the mapping the identified driver sight line to the current road condition image comprises: acquiring a first position relation between the eyeball and the front camera based on the eyeball image of the driver, and acquiring a second position relation between the barrier and the rear camera based on the current road condition image; and obtaining the sight line position of the sight line of the driver in the current road condition image according to the first position relation between the eyeball and the front camera, the second position relation between the obstacle and the rear camera and the preset third position relation between the front camera and the rear camera.
The barrier is at least one of a road pit, a road block, a restriction mark, a red light and a moving object; the driving reminding comprises prompting the driver to pay attention to the current road condition or making driving correction.
The image information of the driver comprises an eyeball image of the driver, and the current road condition information comprises a current road condition image; the electronic equipment acquires the image information of the driver and the current road condition information, and comprises the following steps: the electronic equipment acquires the driver image information and the current road condition information at least twice within a preset time period; the determining the driving state of the driver by combining the driver image information and the current road condition information includes: judging whether the road condition in the preset time period changes or not according to the current road condition image in the current road condition information of at least two times; if the road condition in the preset time period changes, judging whether the eyeball of the driver in the preset time period changes according to the eyeball image of the driver in the at least two times of driver image information; and if the eyeball of the driver is not changed in the preset time period, judging that the driver is in an abnormal driving state.
Before determining whether the eyeball of the driver changes within a preset time period according to the eyeball image of the driver in the at least two times of driver image information, the method further comprises the following steps: judging whether drivers corresponding to the driver image information of the at least two times are the same or not according to the driver image information of the at least two times; and if so, executing the step of judging whether the eyeball of the driver changes in the preset time period.
Wherein the driving reminder includes navigation information providing a nearest rest area.
In order to solve the above technical problem, another technical solution adopted by the present invention is to provide an electronic device, including a memory and a processor coupled to each other; the memory is used for storing program data; the processor executes the program data for implementing the method as described in any one of the above.
In order to solve the above technical problem, another technical solution adopted by the present invention is to provide an apparatus having a storage function, which stores program data that can be executed to implement the method according to any one of the above.
The invention has the beneficial effects that: different from the prior art, the driving state of the driver is judged by acquiring the image information of the driver and the current road condition information through the electronic equipment and combining the image information of the driver and the current road condition information, and if the driver is in an abnormal driving state with dangerous driving, the driving reminding is carried out on the driver so as to improve the driving safety.
Drawings
FIG. 1 is a schematic flow chart diagram of a first embodiment of a driving assistance method provided by the present invention;
FIG. 2 is a schematic flow chart diagram of a second embodiment of a driving assistance method provided by the present invention;
FIG. 3 is a schematic flow chart diagram of a third embodiment of a driving assistance method provided by the present invention;
FIG. 4 is a schematic flow chart diagram illustrating a fourth embodiment of a driving assistance method provided by the present invention;
FIG. 5a is a schematic diagram of an application scenario in which the driving assistance method provided by the present invention is applied;
FIG. 5b is a schematic diagram of an application scenario of the driving assistance method provided by the present invention;
FIG. 6 is a schematic flow chart diagram of a fifth embodiment of a driving assistance method provided by the present invention;
FIG. 7 is a schematic flow chart diagram illustrating a driving assistance method according to a sixth embodiment of the present invention;
FIG. 8 is a schematic structural diagram of an embodiment of an electronic device provided in the present invention;
fig. 9 is a schematic structural diagram of an embodiment of an apparatus with a storage function according to the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Referring to fig. 1, fig. 1 is a schematic flow chart of a driving assistance method according to a first embodiment of the present invention. The driving assistance method in the present embodiment includes:
step S101: the electronic equipment acquires image information of a driver and current road condition information.
The electronic equipment is equipment capable of acquiring image information such as a mobile phone or a vehicle data recorder, and the image information of a driver and the corresponding current road condition information can be acquired in the driving process through the electronic equipment. In a specific application scenario, the electronic device is a mobile phone, and a user can acquire image information of a driver and corresponding current road condition information by using the mobile phone in a driving process by placing the mobile phone on a console of an automobile. In another specific application scenario, the electronic device is an automobile data recorder, and similarly, image information of a driver and corresponding current road condition information can be acquired by using the automobile data recorder in a driving process; in addition, it can be understood that after the electronic equipment is installed, video images and sounds of the whole driving process of the automobile can be recorded, and evidence can be provided for traffic accidents; people who like self-driving tour can also use the method to record the process of overcoming difficult danger; the method includes the steps that when the automobile is driven, video recording is carried out while time, speed and positions are recorded in the video recording, and the automobile is a black box; and parking monitoring and the like can be performed at ordinary times. In other application scenarios, the functions of the electronic device can be realized through the mobile phone and the automobile data recorder, for example, the current road condition information is acquired through the automobile data recorder, and the corresponding driver image information is acquired through the mobile phone.
Step S102: and judging the driving state of the driver by combining the image information of the driver and the current road condition information.
The driving state of the driver in the application is divided into two types, one is an abnormal driving state, and the other is a normal driving state; the abnormal driving state means a state that a driver may affect driving safety due to various reasons during actual driving, in which the driver is in an abnormal driving state, and in contrast, the state that the driving safety is not affected is a normal driving state. In the actual driving process, the more typical abnormal driving states include driver inattention, fatigue driving and the like; for example, when a driver actually drives, the driver may continuously view an outside scene, or may be distracted by an outside scene on a busy street, or drowsiness occurs due to fatigue driving, the electronic device may determine whether the line of sight or attention of the driver deviates from the driving direction by combining the acquired driver image information and the current road condition information, so that the driver may be inattentive or fatigue driving, and further determine the driving state of the driver.
Step S103: and if the driver is in an abnormal driving state, driving reminding is carried out on the driver.
It can be understood that when it is determined that the driver is in an abnormal driving state that may affect driving safety during actual driving, in order to ensure driving safety, the driver needs to be actively prompted to drive accordingly. The driving reminding can have different intensities, the abnormal driving state of the driver can be classified according to the acquired image information of the driver and the current road condition information, and different reminding levels can be defined according to the possibly caused danger degree; for example, the distance between the vehicle and the front obstacle is determined according to the current road condition information, if the distance is shorter, the collision risk possibly generated is higher, and the reminding level is higher at the moment, or the reminding level is set according to the current speed when the driver is determined to be in a stall according to the image information of the driver, and the emergency degree is deeper when the speed is higher, and the emergency degree is shallower otherwise. The driving reminding can be reminded through voice, warning sound, characters or images, or can be reminded through corresponding control on the vehicle, such as controlling a steering wheel or performing emergency braking or vehicle speed control. For example, the driving alert may be a warning sound alert such as a "ticker", and the deeper the emergency, the higher the frequency of the warning sound; or a voice prompt, such as "abnormal ahead", "urgent ahead" or the like, wherein the deeper the urgency is, the faster the voice rate and the higher the volume of the prompt are.
In this embodiment, the image information of the driver and the current road condition information are acquired through the electronic device, the driving state of the driver is judged by combining the image information of the driver and the current road condition information, and if the driver is in an abnormal driving state with dangerous driving, the driver is prompted to drive, so that the driving safety is improved.
Referring to fig. 2, fig. 2 is a schematic flow chart of a driving assistance method according to a second embodiment of the present invention. The image information of the driver comprises an eyeball image of the driver, the current road condition information comprises a current road condition image, and the auxiliary driving method in the embodiment comprises the following steps:
step S201: the electronic equipment acquires image information of a driver and current road condition information.
In this implementation scenario, step S201 provided in this embodiment is substantially similar to step S101 in the first embodiment of the driving assistance method provided in this application, and is not described herein again.
Step S202: and identifying the driver sight line in the eyeball image of the driver.
Step S203: and mapping the identified sight line of the driver to the current road condition image.
Step S204: and judging whether the sight of the driver is in the range of the obstacle in the current road condition image. If not, go to step S205; if so, judging that the driver is in a normal driving state, and ending the process without driving reminding for the driver in the normal driving state.
It is understood that the driver's sight line should focus on the direction of the attention while driving the car, for example, when an obstacle appears in front, when turning or changing lanes, the driver should focus on the rear view mirror and the position to be turned or changed lanes; therefore, in order to ensure safe driving, the sight line of the driver can be identified in the eyeball image of the driver acquired by the electronic equipment, and the identified sight line of the driver is mapped to the current road condition image; because the current road condition image has a direction in which the sight of the driver should pay attention, for example, the direction of an obstacle in the driving direction, it can be determined whether the sight of the driver is within the range of the obstacle in the current road condition image.
Step S205: and judging that the driver is in an abnormal driving state, and carrying out driving reminding on the driver.
It can be understood that if the sight line of the driver is not within the range of the obstacle in the current road condition image, it indicates that the driver is not in the direction to which the current driving attention should be paid, and the driving safety may be affected, so that it can be determined that the driver is in an abnormal driving state, and in order to ensure the driving safety, a corresponding driving prompt needs to be performed on the driver. In this implementation scenario, step S205 in this embodiment is substantially similar to step S103 in the first embodiment of the driving assistance method provided in this application, and is not described herein again.
In this embodiment, the driver sight line in the driver ball image is identified, and the identified driver sight line is mapped to the current road condition image, so that whether the driver sight line is in the range of the obstacle in the current road condition image is judged, and when the driver sight line is not in the range of the obstacle in the current road condition image, it is judged that the driver is in an abnormal driving state with dangerous driving, so as to remind the driver of driving, and improve driving safety.
Referring to fig. 3, fig. 3 is a schematic flow chart of a driving assistance method according to a third embodiment of the present invention. The driver image information includes an eyeball image of the driver, the eyeball image of the driver includes a pupil image of the driver and an obstacle image reflected by the eyeball, and the driving assistance method in the embodiment includes:
step S301: the electronic equipment acquires image information of a driver and current road condition information.
In this implementation scenario, step S301 provided in this embodiment is substantially similar to step S101 in the first embodiment of the driving assistance method provided in this application, and is not described herein again.
Step S302: the center position of the eye socket and the center position of the pupil are acquired from the pupil image of the driver.
Step S303: and obtaining the sight line of the driver according to the central position of the eye socket and the central position of the pupil.
It will be appreciated that the pupil image of the driver includes portions of the driver's eye sockets, and that, in general, the positions of the human pupils in their eye sockets may reflect the human gaze direction. For example, in direct viewing in front, the pupil is generally centered in the orbit; when looking obliquely left or right, the pupil is located in the left or right part of the eye socket; or the pupil may be located in the upper or lower portion of the orbit when looking up or down. Therefore, the central position of the eye socket and the central position of the pupil can be acquired from the pupil image of the driver, and then a ray extending from the central position of the eye socket to the outside through the central position of the pupil can be used as the gaze direction of the eyes of the driver, namely, the gaze direction of the driver can be referred to as the driver sight line.
Step S304: and mapping the identified sight line of the driver to the current road condition image.
Step S305: and judging whether the sight of the driver is in the range of the obstacle in the current road condition image. If not, executing step S306; if so, judging that the driver is in a normal driving state, and ending the process without driving reminding for the driver in the normal driving state.
Step S306: and judging that the driver is in an abnormal driving state, and carrying out driving reminding on the driver.
In this implementation scenario, steps S304 to S306 provided in this embodiment are substantially similar to steps S203 to S205 in the second embodiment of the driving assistance method provided in this application, and are not described herein again.
In the embodiment, the position of the center of the eyeball and the center position of the pupil are obtained through the pupil image of the driver and the obstacle image reflected by the eyeball, and the sight line of the driver is obtained, so that whether the sight line of the driver is in the range of the obstacle in the current road condition image or not can be judged in real time, a prompt is sent when the sight line deviates, and dangerous driving of the driver is avoided.
Referring to fig. 4, fig. 4 is a schematic flow chart of a driving assistance method according to a fourth embodiment of the present invention. The electronic equipment of the application can comprise a front camera and a rear camera, and the driving assistance method in the embodiment comprises the following steps:
step S401: the electronic equipment acquires the eyeball image of the driver by using a front camera and acquires the current road condition image by using a rear camera.
In a specific application scenario, the electronic device is a mobile phone placed on a console of an automobile driver, and the image information of the driver and the corresponding current road condition information can be respectively acquired by using front and rear cameras of the mobile phone in the driving process. In another specific application scenario, the electronic device is a vehicle event data recorder with front and rear cameras, and similarly, the front and rear cameras can be used for respectively acquiring image information of a driver and corresponding current road condition information during driving. In other application scenarios, the functions of the electronic device can also be realized together by the mobile phone and the automobile data recorder, for example, the current road condition information is acquired by the automobile data recorder, the corresponding driver image information is acquired by the mobile phone, at this time, the automobile data recorder is used as the rear camera of the electronic device, and the mobile phone is used as the front camera of the electronic device.
Step S402: the center position of the eye socket and the center position of the pupil are acquired from the pupil image of the driver.
Step S403: and obtaining the sight line of the driver according to the central position of the eye socket and the central position of the pupil.
In this implementation scenario, steps S402 to S403 in this embodiment are substantially similar to steps S302 to S303 in the third embodiment of the driving assistance method provided in this application, and are not described herein again.
Step S404: and acquiring a first position relation between the eyeball and the front camera based on the eyeball image of the driver, and acquiring a second position relation between the barrier and the rear camera based on the current road condition image.
It can be understood that, since the front camera can be used to acquire the eyeball image of the driver, the first position relationship between the front camera and the eyeball can be acquired based on the eyeball image of the driver; similarly, since the current road condition image can be acquired by using the rear camera, the second positional relationship between the rear camera and the obstacle existing in the current road condition can be acquired based on the current road condition image. The first positional relationship and the second positional relationship can be obtained through various technologies, for example, the automatic focusing function of the existing camera is utilized, the automatic focusing purpose is achieved by moving the position of the lens or the position of the photosensitive element in the lens, and the distance of the object to be focused can be converted according to the moving distance of the lens or the photosensitive element; or as an infrared distance meter, the distance can be measured by projecting infrared rays to an object and then receiving reflected light rays reflected back. The acquisition technique regarding the first positional relationship and the second positional relationship in the present application is not particularly limited, and any one or more of various existing positional relationship acquisition techniques may be used.
Step S405: and obtaining the sight line position of the sight line of the driver in the current road condition image according to the first position relation between the eyeball and the front camera, the second position relation between the obstacle and the rear camera and the preset third position relation between the front camera and the rear camera.
Because the electronic equipment comprises the front camera and the rear camera, the relative positions of the front camera and the rear camera are known, namely the third position relation of the front camera and the rear camera can be preset by a designer; or when the functions of the electronic equipment are realized together through the mobile phone and the automobile data recorder, the automobile data recorder serves as a rear camera of the electronic equipment, the mobile phone serves as a front camera of the electronic equipment, and at the moment, the third positional relationship between the mobile phone and the automobile data recorder can be preset by a driver or a user. It can be understood that, since the first positional relationship between the eyeball and the front camera, the second positional relationship between the obstacle and the rear camera, and the third positional relationship between the front camera and the rear camera are known, and since the driver ' S sight line is obtained in step S403, the sight line position of the driver ' S sight line in the current road condition image, that is, the relative relationship between the driver ' S sight line and the obstacle in the current road condition image, can be obtained.
Step S406: and judging whether the sight of the driver is in the range of the obstacle in the current road condition image. If not, executing step S407; if so, judging that the driver is in a normal driving state, and ending the process without driving reminding for the driver in the normal driving state.
Step S407: and judging that the driver is in an abnormal driving state, and carrying out driving reminding on the driver.
In this implementation scenario, steps S406 to S407 provided in this embodiment are substantially similar to steps S305 to S306 in the second embodiment of the driving assistance method provided in this application, and are not repeated here.
In an application scenario, as shown in fig. 5a, fig. 5a is a schematic diagram illustrating an application scenario to which the driving assistance method provided by the present invention is applied, in a driving process, an electronic device may be used to acquire image information of a driver and corresponding current road condition information, an eyeball image of the driver includes a pupil image of the driver, a central position K of an orbit and a central position E of the pupil may be acquired from the pupil image of the driver, and it may be understood that a ray extending from the central position K of the orbit to the outside via the pupil center E may be used as a gaze direction of an eye of the driver, that is, a driver sight line F. Because the electronic equipment can acquire the eyeball image of the driver, the first position relation between the electronic equipment and the eyeball can be acquired based on the eyeball image of the driver; similarly, since the current road condition image can be acquired by using the electronic device, the second positional relationship between the electronic device and the obstacle G existing in the current road condition can be acquired based on the current road condition image; the electronic equipment can respectively acquire eyeball images and obstacle images of a driver through the front camera and the rear camera, at the moment, the relative positions of the front camera and the rear camera are known, namely, the third position relation of the front camera and the rear camera is preset by a designer; when the electronic device acquires the eyeball image and the obstacle image of the driver by using the same imaging component, the third positional relationship is a coincidence relationship. Therefore, the sight line position of the driver sight line F in the current road condition image can be obtained according to the first position relationship, the second position relationship and the third position relationship, that is, whether the driver sight line F is in the range of the obstacle G in the current road condition can be judged.
It is understood that in other embodiments, the driver image information includes an eye image of the driver, the eye image of the driver includes a pupil image of the driver and an obstacle image reflected by an eye, and the driver's sight line may be determined in the following manner.
First, the curvature radius of the eyeball of the driver is acquired from the image of the obstacle reflected from the eyeball, and the position of the center of the eyeball is determined. It can be understood that the eyeball image of the driver includes an obstacle image reflected by the eyeball, the image of the obstacle is displayed on the surface of the eyeball due to reflection, the surface of the eyeball is a curved surface, so that the obstacle image has a curvature, the curvature radius of the eyeball of the driver can be obtained by obtaining the curvature radius of the obstacle image, and the center position of the eyeball can be determined. Or, the outer side of the eyeball can be used as a spherical mirror of a reflecting surface, which has a divergence effect, according to the principle of the convex mirror, parallel light rays are projected onto the convex mirror, the reflected light rays become scattered light rays, and if the scattered light rays extend to the back of the mirror surface of the convex mirror along the opposite direction of the reflected light rays, the scattered light rays can converge and intersect at one point, which is the main focus of the convex mirror; therefore, incident light is emitted from each point of the obstacle, and when the incident light reaches the surface of the eyeball, the incident light is reflected, and a straight line opposite to the reflected light after reflection inevitably passes through a main focus, which is the central position of the eyeball. Of course, in another embodiment, information on a reference object such as a reflection image of a steering wheel or a rearview mirror of a vehicle may be acquired from an image of an eyeball of a driver, and a curvature radius of the driver may be acquired by analyzing the reflection image of the reference object to specify a position of a center of the eyeball.
Then, the center position of the pupil is acquired from the pupil image of the driver, and the driver sight line is obtained according to the position of the center of the eyeball and the center position of the pupil. Since the eyeball image of the driver includes the pupil image of the driver, and the center position of the pupil of the driver can be acquired from the pupil image of the driver, it can be understood that a ray extending from the center position of the eyeball to the outside via the center position of the pupil may be used as the gaze direction of the eyes of the driver, that is, referred to as the driver's sight line.
In the embodiment, the position of the center of the eyeball and the position of the center of the pupil are obtained through the pupil image of the driver and the image of the obstacle reflected by the eyeball, and the sight line of the driver is obtained, so that whether the sight line of the driver is in the range of the obstacle in the current road condition image or not can be judged in real time, a prompt is sent out when the sight line deviates, and the dangerous driving of the driver is avoided.
In another application scenario, as shown in fig. 5B, fig. 5B is a schematic diagram illustrating that the driving assistance method provided by the present invention is applied to another application scenario, the electronic device is a mobile phone placed on a console of an automobile driver, a front camera a and a rear camera B of the mobile phone can be used to respectively obtain image information of the driver and corresponding current road condition information during driving, an eyeball image of the driver includes a barrier image C reflected by an eyeball, and the barrier image C is a reflection image of a barrier G displayed on the surface of the eyeball, so that the barrier image C has a curvature, and the curvature radius of the eyeball of the driver can be obtained by obtaining the curvature radius through the barrier image C, so that the center D of the eyeball can be determined; further, since the eye image of the driver includes the pupil image of the driver, and the pupil center E of the driver can be acquired from the pupil image of the driver, it can be understood that a ray extending from the center D of the eye to the outside via the pupil center E may be referred to as the gaze direction of the eyes of the driver, that is, the driver's sight line F. Because the eyeball image of the driver can be acquired by using the front camera A, the first position relation between the front camera A and the eyeball can be acquired based on the eyeball image of the driver; similarly, since the current road condition image can be acquired by using the rear camera B, a second positional relationship between the rear camera B and the obstacle G existing in the current road condition can be acquired based on the current road condition image; since the electronic device includes the front camera a and the rear camera B, the relative positions of the front camera a and the rear camera B are known, that is, the third positional relationship of the front camera a and the rear camera B may be preset by a designer. Therefore, the sight line position of the driver sight line F in the current road condition image can be obtained according to the first position relation between the eyeball and the front camera a, the second position relation between the obstacle G and the rear camera B, and the third position relation between the front camera a and the rear camera B, that is, whether the driver sight line F is in the range of the obstacle G in the current road condition can be judged.
In addition, the barrier in the application can be at least one of a road pit, a road block, a restriction mark, a red light and a moving object; the driving reminding comprises prompting the driver to pay attention to the current road condition or making driving correction. It can be understood that the driving reminder can also carry out different reminders according to different barrier types, including different reminding levels and reminding modes; for example, the reminding level is set according to the danger possibly caused by different obstacles, the reminding mode can remind through voice, warning sound, characters or images, or remind through correspondingly controlling the vehicle, and the like. In addition, the driving reminding of the driver by the electronic device can be embodied in a navigation route or a navigation mode, for example, a navigation path is changed to avoid a corresponding obstacle and the like.
Referring to fig. 6, fig. 6 is a schematic flow chart of a driving assistance method according to a fifth embodiment of the present invention. The image information of the driver comprises an eyeball image of the driver, and the current road condition information comprises a current road condition image, and the driving assistance method in the embodiment comprises the following steps:
step S601: the electronic equipment obtains the driver image information and the current road condition information at least twice within a preset time period.
Step S602: and judging whether the road condition in the preset time period changes or not according to the current road condition image in the current road condition information at least twice. If the road condition in the preset time period changes, executing step S603; if the road condition in the preset time period does not change, the road condition in the preset time period is judged to not change, and at the moment, the situation that the vehicle does not move can be understood, so that the driver does not need to be reminded of driving, and the process is ended.
Generally, the driving behavior of the driver with inattentive attention or fatigue driving needs to be judged by the sight line change of the driver within a period of time, so that the electronic device can acquire at least two current road condition images within a preset time period, and then judge whether the road condition within the preset time period changes by comparing the current road condition images at different times before and after the current road condition images. It is understood that the preset time period of the present application can be preset by a designer, or can be set by a user, or can be appropriately adjusted according to the reaction time of the user.
Step S603: and judging whether the eyeball of the driver changes in the preset time period according to the eyeball image of the driver in the at least two times of driver image information. If the eyeball of the driver does not change within the preset time period, executing step S604; if the eyeball of the driver changes in the preset time period, it indicates that the attention of the driver changes along with the changed road condition in the preset time period, and also indicates that the driver is not in a state of being stuck due to fatigue driving in the preset time period, and at this time, the driver does not need to be reminded to drive, and the process is ended.
After the current road condition images at different times in the preset time period are compared and the road condition in the preset time period is judged to be changed, the vehicle is in a driving state at the moment, and whether the driver is in an abnormal driving state needs to be further judged. Therefore, the electronic device can acquire the eyeball images of the driver at least twice in the preset time period, and then judge whether the eyeball of the driver in the preset time period changes along with the changed road condition by comparing the eyeball images of the driver in different time periods before and after, so as to judge whether the abnormal driving state such as the drowsiness of the driver due to the fatigue driving or not in the actual road condition occurs.
Step S604: and judging that the driver is in an abnormal driving state, and carrying out driving reminding on the driver.
In this implementation scenario, step S604 in this embodiment is substantially similar to step S205 in the second embodiment of the driving assistance method provided in this application, and is not repeated here.
In this embodiment, whether the vehicle is in a driving state is determined by at least two current road condition images, and whether the driver is in an abnormal driving state is further determined by at least two eyeball images of the driver after the vehicle is determined to be in the driving state, so that the driver is correspondingly prompted to drive when the driver is in the abnormal driving state.
Referring to fig. 7, fig. 7 is a flowchart illustrating a driving assistance method according to a sixth embodiment of the present invention. The driving assistance method in the present embodiment includes:
step S701: the electronic equipment obtains the driver image information and the current road condition information at least twice within a preset time period.
Step S702: and judging whether the road condition in the preset time period changes or not according to the current road condition image in the current road condition information at least twice. If the road condition in the preset time period changes, executing step S703; if the road condition in the preset time period does not change, the road condition in the preset time period is judged to not change, and at the moment, the situation that the vehicle does not move can be understood, so that the driver does not need to be reminded of driving, and the process is ended.
In this implementation scenario, steps S701 to S702 provided in this embodiment are substantially similar to steps S601 to S602 in the fifth embodiment of the driving assistance method provided in this application, and are not described herein again.
Step S703: and judging whether the drivers corresponding to the driver image information of the at least two times are the same or not according to the driver image information of the at least two times. If yes, go to step S704; if the difference is different, the driver is replaced within the preset time period, so that the possibility of fatigue driving does not exist, driving reminding is not needed, and the process is ended.
After the step S702 determines that the road condition within the preset time period changes, the electronic device may further determine whether the driver is in an abnormal driving state through at least two times of eyeball images of the driver, at this time, it is required to first determine whether the drivers corresponding to the eyeball images of the drivers at different times are the same driver, if the drivers are different drivers, it may be considered that the drivers are replaced at this time, so that there is no possibility of fatigue driving, and a driving prompt is not required, so as to prevent that the driving time of the drivers mistakenly analyzed due to the replacement of the drivers in the midway of driving exceeds the preset driving time, and avoid performing false early warning of fatigue driving. In a specific application scene, after driving is started, the electronic equipment can automatically input an image of a driver and establish an image library; if the driver image recorded again in the driving process is the same as the image recorded first, the same driver is judged, if the driver image recorded again is different from the image recorded first, different drivers are judged, and a new driver image is also stored in the image library; when the driver is replaced for multiple times, the electronic equipment can acquire the corresponding driving record of the driver according to the matched information in the image library, so that whether the driving time of the driver meets the time length of fatigue driving or not is judged, and whether the current driver belongs to the fatigue driving or not is judged; it is also possible to determine whether the driving time interval of the driver meets the normal rest time interval, thereby preventing the driver from escaping from the detection of fatigue driving by interchanging drivers when driving two or more vehicles.
Step S704: and judging whether the eyeball of the driver changes in the preset time period according to the eyeball image of the driver in the at least two times of driver image information. If the eyeball of the driver does not change within the preset time period, executing step S705; if the eyeball of the driver changes in the preset time period, it indicates that the attention of the driver changes along with the changed road condition in the preset time period, and the driver does not have the drowsiness caused by fatigue driving in the preset time period, so that the driver does not need to be reminded to drive, and the process is ended.
Step S705: and judging that the driver is in an abnormal driving state, and carrying out driving reminding on the driver.
When it is determined that the current vehicle is in a driving state and the driver is not replaced, it is necessary to further determine whether the driver is in an abnormal driving state. In this implementation scenario, steps S704 to S705 provided in this embodiment are substantially similar to steps S603 to S604 in the fifth embodiment of the driving assistance method provided in this application, and are not described herein again.
It can be understood that, when it is determined that the driver is in the abnormal driving state due to fatigue driving, the driving prompt of the electronic device may include providing navigation information of the nearest rest area, so that the driver may go to the nearest rest area to rest, and driving danger caused by fatigue driving is avoided.
Referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 80 in the present application is the electronic device in the driving assistance method, the electronic device 80 includes a memory 800 and a processor 802 coupled to each other, the memory 800 is used for storing program data, and the processor 802 executes the program data, so as to implement the following methods:
the processor 802 acquires the image information of the driver and the current road condition information, determines the driving state of the driver by combining the image information of the driver and the current road condition information, and performs driving reminding on the driver if the driver is in an abnormal driving state.
Further, the image information of the driver includes an eyeball image of the driver, the current traffic information includes an image of the current traffic condition, and the processor 802 executes the above-mentioned judgment of the driving state of the driver by combining the image information of the driver and the current traffic information, which specifically includes:
the processor 802 identifies the driver's sight line in the eyeball image of the driver, maps the identified driver's sight line to the current road condition image, and determines whether the driver's sight line is within the range of the obstacle in the current road condition image, and if not, determines that the driver is in an abnormal driving state.
Further, the eyeball image of the driver includes a pupil image of the driver, and the processor 802 performs the above recognition of the driver's sight line in the eyeball image of the driver, specifically including:
the processor 802 acquires the center position of the eye socket and the center position of the pupil from the pupil image of the driver; and obtaining the sight line of the driver according to the central position of the eye socket and the central position of the pupil.
Further, the processor 802 performs the above-mentioned obtaining of the image information of the driver and the current road condition information, including:
the processor 802 acquires an eyeball image of the driver by using the front camera and acquires a current road condition image by using the rear camera;
the processor 802 performs the above mapping of the identified driver's sight line to the current road condition image, including:
the processor 802 acquires a first position relation between the eyeball and the front camera based on the eyeball image of the driver, and acquires a second position relation between the obstacle and the rear camera based on the current road condition image; and obtaining the sight line position of the sight line of the driver in the current road condition image according to the first position relation between the eyeballs and the front camera, the second position relation between the barrier and the rear camera and the preset third position relation between the front camera and the rear camera.
As an implementation manner, the obstacle may be at least one of a road pit, a road block, a restriction mark, a red light and a moving object, and the driving reminding includes prompting the driver to pay attention to the current road condition or making a driving correction.
As an implementation manner, the image information of the driver includes an eyeball image of the driver, and the current traffic information includes an image of the current traffic condition, and the processor 802 performs the above-mentioned obtaining of the image information of the driver and the current traffic information, including:
the processor 802 obtains at least two times of driver image information and current road condition information within a preset time period;
the processor 802 performs the above-mentioned judgment of the driving status of the driver by combining the image information of the driver and the current road condition information, including:
the processor 802 determines whether the road condition within a preset time period changes according to the current road condition image in the current road condition information at least twice; if the road condition in the preset time period changes, judging whether the eyeball of the driver in the preset time period changes according to the eyeball image of the driver in the at least two times of driver image information; and if the eyeball of the driver is not changed within the preset time period, judging that the driver is in an abnormal driving state.
Further, the processor 802 is further configured to, before performing the above-mentioned determining whether the eyeball of the driver changes within the preset time period according to the eyeball image of the driver in the at least two times of driver image information:
the processor 802 judges whether the drivers corresponding to the driver image information of at least two times are the same according to the driver image information of at least two times; if the two images are the same, judging whether the eyeball of the driver changes within the preset time period according to the eyeball image of the driver in the at least two times of driver image information.
As one implementable manner, the driving reminder includes navigation information providing the nearest rest area.
For the specific process of the processor 802 in the embodiment of the electronic device 80 of the present application to implement the above functions, reference may be made to the above method embodiment.
Referring to fig. 9, fig. 9 is a diagram illustrating an apparatus with a storage function according to the present invention. The device 90 having a storage function stores therein program data 900, and the program data 900 can be executed to implement the driving assist method as described above. The device 90 with a storage function may be a storage chip in a server, a readable and writable tool such as an SD card, or may be a server.
In the several embodiments provided in the present application, it should be understood that the disclosed method, apparatus, and device may be implemented in other ways. For example, the above-described device architecture implementations are merely illustrative, and for example, a division of a module or a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A driving assist method, characterized by comprising:
the electronic equipment acquires image information of a driver and current road condition information;
judging the driving state of the driver by combining the driver image information and the current road condition information;
and if the driver is in an abnormal driving state, driving reminding is carried out on the driver.
2. The driving assist method according to claim 1, wherein the driver image information includes an eyeball image of a driver, and the current road condition information includes a current road condition image;
the determining the driving state of the driver by combining the driver image information and the current road condition information includes:
identifying a driver sight line in the eyeball image of the driver;
mapping the identified sight line of the driver to the current road condition image;
judging whether the sight of the driver is in the range of the barrier in the current road condition image;
if not, judging that the driver is in an abnormal driving state.
3. The driving assist method according to claim 2, wherein the eyeball image of the driver includes a pupil image of the driver;
the identifying the driver sight line in the eyeball image of the driver comprises:
acquiring the central position of an eye socket and the central position of a pupil from the pupil image of the driver;
and obtaining the sight line of the driver according to the central position of the eye socket and the central position of the pupil.
4. The driving assistance method according to claim 3, wherein the obtaining of the image information of the driver and the current road condition information by the electronic device comprises:
the electronic equipment acquires an eyeball image of the driver by using a front camera and acquires the current road condition image by using a rear camera;
the mapping the identified driver sight line to the current road condition image comprises:
acquiring a first position relation between the eyeball and the front camera based on the eyeball image of the driver, and acquiring a second position relation between the barrier and the rear camera based on the current road condition image;
and obtaining the sight line position of the sight line of the driver in the current road condition image according to the first position relation between the eyeball and the front camera, the second position relation between the obstacle and the rear camera and the preset third position relation between the front camera and the rear camera.
5. The driving assist method according to claim 2,
the barrier is at least one of a road pit, a road block, a restriction mark, a red light and a moving object;
the driving reminding comprises prompting the driver to pay attention to the current road condition or making driving correction.
6. The driving assist method according to claim 1, wherein the driver image information includes an eyeball image of a driver, and the current road condition information includes a current road condition image;
the electronic equipment acquires the image information of the driver and the current road condition information, and comprises the following steps:
the electronic equipment acquires the driver image information and the current road condition information at least twice within a preset time period;
the determining the driving state of the driver by combining the driver image information and the current road condition information includes:
judging whether the road condition in the preset time period changes or not according to the current road condition image in the current road condition information of at least two times;
if the road condition in the preset time period changes, judging whether the eyeball of the driver in the preset time period changes or not according to the eyeball image of the driver in the at least two times of driver image information;
and if the eyeball of the driver is not changed in the preset time period, judging that the driver is in an abnormal driving state.
7. The driving assist method according to claim 6,
before the determining whether the eyeball of the driver changes within the preset time period according to the eyeball image of the driver in the at least two times of driver image information, the method further includes:
judging whether drivers corresponding to the driver image information of the at least two times are the same or not according to the driver image information of the at least two times;
and if so, executing the step of judging whether the eyeball of the driver changes in the preset time period.
8. The driving assist method according to claim 6,
the driving reminder includes navigation information providing a nearest rest area.
9. An electronic device comprising a memory and a processor coupled to each other;
the memory is used for storing program data;
the processor executes the program data for implementing the method according to any one of claims 1-8.
10. An apparatus having a storage function, characterized in that program data are stored, which program data can be executed to implement the method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811340499.5A CN111169483A (en) | 2018-11-12 | 2018-11-12 | Driving assisting method, electronic equipment and device with storage function |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811340499.5A CN111169483A (en) | 2018-11-12 | 2018-11-12 | Driving assisting method, electronic equipment and device with storage function |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111169483A true CN111169483A (en) | 2020-05-19 |
Family
ID=70646085
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811340499.5A Withdrawn CN111169483A (en) | 2018-11-12 | 2018-11-12 | Driving assisting method, electronic equipment and device with storage function |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111169483A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112699821A (en) * | 2021-01-04 | 2021-04-23 | 长安大学 | Driving early warning method based on driver visual attention prediction |
| CN114162130A (en) * | 2021-10-26 | 2022-03-11 | 东风柳州汽车有限公司 | Driving assistance mode switching method, device, equipment and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101633343A (en) * | 2008-07-23 | 2010-01-27 | 佛山市顺德区顺达电脑厂有限公司 | Portable electronic device having the function of preventing deviation from traveling track and method thereof |
| CN102881116A (en) * | 2011-07-13 | 2013-01-16 | 上海库源电气科技有限公司 | System and method for pre-warning of fatigue driving |
| CN106274693A (en) * | 2016-09-26 | 2017-01-04 | 惠州Tcl移动通信有限公司 | A kind of drive in carry out method, system and the electronic equipment reminded |
| JP2017001548A (en) * | 2015-06-11 | 2017-01-05 | スズキ株式会社 | Driving support apparatus for vehicle |
| CN107640160A (en) * | 2016-07-14 | 2018-01-30 | 奥迪股份公司 | Safe driving of vehicle accessory system and its control method |
| CN108275156A (en) * | 2018-03-27 | 2018-07-13 | 斑马网络技术有限公司 | Driving behavior detection method, storage medium, equipment and vehicle |
| CN108423003A (en) * | 2018-02-08 | 2018-08-21 | 深圳市芝麻开门电子科技有限公司 | A kind of driving safety monitoring method and system |
| CN108657184A (en) * | 2018-07-06 | 2018-10-16 | 京东方科技集团股份有限公司 | Vehicle DAS (Driver Assistant System), method, apparatus and storage medium |
-
2018
- 2018-11-12 CN CN201811340499.5A patent/CN111169483A/en not_active Withdrawn
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101633343A (en) * | 2008-07-23 | 2010-01-27 | 佛山市顺德区顺达电脑厂有限公司 | Portable electronic device having the function of preventing deviation from traveling track and method thereof |
| CN102881116A (en) * | 2011-07-13 | 2013-01-16 | 上海库源电气科技有限公司 | System and method for pre-warning of fatigue driving |
| JP2017001548A (en) * | 2015-06-11 | 2017-01-05 | スズキ株式会社 | Driving support apparatus for vehicle |
| CN107640160A (en) * | 2016-07-14 | 2018-01-30 | 奥迪股份公司 | Safe driving of vehicle accessory system and its control method |
| CN106274693A (en) * | 2016-09-26 | 2017-01-04 | 惠州Tcl移动通信有限公司 | A kind of drive in carry out method, system and the electronic equipment reminded |
| CN108423003A (en) * | 2018-02-08 | 2018-08-21 | 深圳市芝麻开门电子科技有限公司 | A kind of driving safety monitoring method and system |
| CN108275156A (en) * | 2018-03-27 | 2018-07-13 | 斑马网络技术有限公司 | Driving behavior detection method, storage medium, equipment and vehicle |
| CN108657184A (en) * | 2018-07-06 | 2018-10-16 | 京东方科技集团股份有限公司 | Vehicle DAS (Driver Assistant System), method, apparatus and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112699821A (en) * | 2021-01-04 | 2021-04-23 | 长安大学 | Driving early warning method based on driver visual attention prediction |
| CN114162130A (en) * | 2021-10-26 | 2022-03-11 | 东风柳州汽车有限公司 | Driving assistance mode switching method, device, equipment and storage medium |
| WO2023071024A1 (en) * | 2021-10-26 | 2023-05-04 | 东风柳州汽车有限公司 | Driving assistance mode switching method, apparatus, and device, and storage medium |
| CN114162130B (en) * | 2021-10-26 | 2023-06-20 | 东风柳州汽车有限公司 | Driving assistance mode switching method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11597390B2 (en) | Method and system for driving mode switching based on driver's state in hybrid driving | |
| CN112965502B (en) | Visual tracking confirmation method, device, equipment and storage medium | |
| US11609566B2 (en) | Method and system for driving mode switching based on self-aware capability parameters in hybrid driving | |
| US11590890B2 (en) | Method and system for augmented alerting based on driver's state in hybrid driving | |
| JP7080598B2 (en) | Vehicle control device and vehicle control method | |
| KR20210088565A (en) | Information processing devices, mobile devices and methods, and programs | |
| CN113168772A (en) | Information processing apparatus, information processing method, and program | |
| EP4140795A1 (en) | Handover assistant for machine to driver transitions | |
| CN113492849A (en) | Driving support device and data collection system | |
| CN111169483A (en) | Driving assisting method, electronic equipment and device with storage function | |
| EP3727984A1 (en) | Method and system for risk based driving mode switching in hybrid driving | |
| RU2703341C1 (en) | Method for determining hazardous conditions on public roads based on monitoring the situation in the cabin of a vehicle | |
| CA3083411C (en) | Method and system for adapting augmented switching warning | |
| Shahini | Modeling and Analysis of Advanced Driver Assistance Systems in Police Vehicles |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200519 |
|
| WW01 | Invention patent application withdrawn after publication |