CN109672868B - Multi-camera safety early warning system and method, vehicle and terminal equipment - Google Patents

Multi-camera safety early warning system and method, vehicle and terminal equipment Download PDF

Info

Publication number
CN109672868B
CN109672868B CN201910092790.3A CN201910092790A CN109672868B CN 109672868 B CN109672868 B CN 109672868B CN 201910092790 A CN201910092790 A CN 201910092790A CN 109672868 B CN109672868 B CN 109672868B
Authority
CN
China
Prior art keywords
camera
target object
head
angle range
angle
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910092790.3A
Other languages
Chinese (zh)
Other versions
CN109672868A (en
Inventor
赵起超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kingfar International Inc
Original Assignee
Kingfar International Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kingfar International Inc filed Critical Kingfar International Inc
Priority to CN201910092790.3A priority Critical patent/CN109672868B/en
Publication of CN109672868A publication Critical patent/CN109672868A/en
Application granted granted Critical
Publication of CN109672868B publication Critical patent/CN109672868B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G08B21/24—Reminder alarms, e.g. anti-loss alarms

Abstract

The invention provides a multi-camera safety early warning system, a multi-camera safety early warning method, a multi-camera safety early warning vehicle and terminal equipment, wherein the system comprises: a first camera for acquiring a first head image of a target object; the first eye tracker is used for determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range; the second camera is used for acquiring a second head image of the target object; the second eye tracker determines whether the deflection angle of the head of the target object relative to the second camera is within a second preset safety angle range; the early warning device is used for carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range. Through the scheme, the driving safety can be effectively improved, and meanwhile, the technical effect of interference on a driver can be reduced.

Description

Multi-camera safety early warning system and method, vehicle and terminal equipment
Technical Field
The invention relates to the technical field of equipment safety, in particular to a multi-camera safety early warning system, a multi-camera safety early warning method, a multi-camera safety early warning vehicle and terminal equipment.
Background
Along with the higher living standard of people, the utilization rate of automobiles is also higher. The safety of the automobile as a vehicle is greatly related to the attention of the driver, and if the driver turns around, the driver cannot see the road condition ahead, which is easy to cause traffic accidents.
In the normal driving process, because the driver is not concentrated, the traffic accident is very high due to the problems of head turning and the like, and the driving safety is not guaranteed.
In view of the above problems, no effective solution has been proposed.
Disclosure of Invention
The embodiment of the invention provides a multi-camera safety early warning system, a multi-camera safety early warning method, a multi-camera safety early warning vehicle and terminal equipment, and aims to improve the driving safety of the vehicle.
The embodiment of the invention provides a multi-camera safety early warning system, a multi-camera safety early warning method, a multi-camera safety early warning vehicle and terminal equipment, and specifically comprises the following steps:
a multi-camera safety precaution system, comprising:
the first camera is used for acquiring a first head image of the target object under the condition that the target object is in a detection range of the first camera;
the first eye tracker is used for determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
the second camera is used for acquiring a second head image of the target object under the condition that the target object is not in the detection range of the first camera;
the second eye tracker is used for determining whether the deflection angle of the head of the target object relative to the second camera is within a second preset safety angle range or not according to the second head image;
the early warning device is used for carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
In one embodiment, the first camera and the first eye tracker are arranged right in front of a vehicle owner, and the second camera and the second eye tracker are arranged on the right side of the first camera and the second eye tracker.
In one embodiment, the precaution device comprises at least one of: vibration sensor, audio equipment, display lamp.
In one embodiment, the first eye tracker is a tobiiix series eye tracker and the second eye tracker is a tobiiix series eye tracker.
A multi-camera safety early warning method comprises the following steps:
determining whether the target object is within the detection range of the first camera;
acquiring a first head image of the target object under the condition that the target object is within the detection range of the first camera;
determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
under the condition that the target object is not in the detection range of the first camera, acquiring a second head image of the target object;
determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
and carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
In one embodiment, in a case where a deflection angle of the head of the target object with respect to the first camera is not within a first preset safety angle range and a deflection angle of the head of the target object with respect to the second camera is not within a second preset safety angle range, performing a safety precaution includes:
when the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the duration of the deflection angle of the head of the target object relative to the second camera which is not within a second preset safety angle range reaches a first duration, early warning is carried out according to a first early warning mode;
when the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the duration of the deflection angle of the head of the target object relative to the second camera which is not within a second preset safety angle range reaches a second duration, early warning is carried out according to a second early warning mode;
and the second time length is longer than the first time length, and the intensity of the second early warning mode is higher than that of the first early warning mode.
In one embodiment, in the case that the deflection angle of the head of the target object with respect to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object with respect to the second camera is not within a second preset safety angle range, the method further includes:
acquiring the current running speed of the vehicle;
and determining the first time length and the second time length according to the running speed, wherein the higher the running speed is, the shorter the first time length and the second time length are.
In one embodiment, the first pre-warning mode comprises at least one of: visualization progress bar mode, sound mode and vibration mode; the second early warning mode comprises at least one of the following modes: visualization progress bar mode, sound mode, vibrations mode.
In one embodiment, determining whether a deflection angle of the head of the target object relative to the second camera is within a second preset safety angle range according to the second head image comprises:
determining an angle at which the head of the target object is tilted to the right;
acquiring a second preset safety angle range corresponding to the right-inclined angle according to the right-inclined angle;
determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
wherein, different angles of inclination to the right correspond to different second preset safety angle ranges.
In one embodiment, the first preset safety angle range includes: the safe angle scope on the left right direction and the safe angle scope on the upper and lower direction, the safe angle scope of second preset includes: a safety angle range in the left-right direction and a safety angle range in the up-down direction.
In one embodiment, prior to determining whether the target object is within the detection range of the first camera, the method further comprises:
calibrating the spatial position of the head of the target object when the target object is normally driven and directly in front of the target object is seen according to the position of the target object facial feature point relative to the first camera;
and controlling the head of the target object to move so as to determine the first preset safety angle range.
A vehicle, comprising: the multi-camera safety early warning system is provided.
A terminal device comprising a processor and a memory for storing processor-executable instructions, the instructions when executed by the processor implementing the steps of the method of:
determining whether the target object is within the detection range of the first camera;
acquiring a first head image of the target object under the condition that the target object is within the detection range of the first camera;
determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
under the condition that the target object is not in the detection range of the first camera, acquiring a second head image of the target object;
determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
and carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
A computer readable storage medium having stored thereon computer instructions which, when executed, implement the steps of a method comprising:
determining whether the target object is within the detection range of the first camera;
acquiring a first head image of the target object under the condition that the target object is within the detection range of the first camera;
determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
under the condition that the target object is not in the detection range of the first camera, acquiring a second head image of the target object;
determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
and carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
In an embodiment of the present invention, a multi-camera safety pre-warning system is provided, including: first camera, first eye movement appearance, second camera and second eye movement appearance because set up two sets of detecting system, consequently, if eyes are the front of looking forward when the user bows or gets the thing to the right, can not be by the misjudgement as not seeing the place ahead to can reduce the possibility that the existence misjudgement that leads to through only one set of check out test set, thereby can effectively promote driving safety's while, can reduce the technological effect to driver's interference.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:
fig. 1 is a flow chart of a method of a multi-camera security pre-warning method according to an embodiment of the invention;
FIG. 2 is a schematic diagram of the components of a multi-camera security pre-warning system according to an embodiment of the invention;
FIG. 3 is a schematic diagram of 68 facial feature points according to an embodiment of the present invention;
fig. 4 is an architecture diagram of a computer terminal according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail with reference to the following embodiments and accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention, but not to limit the present invention.
In this specification, adjectives such as first and second may only be used to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. References to an element or component or step (etc.) should not be construed as limited to only one of the element, component, or step, but rather to one or more of the element, component, or step, etc., where the context permits.
In the existing vehicle driving process, because the user turns around and the like to cause traffic accidents, if the head deviation degree of the driver can be detected and the driver is reminded under the condition that the head deviation degree is determined to be too high, the attention of the driver during driving can be effectively improved, and the traffic accidents are reduced. Therefore, in the present example, considering that the driver does not need to be reminded when the driver inclines to the right or the like, a method for reducing the occurrence of traffic accidents and reducing the interference to the driver is provided.
Fig. 1 is a flowchart of a method of an embodiment of a multi-camera security early warning method according to the present application. Although the present application provides method operational steps or apparatus configurations as illustrated in the following examples or figures, more or fewer operational steps or modular units may be included in the methods or apparatus based on conventional or non-inventive efforts. In the case of steps or structures which do not logically have the necessary cause and effect relationship, the execution sequence of the steps or the module structure of the apparatus is not limited to the execution sequence or the module structure described in the embodiments and shown in the drawings of the present application. When the described method or module structure is applied in an actual device or end product, the method or module structure according to the embodiments or shown in the drawings can be executed sequentially or executed in parallel (for example, in a parallel processor or multi-thread processing environment, or even in a distributed processing environment).
As shown in fig. 1, in this example, a multi-camera security early warning method is provided, which may include the following steps:
step 101: determining whether the target object is within the detection range of the first camera;
step 102: acquiring a first head image of the target object under the condition that the target object is within the detection range of the first camera;
step 103: determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
step 104: under the condition that the target object is not in the detection range of the first camera, acquiring a second head image of the target object;
step 105: determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
step 106: and carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
In the above example, a multi-camera safety early warning method is provided, only when the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and when the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range, the safety early warning is triggered, because two cameras are arranged, thereby realizing more accurate judgment, avoiding the problem that the safety early warning is carried out because the driver only inclines right, improving the user experience, and avoiding error reminding.
Considering that in the actual driving operation process, the longer the time that the driver turns the head is, the higher the danger is, therefore, different degrees of reminding can be set according to the duration of the turning, for example, if the duration lasts for 1s, the reminding can be omitted, 3s can be simply reminded, 5s or more can be reminded with high intensity, so that the reminding with different intensities based on the duration is realized, and the height of the current safety factor is also represented. For example, when performing safety warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range, warning may be performed according to a first warning manner under the condition that the deflection angle of the head of the target object relative to the first camera is not within the first preset safety angle range and the duration of the deflection angle of the head of the target object relative to the second camera being not within the second preset safety angle range reaches a first duration; when the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the duration of the deflection angle of the head of the target object relative to the second camera which is not within a second preset safety angle range reaches a second duration, early warning is carried out according to a second early warning mode; and the second time length is longer than the first time length, and the intensity of the second early warning mode is higher than that of the first early warning mode.
However, it is to be noted that the above-listed durations and the like are only an exemplary description and may include: the first time period and the second time period may further include: a third time period, a fourth time period, and so on. The specific setting of several security levels can be set according to actual needs and conditions, which is not limited in the present application.
Considering that the time for allowing the driver to be distracted is short under the condition that the running speed of the vehicle is high, therefore, under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range, the current running speed of the vehicle can be obtained in the process of carrying out safety early warning; and determining the first time length and the second time length according to the running speed, wherein the higher the running speed is, the shorter the first time length and the second time length are.
That is, different time periods may be set for different operation speeds, and the higher the operation speed is, the shorter the time period threshold is set, so that the safety of driving may be improved.
The first warning manner may include, but is not limited to, at least one of the following: visualization progress bar mode, sound mode and vibration mode; the second warning manner may include, but is not limited to, at least one of the following: visualization progress bar mode, sound mode, vibrations mode.
However, it should be noted that the specific manner of the first warning manner and the second warning manner listed above is only an exemplary description, and in practical implementation, the warning may be performed in other manners, and the warning manner may be any manner that can remind the driver of the attention, and the present application is not limited thereto.
When a driver sits in a driving seat to realize body inclined side detection, the whole movement of the left and right positions of the buttocks cannot occur, the buttocks are inclined by taking the buttocks as an axis, the upper body and the head are inclined together, and the height of the upper body of each person is fixed, so that the human body sitting type inclined side detection is similar to the rotation of a rod around a fixed axis. The connecting line of the two shoulders is vertical to the body direction, when the body is obliquely laterally probed, the inclined angle alpha of the upper half body relative to the vertical direction can be obtained through the included angle beta between the connecting line of the two shoulders and the vertical direction, and the alpha is 90-beta. Since the height of the upper half of the human body is constant, when the driver sits in the driving seat to realize body inclination and lateral exploration, the whole movement of the left and right positions of the hip does not occur, the hip is inclined with the hip as an axis, and the upper half and the head are inclined together, so that the position of the head is determined after the inclination angle alpha of the upper half relative to the vertical direction is determined.
Accordingly, for different inclination angles, a different second preset safety angle range may be determined, and accordingly, determining whether a deflection angle of the head of the target object with respect to the second camera is within the second preset safety angle range according to the second head image may include: determining an angle at which the head of the target object is tilted to the right; acquiring a second preset safety angle range corresponding to the right-inclined angle according to the right-inclined angle; determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image; wherein, different angles of inclination to the right correspond to different second preset safety angle ranges.
In one embodiment, the first preset safety angle range may include: the safety angle range in the left-right direction and the safety angle range in the up-down direction, the second preset safety angle range may include: a safety angle range in the left-right direction and a safety angle range in the up-down direction. That is, the determination can be made from two dimensions of the yaw in the left-right direction and the yaw in the up-down direction.
Because the heights or sitting heights of different vehicles or different driving vehicles are different, in order to enable the detection result to be more accurate, the spatial position of the head of the target object when the target object is normally driven and directly in front of the target object can be calibrated according to the position of the facial feature point of the target object relative to the camera before the head image of the target object is acquired; the head of the target object can be controlled to move to determine the preset safety angle range. That is, the position and the preset safe angle range are calibrated for each driver of each vehicle. Of course, the same set of preset safety angles can be adopted, but the accuracy can be effectively improved through personalized calibration.
If an individualized calibration mode is adopted, a memory and a face recognition function can be provided, calibration is carried out on each driver, the camera position, the first preset safety angle range, the second preset safety angle range and the like corresponding to the driver are stored, when a user drives, the identity of the current driver is recognized through the face, and then corresponding calibration information is called for setting.
In this example, a multi-camera safety precaution system is also provided, as shown in fig. 2, may include:
a first camera 201, which can be used to acquire a first head image of a target object when the target object is within the detection range of the first camera;
a first eye tracker 202, configured to determine, according to the first head image, whether a deflection angle of the head of the target object with respect to the first camera is within a first preset safety angle range;
the second camera 203 can be used for acquiring a second head image of the target object under the condition that the target object is not in the detection range of the first camera;
a second eye tracker 204, configured to determine, according to the second head image, whether a deflection angle of the head of the target object with respect to the second camera is within a second preset safety angle range;
the early warning device 205 may be configured to perform a safety early warning when the deflection angle of the head of the target object with respect to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object with respect to the second camera is not within a second preset safety angle range.
The first camera 201 and the first eye tracker 202 are arranged right in front of the vehicle owner, and the second camera 203 and the second eye tracker 204 are arranged on the right side of the first camera and the second eye tracker.
Wherein, the above-mentioned early warning device may include but is not limited to at least one of the following: vibration sensor, audio equipment, display lamp. For example, the first eye tracker may be a tobiiix-series eye tracker, and the second eye tracker may be a tobiiix-series eye tracker.
The above method is described below with reference to a specific example, however, it should be noted that the specific example is only for better describing the present application and is not to be construed as limiting the present application.
In this example, a driver attention tracking feedback system based on facial recognition is provided, based on which the head position of the driver can be recognized by recognizing 68 feature points of the face by a facial expression camera; judging the safety angle ranges of the driver relative to the right front, the upper, the lower, the left and the right through the eye tracker; and judging whether safety warning feedback is given to the driver according to comparison between the vertical and horizontal deflection positions of the head relative to the front and the vertical and horizontal safety angles and the accumulated deflection duration. The system and method are described in detail below:
1) the arrangement of the facial feature points is as follows:
as shown in fig. 3, the human face may be provided with 68 feature points, wherein the 68 feature points may be respectively 5 points of the left and right eyebrows, which are sequentially arranged according to the positions of the eyebrows, and are respectively LB1, LB2, LB3, LB4, LB5, RB1, RB2, RB3, RB4, RB5 from the inside to the outside; 6 points of the left eye and the right eye respectively are 2 points of the upper eyelid and the lower eyelid of each eye and 1 point of the left canthus and the right canthus respectively, and are LE1, LE2, LE3, LE4, LE5, LE6, RE1, RE2, RE3, RE4, RE5 and RE6 respectively; the nose has 9 points, namely 4 points of the nose bridge, N1, N2, N3 and N4 from top to bottom, and 5 points at the lower part of the nostril, namely N5, N6, N7, N8 and N9 from left to right; 20 points of the mouth are respectively 7 points of the upper lip and the lower lip, namely UL1, UL2, UL3, UL4, UL5, UL6, UL7, LL1, LL2, LL3, LL4, LL5, LL6 and LL7 from left to right, and 3 points of the upper tooth and the lower tooth are respectively UT1, UT2, UT3, LT1, LT2 and LT3 from left to right; the cheek points of 17 are C1, C2, C3, C4, C5, C6, C7, C8, C9, C10, C11, C12, C13, C14, C15, C16 and C17 from left to right; wherein the points N1, N2, N3, N4, N7, UL4, LL4, UT2, LT2 and C9 are on a straight line. By 68 face feature points, the face can be effectively recognized.
2) The system platform may include three modules, which are: system setting module, parameter control module, real-time status display module, wherein:
and the system setting module is used for setting the configuration of the whole system, identifying a facial expression camera, an eye tracker, a vibration sensor, feedback audio and the like.
The parameter control module is used for setting a feedback mode in the whole feedback process, and four test stages can be preset according to time in the feedback stage: a first stage, a second stage, a third stage and a fourth stage; the first stage is a safe driving stage, the deflection range of the tested head is allowed to exceed a safe angle range in the stage, safety warning is not needed, the later three stages can be regarded as a non-safe driving stage, and when the accumulated deflection time is in the later three stages, different safety warning can be given according to different feedback settings so as to prompt a tested person.
Wherein, the feedback form may include but is not limited to at least one of the following: the vibration sensor, the visual progress bar, the sound warning and the like are triggered, and in the actual feedback process, prompt feedback with different intensities can be carried out according to the difference of duration when a testee does not watch the front.
To achieve the above-mentioned needs, the driver attention tracking feedback system based on facial recognition may include: attention is paid to a tracking software platform, a face recognition behavior camera, a vibration sensor, an external sound playing device and a Tobii X series eye tracker, and the left and right measurable ranges of the eye tracker can be set as follows: width by height was about 50 by 36cm (24 by 14 °). The camera, the vibration sensor, the sound equipment, the eye tracker and the like can be inserted into the equipment where the software platform is located through a USB interface.
However, it should be noted that the components included in the driver attention tracking feedback system based on facial recognition are only an exemplary description, and several of the components may be selected, other components may be adopted, or other types of component arrangements may be adopted in practical implementation, and the present application is not limited thereto.
Specifically, the tracking reminding can be performed according to the following steps:
s1: face recognition, head position determination:
place the face identification camera in the head dead ahead, calibrate the relative position of head for the camera, specifically, can calibrate according to following mode:
taking the camera position as a space origin 0(0,0, 0) point, placing the camera right in front of the head, calculating the space position of the head by calculating the position of an N4 point in the facial feature points relative to the camera, and taking N4 point coordinates as N4N (Xn, Yn, Zn), and taking N40(X0, Y0, Z0) as the position of the head relative to the origin of the camera through calibration. Here, the position N40(X0, Y0, Z0) of the head with respect to the origin of the camera means: the spatial position of the head was observed right ahead while the test was driving normally.
S2: head safety angle range determination:
the eye tracker is placed right in front of the tested head, and the safety angle range of the deflection of the tested head is calculated. Calculating a first safety angle range of head deflection by moving the head left, right, up, and down, the specific calculation method is as follows:
1) determining X, Y, Z value of the head in space position, wherein X, Y, Z value is the position of the head relative to the position of the camera, wherein the deflection angle of the head in the left-right direction is calculated by X, the deflection angle of the head in the up-down direction is calculated by Y, the distance of the head from the camera is calculated by Z, and n is each sampling point. For example, the distance of the head with respect to the camera may be set to be about 500 mm; the specific distance can be determined according to actual conditions, and the application does not limit the specific distance.
The deflection angle of the head with respect to the camera is an angle between a connection line between the camera and the point N4, a connection line between the camera and N40(X0, Y0, Z0), and two connection lines at a certain sampling time.
2) Assuming that the deflection angle of the head with respect to the camera is Qn (an, bn), where an is the deflection angle in the left-right direction and bn is the deflection angle in the up-down direction, after calibration is completed, the deflection angle of the origin position N40(X0, Y0, Z0) of the head with respect to the camera is Q0(0,0), that is, the deflection angle of the head with respect to the camera is Q0(0, 0);
3) recording the head position N4N (Xn, Yn, Zn) of each sample in real time;
4) the method includes the steps of calculating the deflection angle of each frame head relative to the camera in real time, specifically, calculating according to the following formula:
an＝arccos[(X0-0)2+(Xn-0)2-(Xn-X0)2]/[2(X0-0)(Xn-0)]＝arccos[X02+Xn2-(Xn-X0)2]/(2X0Xn)，
bn＝arccos[Y02+Yn2-(Yn-Y0)2]/(2Y0Yn)；
5) the boundary head position when the tested gazing is ahead is obtained through the eye tracker, and the angle between the boundary head position and the camera is determined, so that the first safety angle range is determined.
And determining the angle of the eye tracker when the eye can not capture the eyes when the head deflects leftwards, rightwards, upwards and downwards, wherein the head left-right deviation safety angle range is A (L1 and R1), and the head up-down deviation safety angle range is B (U1 and D1).
The eye tracker is arranged in front of a testee, and is realized by debugging, when the testee watches the front, the eye tracker can always capture the eyes of the testee, and when the testee deflects the head leftwards or rightwards and does not see the front, the eye tracker can not capture the tested eyes completely.
When the head of the subject is deflected to the left and the eye tracker does not capture the eyes at all, the position of the subject's head at this time, i.e., the position of N4 point, is recorded as the left boundary head position, and the angle between the N4 point and the camera at this time is calculated as the left interference start angle L1.
When the head of the subject is deflected to the right and the eye tracker does not capture the eyes at all, the position of the subject's head at this time, i.e., the position of N4 point, is recorded as the right boundary head position, and the angle between the N4 point and the camera at this time is calculated as the right interference start angle R1.
The head left-right deviation safety angle range A (L1, R1) is between the left interference starting angle and the right interference starting angle, namely, when the angle between the tested head position and the camera is in the safety angle range, the tested head is considered to be in the front of the fixation, and the normal driving is considered.
Similarly, the upper and lower deviation safety angle range is B (U1, D1).
That is, the head left-right deviation safety angle range a (L1, R1) and the head up-down deviation safety angle range B (U1, D1) can be determined in the above manner, and the determined a (L1, R1) and B (U1, D1) are taken as the first safety angle range.
S3: judging and executing safety warning feedback:
calculating the deflection angle of each frame head relative to the camera in real time by the calculating method of the step S2 4), and judging whether an and bn of the head deflection angles Qn (an and bn) are within an angle range of A, B in real time, if so, not performing early warning, and if not, performing early warning feedback at different stages by accumulating along with time.
Specifically, the feedback phase in which the head-deflecting angle is not fed back within the first safety angle range may be set to a plurality of phases, for example, may be set to 4 phases: first phase-safe phase, second phase-unsafe 1 phase, third phase-unsafe 2 phase and fourth phase-unsafe 3 phase, wherein the first phase does not feedback (because the duration is shorter, the danger is less), and the other three phases can freely combine the feedback modes, for example: the early warning can be fed back through one or more of visual progress bar, sound, vibration and the like.
For example, the subject's head-deflecting action is continued for a certain time, i.e., in the first stage, in which no feedback is given. If the tested head deflection action continues continuously, the first stage is ended, and the second stage is entered; if the tested head deflection action continues, ending the second stage and entering a third stage; if the subject head deflection motion continues, the third phase ends and the fourth phase is entered. Feedback is given in the second stage, the third stage and the fourth stage.
When the deflection action of the tested head is finished and the normal angle is recovered, the timing of the feedback stage returns to zero. When the tested head deflection action occurs again, the timing is restarted.
For example, if the duration of the first phase is 1s and the duration of the second phase is 5s, the second phase is performed when the deflection angle of the tested head is out of the first safety angle range and the duration of the deflection angle is 2 s; if the head deflection time is returned to the normal angle, i.e., within the first safety angle range, at 6s, the timing is reset to zero. Then again the head deviation angle is calculated for the total time and again counted from 0.
In the second stage, the third stage, and the fourth stage, the feedback strength may be increased in sequence. Selecting a visual feedback mode, and setting visual interval time, wherein the display mode can comprise visual progress bar gradient display, visual flicker display and visual gradual change display; selecting a sound warning feedback mode, and setting sound warning feedback interval time; selecting a vibration feedback mode, setting a vibration feedback time interval and the like.
The duration of each of the first three phases may be determined in combination with the current vehicle speed, and the respective durations of the first three phases may be set to be different, for example, the greater the vehicle speed, the shorter the duration of each phase, and thus the shorter the duration of the second, third and fourth phases.
For example, if the vehicle speed is 10km/h, the duration of the first stage is 2 seconds, and the duration of the second stage is 3 seconds; when the vehicle speed is 50km/h, the duration time of the first stage is 1 second, and the duration time of the second stage is 2 seconds; since the faster the vehicle speed, the shorter the time required to travel a certain distance, and the shorter the time required to generate a hazard, the more necessary it is for the driver to return to the normal attitude as soon as possible.
Through the attention tracking feedback system, on one hand, a warning feedback can be provided for a driver by monitoring the duration time of the relative deflection angle of the head deviating from the right front under different vehicle speeds, so that the attention concentration of the driver is prevented from being influenced by the interference of other stimuli, and the occurrence of traffic accidents is reduced; on the other hand, the safety early warning time ranges in different stages are set by acquiring different vehicle speeds, so that an early warning feedback mechanism can be given in real time according to the change of the vehicle speed by monitoring the vehicle speed in real time in the processes of safe driving and automatic driving.
Further, consider that sometimes the driver may tilt his/her side to leave the normal driving position and get access to the contents of the cab, for example: an article placed at or in front of the passenger seat. After the probe body reaches a certain position, the head part is fixed at the left and right positions and further rotates to observe the front of the vehicle body or other objects.
In this case, the eyes of the driver may or may not be looking ahead, the first camera located right ahead of the normal driving position of the driver detects that the deflection angle of the head of the driver relative to the first camera exceeds the first safety angle range (i.e. the safety angle range of the driver in the normal driving position), and the head completely departs from the visible range of the first camera, and the feedback system can perform feedback at a certain time according to the above-mentioned early warning rule. Then, if the eyes of the driver are looking forward at this time, the detection and feedback are not in accordance with the actually required judgment requirement, that is, the above-mentioned feedback system cannot correctly reflect whether the driver is looking forward or not.
Because camera, eye tracker are close apart from driver's installation distance, the detection error problem in the detection range of camera, eye tracker itself and the wide field of vision scope, consequently, only detect whether the head deflection angle of driver in the great scope is in safe angle range through camera, eye tracker, and then judge whether the driver is stared at the place ahead, realize that the driver notices and follows the pursuit feedback, is defective. Therefore, the method for joint detection by using multiple sets of equipment is provided in the embodiment, so that the aim of accurate detection is fulfilled.
Specifically, in this example, a tracking system is provided, which may include: the first camera, the first eye tracker, the second camera and the second eye tracker.
The first camera and the first eye tracker are used for attention tracking of a normal driving position of a driver, the second camera and the second eye tracker are arranged on the right side of the first camera and the first eye tracker and used for moving rightwards to be separated from a first safety angle range and then detecting through the second camera and the second eye tracker when entering a detection range of the second camera and the second eye tracker. When the device enters the detection range of the second camera and the second eye tracker, the first camera and the first eye tracker do not perform feedback and do not act, and the second set of equipment performs feedback and act. The left detection range of the second camera and the second eye tracker partially overlaps the right detection range of the first camera and the first eye tracker.
The first camera and the first eye tracker, the second camera and the second eye tracker, as long as one set of equipment detects that the driver is watching the front, the driver is judged to be watching the front.
First, as analyzed from the behavior habits, when the driver sits at the driving position to perform the body inclination sidetracking, the entire movement of the left and right positions of the hip is not generated, the hip is inclined as an axis, the upper body and the head are inclined together, and the height of the upper body of each person is constant, so that the driver, like a single rod, rotates around the fixed axis when sitting in the human body. The connecting line of the two shoulders is vertical to the body direction, when the body is obliquely laterally probed, the inclined angle alpha of the upper half body relative to the vertical direction can be obtained through the included angle beta between the connecting line of the two shoulders and the vertical direction, and the alpha is 90-beta. Since the height of the upper half of the human body is constant, when the driver sits in the driving seat to realize body inclination and lateral exploration, the whole movement of the left and right positions of the hip does not occur, the hip is inclined with the hip as an axis, and the upper half and the head are inclined together, so that the position of the head is determined after the inclination angle alpha of the upper half relative to the vertical direction is determined.
The second camera position may be used as the second spatial origin 0(0,0, 0) point when performing face recognition and head position determination. When the upper half body is inclined at an angle α of C to the right with respect to the vertical direction and the driver's eyes are looking forward, the point N4 on the head is set to N4 by calibrationC0(XC0，YC0，ZC0) The position of the head relative to the origin of the second camera is counted.
Specifically, the origin position of the head with respect to the second camera at the angle α of various values may be recorded, for example: alpha is respectively 10 degrees, 15 degrees and 20 degrees, 80 degrees, the head is in the original position N4 relative to the second cameraα0(Xα0，Yα0，Zα0). Wherein, the value interval of alpha can be determined according to the actual conditionIt needs to be determined that, for example, the value may be 5 °, that is, an angle is taken every 5 °, and the origin position of a head with respect to the second camera is determined.
When implemented, for each sampling angle, a second safety angle range is determined, i.e. the second safety angle range when the recording angle α is of various values is determined separately. For example, a second safety angle range is determined for α of 10 °, and a second safety angle range is determined for α of 15 °.
After the second safe angle range of each angle is determined, safety warning feedback can be carried out based on the determined second safe angle range. Since the plurality of angles α have been determined, the head position relative to the origin of the second camera and the second safety angle range. Therefore, the body inclination angle alpha is judged in real time, whether the head deflection angle is in the second safety angle range or not is determined, if yes, early warning is not carried out, and if not, early warning feedback in different stages is carried out.
Further, since it is considered that the driver's reaction is slow due to the inclination of the body, the early warning feedback can be performed from the first stage.
The method provided by the embodiment of the application can be executed in a mobile terminal, a computer terminal or a similar operation device. Taking the operation on a computer terminal as an example, fig. 4 is a block diagram of a hardware structure of a computer terminal of the multi-camera security early warning method according to the embodiment of the present invention. As shown in fig. 4, the computer terminal 10 may include one or more (only one shown) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions. It will be understood by those skilled in the art that the structure shown in fig. 4 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 4, or have a different configuration than shown in FIG. 1.
The memory 104 may be configured to store software programs and modules of application software, such as program instructions/modules corresponding to the multi-camera safety precaution method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the multi-camera safety precaution method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission module 106 is used to receive or transmit data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a Network adapter (NIC) that can be connected to other Network devices through a base station to communicate with the internet. In one example, the transmission module 106 may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.
In a software aspect, the safety precaution device may include:
the first determining module is used for determining whether the target object is in the detection range of the first camera;
the first acquisition module is used for acquiring a first head image of the target object under the condition that the target object is in the detection range of the first camera;
the second determining module is used for determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
the second acquisition module is used for acquiring a second head image of the target object under the condition that the target object is not in the detection range of the first camera;
a third determining module, configured to determine, according to the second head image, whether a deflection angle of the head of the target object with respect to the second camera is within a second preset safety angle range;
and the early warning module is used for carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
In an embodiment, the early warning module may specifically perform early warning in a first early warning manner when a deflection angle of the head of the target object with respect to the first camera is not within a first preset safety angle range, and a duration of the deflection angle of the head of the target object with respect to the second camera being not within a second preset safety angle range reaches a first duration; when the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the duration of the deflection angle of the head of the target object relative to the second camera which is not within a second preset safety angle range reaches a second duration, early warning is carried out according to a second early warning mode; and the second time length is longer than the first time length, and the intensity of the second early warning mode is higher than that of the first early warning mode.
In an embodiment, the early warning module may further acquire a current running speed of the vehicle during a safety early warning process when a deflection angle of the head of the target object with respect to the first camera is not within a first preset safety angle range and a deflection angle of the head of the target object with respect to the second camera is not within a second preset safety angle range; and determining the first time length and the second time length according to the running speed, wherein the higher the running speed is, the shorter the first time length and the second time length are.
In one embodiment, the first warning manner may include, but is not limited to, at least one of the following: visualization progress bar mode, sound mode and vibration mode; the second warning manner may include, but is not limited to, at least one of the following: visualization progress bar mode, sound mode, vibrations mode.
In one embodiment, the third determining module may specifically determine an angle at which the head of the target object is tilted to the right; acquiring a second preset safety angle range corresponding to the right-inclined angle according to the right-inclined angle; determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image; wherein, different angles of inclination to the right correspond to different second preset safety angle ranges.
In one embodiment, the first preset safety angle range may include: the safety angle range in the left-right direction and the safety angle range in the up-down direction, the second preset safety angle range may include: a safety angle range in the left-right direction and a safety angle range in the up-down direction.
In one embodiment, before determining whether the target object is within the detection range of the first camera, the second determination module may calibrate the spatial position of the head when the target object is driving normally and looking straight ahead according to the position of the target object facial feature point relative to the first camera; and controlling the head of the target object to move so as to determine the first preset safety angle range.
An embodiment of the present application further provides a specific implementation manner of an electronic device, which can implement all steps in the multi-camera safety warning method in the foregoing embodiment, where the electronic device specifically includes the following contents:
a processor (processor), a memory (memory), a communication Interface (Communications Interface), and a bus;
the processor, the memory, and the communication interface complete mutual communication through the bus, and the processor is configured to call a computer program in the memory, where the processor implements all steps in the above embodiments when executing the computer program, for example, the processor implements the following steps when executing the computer program:
step 1: determining whether the target object is within the detection range of the first camera;
step 2: acquiring a first head image of the target object under the condition that the target object is within the detection range of the first camera;
and step 3: determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
and 4, step 4: under the condition that the target object is not in the detection range of the first camera, acquiring a second head image of the target object;
and 5: determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
step 6: and carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
It can be known from the above description that, this application embodiment only is not in first preset safe angle within range at the deflection angle of target object head for first camera, and, just, the target object head just can trigger and carry out safety precaution under the condition that the deflection angle of second camera is not in second preset safe angle within range, because through having set up two cameras, thereby can realize more accurate judgement, can avoid just carrying out safety precaution's problem because the driver just inclines right, can promote user experience, avoid the wrong warning.
An embodiment of the present application further provides a computer-readable storage medium capable of implementing all the steps in the multi-camera safety precaution method in the foregoing embodiment, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements all the steps of the multi-camera safety precaution method in the foregoing embodiment, for example, when the processor executes the computer program, implements the following steps:
step 1: determining whether the target object is within the detection range of the first camera;
step 2: acquiring a first head image of the target object under the condition that the target object is within the detection range of the first camera;
and step 3: determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
and 4, step 4: under the condition that the target object is not in the detection range of the first camera, acquiring a second head image of the target object;
and 5: determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
step 6: and carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range.
It can be known from the above description that, this application embodiment only is not in first preset safe angle within range at the deflection angle of target object head for first camera, and, just, the target object head just can trigger and carry out safety precaution under the condition that the deflection angle of second camera is not in second preset safe angle within range, because through having set up two cameras, thereby can realize more accurate judgement, can avoid just carrying out safety precaution's problem because the driver just inclines right, can promote user experience, avoid the wrong warning.
For convenience of description, the above devices are described as being divided into various modules by functions, and are described separately. Of course, in implementing the embodiments of the present description, the functions of each module may be implemented in one or more software and/or hardware, or a module implementing the same function may be implemented by a combination of multiple sub-modules or sub-units, and the like. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
Those skilled in the art will also appreciate that, in addition to implementing the controller as pure computer readable program code, the same functionality can be implemented by logically programming method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers and the like. Such a controller may therefore be considered as a hardware component, and the means included therein for performing the various functions may also be considered as a structure within the hardware component. Or even means for performing the functions may be regarded as being both a software module for performing the method and a structure within a hardware component.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
As will be appreciated by one skilled in the art, embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, embodiments of the present description may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present description may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.
The embodiments of this specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The described embodiments may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for the system embodiment, since it is substantially similar to the method embodiment, the description is simple, and for the relevant points, reference may be made to the partial description of the method embodiment. In the description herein, references to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of an embodiment of the specification. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
The above description is only an example of the embodiments of the present disclosure, and is not intended to limit the embodiments of the present disclosure. Various modifications and variations to the embodiments described herein will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present specification should be included in the scope of the claims of the embodiments of the present specification.
The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention, and various modifications and changes may be made to the embodiment of the present invention by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (9)

1. The utility model provides a many cameras safety precaution system which characterized in that includes:
the first camera is used for acquiring a first head image of the target object under the condition that the target object is in a detection range of the first camera;
the first eye tracker is used for determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
the second camera is used for acquiring a second head image of the target object under the condition that the target object is not within the first preset safety angle range;
the second eye tracker is used for determining whether the deflection angle of the head of the target object relative to the second camera is within a second preset safety angle range or not according to the second head image;
the early warning device is used for carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range;
the second eye tracker is specifically used for determining the right inclination angle of the head of the target object; acquiring a second preset safety angle range corresponding to the right-inclined angle according to the right-inclined angle; determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image; wherein, different angles of inclination to the right correspond to different second preset safety angle ranges.
2. The system of claim 1, wherein the first camera and the first eye tracker are disposed directly in front of a vehicle owner's ride, and the second camera and the second eye tracker are disposed to the right of the first camera and the second eye tracker.
3. A multi-camera safety early warning method is characterized by comprising the following steps:
determining whether the target object is within the detection range of the first camera;
acquiring a first head image of the target object under the condition that the target object is within the detection range of the first camera;
determining whether the deflection angle of the head of the target object relative to the first camera is within a first preset safety angle range or not according to the first head image;
under the condition that the target object is not within the first preset safety angle range, acquiring a second head image of the target object;
determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
carrying out safety early warning under the condition that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range;
determining whether the deflection angle of the head of the target object relative to the second camera is within a second preset safety angle range, including:
determining an angle at which the head of the target object is tilted to the right;
acquiring a second preset safety angle range corresponding to the right-inclined angle according to the right-inclined angle;
determining whether the deflection angle of the head of the target object relative to a second camera is within a second preset safety angle range or not according to the second head image;
wherein, different angles of inclination to the right correspond to different second preset safety angle ranges.
4. The method of claim 3, wherein in the case that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range, performing a safety precaution comprises:
when the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the duration of the deflection angle of the head of the target object relative to the second camera which is not within a second preset safety angle range reaches a first duration, early warning is carried out according to a first early warning mode;
when the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the duration of the deflection angle of the head of the target object relative to the second camera which is not within a second preset safety angle range reaches a second duration, early warning is carried out according to a second early warning mode;
and the second time length is longer than the first time length, and the intensity of the second early warning mode is higher than that of the first early warning mode.
5. The method according to claim 4, wherein in the case that the deflection angle of the head of the target object relative to the first camera is not within a first preset safety angle range, and the deflection angle of the head of the target object relative to the second camera is not within a second preset safety angle range, the method further comprises:
acquiring the current running speed of the vehicle;
and determining the first time length and the second time length according to the running speed, wherein the higher the running speed is, the shorter the first time length and the second time length are.
6. The method of claim 3, wherein the first preset safe angle range comprises: the safe angle scope on the left right direction and the safe angle scope on the upper and lower direction, the safe angle scope of second preset includes: a safety angle range in the left-right direction and a safety angle range in the up-down direction.
7. The method of claim 3, wherein prior to determining whether the target object is within the detection range of the first camera, the method further comprises:
calibrating the spatial position of the head of the target object when the target object is normally driven and directly in front of the target object is seen according to the position of the target object facial feature point relative to the first camera;
and controlling the head of the target object to move so as to determine the first preset safety angle range.
8. A vehicle, characterized by comprising: the multi-camera safety precaution system of any one of claims 1 to 2.
9. A terminal device comprising a processor and a memory for storing processor-executable instructions which, when executed by the processor, implement the steps of the method of any one of claims 3 to 7.
CN201910092790.3A 2019-01-30 2019-01-30 Multi-camera safety early warning system and method, vehicle and terminal equipment Active CN109672868B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910092790.3A CN109672868B (en) 2019-01-30 2019-01-30 Multi-camera safety early warning system and method, vehicle and terminal equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910092790.3A CN109672868B (en) 2019-01-30 2019-01-30 Multi-camera safety early warning system and method, vehicle and terminal equipment

Publications (2)

Publication Number Publication Date
CN109672868A CN109672868A (en) 2019-04-23
CN109672868B true CN109672868B (en) 2020-10-27

Family

ID=66150008

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910092790.3A Active CN109672868B (en) 2019-01-30 2019-01-30 Multi-camera safety early warning system and method, vehicle and terminal equipment

Country Status (1)

Country Link
CN (1) CN109672868B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113923349A (en) * 2021-09-02 2022-01-11 深圳壹秘科技有限公司 Camera privacy protection method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102012222519A1 (en) * 2011-12-09 2013-06-13 Denso Corporation DRIVING STATE DETERMINATION DEVICE
WO2014006909A3 (en) * 2012-07-06 2014-02-27 Yazaki Corporation Driver alertness detecting device and driver alertness informing device
CN203651621U (en) * 2013-12-31 2014-06-18 北京兴科迪科技有限公司 Inner rearview mirror with fatigue driving prompting device and vehicle with the inner rearview mirror
CN105469035A (en) * 2015-11-17 2016-04-06 中国科学院重庆绿色智能技术研究院 Driver's bad driving behavior detection system based on binocular video analysis
CN106709420A (en) * 2016-11-21 2017-05-24 厦门瑞为信息技术有限公司 Method for monitoring driving behaviors of driver of commercial vehicle

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2414059A1 (en) * 2002-12-12 2004-06-12 Pano Yiotis Patrikakis Retractable blind spot mirror
US20170210289A1 (en) * 2016-01-22 2017-07-27 Arjun Kundan Dhawan Driver Focus Analyzer
JP2018127084A (en) * 2017-02-08 2018-08-16 トヨタ自動車株式会社 Automatic drive vehicle

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102012222519A1 (en) * 2011-12-09 2013-06-13 Denso Corporation DRIVING STATE DETERMINATION DEVICE
WO2014006909A3 (en) * 2012-07-06 2014-02-27 Yazaki Corporation Driver alertness detecting device and driver alertness informing device
CN203651621U (en) * 2013-12-31 2014-06-18 北京兴科迪科技有限公司 Inner rearview mirror with fatigue driving prompting device and vehicle with the inner rearview mirror
CN105469035A (en) * 2015-11-17 2016-04-06 中国科学院重庆绿色智能技术研究院 Driver's bad driving behavior detection system based on binocular video analysis
CN106709420A (en) * 2016-11-21 2017-05-24 厦门瑞为信息技术有限公司 Method for monitoring driving behaviors of driver of commercial vehicle

Also Published As

Publication number Publication date
CN109672868A (en) 2019-04-23

Similar Documents

Publication Publication Date Title
US7460940B2 (en) Method and arrangement for interpreting a subjects head and eye activity
US20200334477A1 (en) State estimation apparatus, state estimation method, and state estimation program
CN103110400B (en) The vision input of vehicle driver
CN103318023B (en) Vehicle-mounted real-time intelligent fatigue monitoring and auxiliary device
EP2032034B1 (en) Method for determining and analyzing a location of visual interest
US9526448B2 (en) State estimation device and state estimation program
CN101877051A (en) Driver attention state monitoring method and device
WO2020029444A1 (en) Method and system for detecting attention of driver while driving
US11715333B2 (en) Human monitoring system incorporating calibration methodology
EP3440592B1 (en) Method and system of distinguishing between a glance event and an eye closure event
WO2019198179A1 (en) Passenger state determination device, alarm output control device, and passenger state determination method
CN109672868B (en) Multi-camera safety early warning system and method, vehicle and terminal equipment
CN109859438B (en) Safety early warning method, system, vehicle and terminal equipment
CN113838265B (en) Fatigue driving early warning method and device and electronic equipment
CN113544471A (en) DMS device performance evaluation system, method, storage medium and electronic device
Zhang et al. Driver fatigue state detection based on facial key points
Reimer et al. Detecting eye movements in dynamic environments
CN105894732A (en) Fatigue driving monitoring system
Boverie et al. Driver vigilance diagnostic based on eyelid movement observation
Anumas et al. Driver fatigue monitoring system using video face images & physiological information
DE102019109491A1 (en) DATA PROCESSING DEVICE, MONITORING SYSTEM, WECKSYSTEM, DATA PROCESSING METHOD AND DATA PROCESSING PROGRAM
CN108111801A (en) The heatproof pilot system and method for ADAS vision processing systems
CN113095108A (en) Fatigue driving detection method and device
WO2020255238A1 (en) Information processing device, program, and information processing method
Jia An Analysis ofDriver Cognitive Distraction

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant