CN104504378A - Method and device for detecting image information - Google Patents
Method and device for detecting image information Download PDFInfo
- Publication number
- CN104504378A CN104504378A CN201410838292.6A CN201410838292A CN104504378A CN 104504378 A CN104504378 A CN 104504378A CN 201410838292 A CN201410838292 A CN 201410838292A CN 104504378 A CN104504378 A CN 104504378A
- Authority
- CN
- China
- Prior art keywords
- human eye
- video
- frame
- eye area
- num
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Ophthalmology & Optometry (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
An embodiment of the invention discloses a method and a device for detecting image information, and relates to the field of technologies for processing images. The method includes determining human eye regions in various video frames in video segments with preset durations; respectively computing light stream information of various pixel points in the human eye regions in the various video frames in the video segments; respectively counting the quantities of the downwardly moving pixel points and the upwardly moving pixel points in the human eye regions of the various video frames in the video segments according to the computed light stream information; acquiring human eye state characteristics of the various video frames in the video segments according to counting results; detecting to determine whether blinking phenomena are available in the video segments or not according to the acquired human eye state characteristics. The light stream information of the various pixel points in the human eye regions in each video frame in the corresponding video segment corresponds to a front frame of the video frame. According to the scheme, the method and the device in the embodiment of the invention have the advantages that the human eye state characteristics of the video frames are acquired by the aid of the image information among the video frames, and are utilized when whether the blinking phenomena are available in the video segments or not is detected, and accordingly the blinking detection accuracy can be improved.
Description
Technical field
The present invention relates to technical field of image processing, particularly a kind of image information detecting method and device.
Background technology
Along with the development of intelligent terminal, various new man-machine interaction mode constantly occurs, such as, realizes man-machine interaction etc. by speech recognition or action recognition.Compared with voice recognition mode, action recognition by the impact of surrounding acoustic environment, can not adapt to the application scenarios of various acoustic environment.
Identification nictation is the one of action recognition, when whether detection exists phenomenon nictation, it is generally the human eye state feature of video segment each frame of video interior first obtaining certain time length, wherein, human eye state feature is for representing the state of human eye in frame of video, such as, eye opening motion state, eye closing motion state etc., then judge whether to there is phenomenon nictation according to the human eye state feature of each obtained frame of video.In prior art, usual employing obtains the human eye state feature of each frame of video based on the human eye state feature preparation method of still image, concrete, detect the human eye key point in each frame of video, according to the human eye key point detected, determine the upper palpebra inferior position of human eye in each frame of video, using the human eye state feature of the upper palpebra inferior position of human eye in each frame of video determined as each frame of video.Under normal circumstances, the human eye state feature that application said method obtains, can detect whether there is phenomenon nictation.But, when application said method obtains human eye state feature, be subject to the impact of the factors such as ambient lighting condition, customer location, user's human eye individual difference, detected human eye key point is caused to there is larger error, and then make the human eye state feature of each obtained frame of video correctly can not reflect the change of human eye state between each frame of video, cause blink detection accuracy low.
Summary of the invention
The embodiment of the invention discloses a kind of image information detecting method and device, with the human eye state feature utilizing the image information between frame of video to obtain frame of video, improve the accuracy of blink detection.
For achieving the above object, the embodiment of the invention discloses a kind of image information detecting method, described method comprises:
Determine the human eye area in the video segment of preset duration each frame of video interior;
Calculate the Optic flow information of human eye area each pixel interior in described video segment in each frame of video relative to the former frame of this frame of video respectively;
According to the Optic flow information calculated, add up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
According to statistics, obtain the human eye state feature of each frame of video in described video segment;
According to obtained human eye state feature, detect in described video segment whether there is phenomenon nictation.
Concrete, the human eye area in the described video segment determining preset duration in each frame of video, comprising:
Human eye area in the video segment determining preset duration according to following steps respectively in each frame of video:
Detect arbitrary frame of video P in described video segment
1in human eye area A
1;
Detect frame of video P
1former frame in human eye area A
2;
Determine frame of video P
1middle A
1and A
2union region be frame of video P
1in human eye area.
Concrete, arbitrary frame of video P in the described video segment of described detection
1in human eye area A
1, comprising:
Detect arbitrary frame of video P in described video segment
1in human face region;
Human eye key point is detected in detected human face region;
According to the human eye key point determination frame of video P detected
1in human eye area A
1.
Concrete, the Optic flow information that described basis calculates, the quantity of adding up the pixel moved downward in the human eye area in described video segment in each frame of video respectively and the quantity of the pixel moved upward, comprising:
The quantity of the pixel moved downward in the human eye area of adding up in described video segment in each frame of video respectively according to following steps and the quantity of pixel moved upward:
Arbitrary frame of video P in described video segment
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i<-Th
1when, upgrade Num
ufor: current Num
uvalue+1, wherein, Num
ufor representing frame of video P
2in human eye area in the quantity of pixel that moves downward, its initial value is 0, Th
1for the first threshold preset;
At frame of video P
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i> Th
2when, upgrade Num
ofor: current Num
ovalue+1, wherein, Num
ofor representing frame of video P
2in human eye area in the quantity of pixel that moves upward, its initial value is 0, Th
2for the Second Threshold preset.
Concrete, described according to statistics, obtain the human eye state feature of each frame of video in described video segment, comprising:
For the arbitrary frame of video P in described video segment
2, according to statistics, obtain the human eye state feature of this frame of video according to following steps:
At (Num
u-Num
o)/Num
t> Th
3when, determine frame of video P
2human eye state be characterized as: eye closing motion state;
At (Num
u-Num
o)/Num
t<-Th
3when, determine frame of video P
2human eye state be characterized as: eye opening motion state;
In other situations, determine frame of video P
2human eye state be characterized as: stationary state;
Wherein, Num
tfor frame of video P
2in human eye area in the quantity of pixel, Th
3for the 3rd threshold value preset.
For achieving the above object, the embodiment of the invention discloses a kind of image information detecting device, described device comprises:
Human eye area determination module, for determine preset duration video segment in human eye area in each frame of video;
Optic flow information computing module, for calculating in the human eye area in described video segment in each frame of video each pixel respectively relative to the Optic flow information of the former frame of this frame of video;
Pixel quantity statistical module, for according to the Optic flow information calculated, adds up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
Human eye state feature obtains module, for according to statistics, obtains the human eye state feature of each frame of video in described video segment;
Blink detection module, for according to obtained human eye state feature, detects in described video segment whether there is phenomenon nictation.
Concrete, described human eye area determination module, comprising: the first human eye region detection submodule, the second human eye area detection sub-module and human eye area determination submodule;
Described human eye area determination module, specifically for by each submodule above-mentioned, determines the human eye area in the video segment of preset duration each frame of video interior respectively;
Wherein, described first human eye region detection submodule, for detecting arbitrary frame of video P in described video segment
1in human eye area A
1;
Described second human eye area detection sub-module, for detecting frame of video P
1former frame in human eye area A
2;
Described human eye area determination submodule, for determining frame of video P
1middle A
1and A
2union region be frame of video P
1in human eye area.
Concrete, described first human eye region detection submodule, comprising:
Human face region detecting unit, for detecting arbitrary frame of video P in described video segment
1in human face region;
Human eye critical point detection unit, for detecting human eye key point in detected human face region;
Human eye area determining unit, for the human eye key point determination frame of video P that basis detects
1in human eye area A
1.
Concrete, described pixel quantity statistical module, specifically for according to following situation, adds up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
Arbitrary frame of video P in described video segment
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i<-Th
1when, upgrade Num
ufor: current Num
uvalue+1, wherein, Num
ufor representing frame of video P
2in human eye area in the quantity of pixel that moves downward, its initial value is 0, Th
1for the first threshold preset;
At frame of video P
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i> Th
2when, upgrade Num
ofor: current Num
ovalue+1, wherein, Num
ofor representing frame of video P
2in human eye area in the quantity of pixel that moves upward, its initial value is 0, Th
2for the Second Threshold preset.
Concrete, described human eye state feature obtains module, specifically for for the arbitrary frame of video P in described video segment
2, according to statistics, obtain the human eye state feature of this frame of video according to following situation:
At (Num
u-Num
o)/Num
t> Th
3when, determine frame of video P
2human eye state be characterized as: eye closing motion state;
At (Num
u-Num
o)/Num
t<-Th
3when, determine frame of video P
2human eye state be characterized as: eye opening motion state;
In other situations, determine frame of video P
2human eye state be characterized as: stationary state;
Wherein, Num
tfor frame of video P
2in human eye area in the quantity of pixel, Th
3for the 3rd threshold value preset.
As seen from the above, in the scheme that the embodiment of the present invention provides, according to the quantity of the pixel moved downward in the human eye area in each frame of video in the video segment of preset duration and the quantity of the pixel moved upward, obtain the human eye state feature of each frame of video, and according to obtained human eye state feature, detect in preset duration whether there is phenomenon nictation in each frame of video.Although in practical application, by ambient lighting condition, user's human face posture, whether wear glasses, the impact of the factor such as user's human eye individual difference, may error be there is in the human eye area in each frame of video determined, but in human eye area, each pixel is constant relative to the movement tendency of its former frame in each frame of video, so, the scheme that the application embodiment of the present invention provides, when whether there is phenomenon nictation in detection video segment, make use of the human eye state feature of the frame of video that the image information between frame of video obtains, the accuracy of blink detection can be improved.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
The schematic flow sheet of a kind of image information detecting method that Fig. 1 provides for the embodiment of the present invention;
The schematic flow sheet of the another kind of image information detecting method that Fig. 2 provides for the embodiment of the present invention;
The structural representation of a kind of image information detecting device that Fig. 3 provides for the embodiment of the present invention;
The structural representation of the another kind of image information detecting device that Fig. 4 provides for the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
The schematic flow sheet of a kind of image information detecting method that Fig. 1 provides for the embodiment of the present invention, the method comprises:
S101: determine the human eye area in the video segment of preset duration each frame of video interior.
In practical application, may there is multiple personage in a frame of video, then, when determining the human eye area in this frame of video, can be the human eye area of part personage, also can be the human eye area of all personages.
Preferably, people is when blinking, and the state of two eyes is generally identical, so, for reducing calculated amount, only the region at an eyes place can be defined as the human eye area of personage.
During human eye area in the video segment determining preset duration in each frame of video, directly according to the image information of each frame of video, the human eye area in each frame of video can be detected respectively.But, in practical application, when only utilizing the human eye area in the current image information detection frame of video being frame of video, the error of calculation existed is larger, and life period correlativity between continuous print two frame of video in video segment, that is: in video segment, between continuous print two frame of video, image content is similar, so, when determining the human eye area in frame of video, the human eye area detected in current video frame and the human eye area detected in the former frame of current video frame can be considered.
In one particular embodiment of the present invention, see Fig. 2, provide the schematic flow sheet of another kind of image information detecting method, in the present embodiment, determine the human eye area in the video segment of preset duration each frame of video interior, comprising:
Human eye area in the video segment determining preset duration according to following steps respectively in each frame of video:
S101A: detect arbitrary frame of video P in video segment
1in human eye area A
1.
In practical application, can by first detecting arbitrary frame of video P in video segment
1in human face region, then detect human eye key point in detected human face region, and according to the human eye key point determination frame of video P detected
1in human eye area A
1method detect arbitrary frame of video P in video segment
1in human eye area A
1.
Wherein, detect human face region in frame of video to be realized by the boosting human-face detector based on Haar or LBP (Local Binary Patterns) feature of comparative maturity in prior art, certainly, the method for human face region in frame of video that detects in practical application is not limited in this.
The human eye key point detected in human face region can be realized by methods such as AAM (Active Appearance Model), the ESR (Explicit Shape Regression) of comparative maturity in prior art, certainly, the method detecting human eye key point in human face region in practical application is not limited in this.
S101B: detect frame of video P
1former frame in human eye area A
2.
S101C: determine frame of video P
1middle A
1and A
2union region be frame of video P
1in human eye area.
As seen from the above, in embodiment illustrated in fig. 2, by frame of video P
1in the human eye area that detects and the union of human eye area detected in its former frame, at frame of video P
1the middle human eye area determined in this frame of video, make use of the temporal correlation between frame of video, can compared with limits at frame of video P
1in determine human eye area.
S102: calculate the Optic flow information of human eye area each pixel interior in video segment in each frame of video relative to the former frame of this frame of video respectively.
Those skilled in the art, it is appreciated that optical flow method is the important method of current movement image analysis, refer to the pattern further speed in time varying image.Because when object is when moving, the luminance patterns of its corresponding point on image is also in motion.The apparent motion of this brightness of image pattern is exactly light stream.Light stream have expressed the change of image, because it contains the information of target travel, therefore observed person can be used for determining the motion conditions of target.Can amplify out optical flow field by the definition of light stream, it refers to the one two dimension instantaneous velocity field that in image, all pixels are formed, and two-dimension speed vector is wherein the projection of three dimensional velocity vectors at imaging surface of visible point in scenery.So light stream not only contains the movable information of observed object, but also comprise the abundant information about scenery three-dimensional structure.
Seen from the above description, aforesaid Optic flow information can be two-dimensional signal, also can be three-dimensional information, when aforementioned Optic flow information is two-dimensional signal, this Optic flow information comprises horizontal direction component and vertical direction component, represents the horizontal direction of pixel relative to last frame of video and the pattern further speed of vertical direction respectively.
Concrete, Optic flow information can use the acquisition such as cvCalcOpticalFlowFarneback () function, cvCalcOpticalFlowLK () function, cvCalcOpticalFlowHS () function in opencv, wherein, the Optic flow information precision using cvCalcOpticalFlowFarneback () function to obtain is higher.
S103: according to the Optic flow information calculated, adds up the quantity of the pixel moved downward in the human eye area in video segment in each frame of video and the quantity of the pixel moved upward respectively.
Under normal circumstances, when human eye is in eye closing motion state, each pixel in upper eyelid moves downward, and each pixel of palpebra inferior is almost motionless; When human eye is in eye opening motion state, each pixel in upper eyelid moves upward, and each pixel of palpebra inferior is almost motionless.In practical application, when human eye can be utilized to be in different motion state, whether the motion conditions of each pixel in upper eyelid, exist phenomenon nictation in the video segment judging preset duration.
In an alternate embodiment of the present invention where, add up the quantity of the pixel moved downward in the human eye area in video segment in each frame of video and the quantity of the pixel moved upward respectively, comprising:
The quantity of the pixel moved downward in the human eye area of adding up in video segment in each frame of video respectively according to following steps and the quantity of pixel moved upward:
Arbitrary frame of video P in video segment
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i<-Th
1when, upgrade Num
ufor: current Num
uvalue+1, wherein, Num
ufor representing frame of video P
2in human eye area in the quantity of pixel that moves downward, its initial value is 0, Th
1for the first threshold preset;
At frame of video P
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i> Th
2when, upgrade Num
ofor: current Num
ovalue+1, wherein, Num
ofor representing frame of video P
2in human eye area in the quantity of pixel that moves upward, its initial value is 0, Th
2for the Second Threshold preset.
S104: according to statistics, obtains the human eye state feature of each frame of video in video segment.
Above-mentioned human eye movement's state can comprise: eye closing motion state, eye opening motion state and stationary state etc.
Under normal circumstances, when human eye is in eye closing motion state, most of pixels in human eye area are all in the state of moving downward, and when human eye is in eye opening motion state, most of pixels in human eye area are all in the state of moving upward, even if detected for determining that the human eye key point of human eye area has error in above-mentioned S101A, the human eye area in other words in determined frame of video has error, and in human eye area, the motion state of most of pixel also can not change.
For the arbitrary frame of video P in video segment
2, according to statistics, the human eye state feature of this frame of video can be obtained according to following steps:
At (Num
u-Num
o)/Num
t> Th
3when, frame of video P is described
2the pixel moved downward in middle human eye area is more, determines frame of video P
2human eye state be characterized as: eye closing motion state;
At (Num
u-Num
o)/Num
t<-Th
3when, frame of video P is described
2the pixel moved upward in middle human eye area is more, determines frame of video P
2human eye state be characterized as: eye opening motion state;
In other situations, determine frame of video P
2human eye state be characterized as: stationary state;
Wherein, Num
tfor frame of video P
2in human eye area in the quantity of pixel, Th
3for the 3rd threshold value preset.
When face planar the anglec of rotation angle time, the light stream direction of motion of pixel i arbitrary in human eye area can be decomposed along with the vertical of eyes line and horizontal direction, thus the direction of motion component obtained perpendicular to eyes line, then carry out calculating above.
Certainly, the method obtaining the human eye state feature of each frame of video in video segment in practical application is not limited in this, such as, by frame of video P
2in human eye area in ratio, the frame of video P of pixel quantity in the quantity of pixel that moves downward and this human eye area
2in human eye area in the ratio of pixel quantity in the quantity of pixel that moves upward and this human eye area, as foundation, obtain the human eye state feature etc. of each frame of video in video segment.
S105: according to obtained human eye state feature, detects in video segment whether there is phenomenon nictation.
Be understandable that, in human eye process nictation, comprise the eye closing campaign of human eye and motion of opening eyes, then obtain preset duration video segment in each frame of video human eye state feature after, if the human eye state feature of continuous some frame of video is converted to eye opening motion state by motion state of closing one's eyes, or by eye opening motion state be converted to eye closing motion state all can be regarded as in this video segment exist nictation phenomenon.
As seen from the above, in the scheme that each embodiment above-mentioned provides, according to the quantity of the pixel moved downward in the human eye area in each frame of video in the video segment of preset duration and the quantity of the pixel moved upward, obtain the human eye state feature of each frame of video, and according to obtained human eye state feature, detect in preset duration whether there is phenomenon nictation in each frame of video.Although in practical application, by ambient lighting condition, user's attitude, whether wear glasses, the impact of the factor such as user's human eye individual difference, may error be there is in the human eye area in each frame of video determined, but in human eye area, each pixel is constant relative to the movement tendency of its former frame in each frame of video, so, apply the scheme that each embodiment above-mentioned provides, when whether there is phenomenon nictation in detection video segment, make use of the human eye state feature of the frame of video that the image information between frame of video obtains, the accuracy of blink detection can be improved.
Corresponding with above-mentioned image information detecting method, the embodiment of the present invention additionally provides a kind of image information detecting device.
The structural representation of a kind of image information detecting device that Fig. 3 provides for the embodiment of the present invention, this device comprises: human eye area determination module 301, Optic flow information computing module 302, pixel quantity statistical module 303, human eye state feature obtain module 304 and blink detection module 305.
Wherein, human eye area determination module 301, for determine preset duration video segment in human eye area in each frame of video;
Optic flow information computing module 302, for calculating in the human eye area in described video segment in each frame of video each pixel respectively relative to the Optic flow information of the former frame of this frame of video;
Pixel quantity statistical module 303, for according to the Optic flow information calculated, adds up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
Human eye state feature obtains module 304, for according to statistics, obtains the human eye state feature of each frame of video in described video segment;
Blink detection module 305, for according to obtained human eye state feature, detects in described video segment whether there is phenomenon nictation.
In one particular embodiment of the present invention, see Fig. 4, provide the structural representation of another kind of image information detecting device, compared with embodiment illustrated in fig. 3, in the present embodiment, human eye area determination module 301, comprising: the first human eye region detection submodule 3011, second human eye area detection sub-module 3012 and human eye area determination submodule 3013.
Human eye area determination module 301, specifically for by each submodule above-mentioned, determines the human eye area in the video segment of preset duration each frame of video interior respectively;
Wherein, the first human eye region detection submodule 3011, for detecting arbitrary frame of video P in described video segment
1in human eye area A
1;
Second human eye area detection sub-module 3012, for detecting frame of video P
1former frame in human eye area A
2;
Human eye area determination submodule 3013, for determining frame of video P
1middle A
1and A
2union region be frame of video P
1in human eye area.
Concrete, the first above-mentioned human eye region detection submodule 3011 can comprise: human face region detecting unit, human eye critical point detection unit and human eye area determining unit (not shown).
Wherein, human face region detecting unit, for detecting arbitrary frame of video P in described video segment
1in human face region;
Human eye critical point detection unit, for detecting human eye key point in detected human face region;
Human eye area determining unit, for the human eye key point determination frame of video P that basis detects
1in human eye area A
1.
In a kind of Alternate embodiments of the present invention, above-mentioned pixel quantity statistical module 303, specifically for according to following situation, add up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
Arbitrary frame of video P in described video segment
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i<-Th
1when, upgrade Num
ufor: current Num
uvalue+1, wherein, Num
ufor representing frame of video P
2in human eye area in the quantity of pixel that moves downward, its initial value is 0, Th
1for the first threshold preset;
At frame of video P
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i> Th
2when, upgrade Num
ofor: current Num
ovalue+1, wherein, Num
ofor representing frame of video P
2in human eye area in the quantity of pixel that moves upward, its initial value is 0, Th
2for the Second Threshold preset.
Optionally, above-mentioned human eye state feature obtains module 304, specifically can be used for for the arbitrary frame of video P in described video segment
2, according to statistics, obtain the human eye state feature of this frame of video according to following situation:
At (Num
u-Num
o)/Num
t> Th
3when, determine frame of video P
2human eye state be characterized as: eye closing motion state;
At (Num
u-Num
o)/Num
t<-Th
3when, determine frame of video P
2human eye state be characterized as: eye opening motion state;
In other situations, determine frame of video P
2human eye state be characterized as: stationary state;
Wherein, Num
tfor frame of video P
2in human eye area in the quantity of pixel, Th
3for the 3rd threshold value preset.
When face planar the anglec of rotation angle time, the light stream direction of motion of pixel i arbitrary in human eye area can be decomposed along with the vertical of eyes line and horizontal direction, thus the direction of motion component obtained perpendicular to eyes line, then carry out calculating above.
As seen from the above, in the scheme that each embodiment above-mentioned provides, according to the quantity of the pixel moved downward in the human eye area in each frame of video in the video segment of preset duration and the quantity of the pixel moved upward, obtain the human eye state feature of each frame of video, and according to obtained human eye state feature, detect in preset duration whether there is phenomenon nictation in each frame of video.Although in practical application, by ambient lighting condition, user's human face posture, whether wear glasses, the impact of the factors such as user's human eye individual difference, may error be there is in the human eye area in each frame of video determined, but in human eye area, each pixel is constant relative to the movement tendency of its former frame in each frame of video, so, apply the scheme that each embodiment above-mentioned provides, when whether there is phenomenon nictation in detection video segment, make use of the human eye state feature of the frame of video that the image information between frame of video obtains, the accuracy of blink detection can be improved.
For systems/devices embodiment, because it is substantially similar to embodiment of the method, so description is fairly simple, relevant part illustrates see the part of embodiment of the method.
It should be noted that, in this article, the such as relational terms of first and second grades and so on is only used for an entity or operation to separate with another entity or operational zone, and not necessarily requires or imply the relation that there is any this reality between these entities or operation or sequentially.And, term " comprises ", " comprising " or its any other variant are intended to contain comprising of nonexcludability, thus make to comprise the process of a series of key element, method, article or equipment and not only comprise those key elements, but also comprise other key elements clearly do not listed, or also comprise by the intrinsic key element of this process, method, article or equipment.When not more restrictions, the key element limited by statement " comprising ... ", and be not precluded within process, method, article or the equipment comprising described key element and also there is other identical element.
One of ordinary skill in the art will appreciate that all or part of step realized in said method embodiment is that the hardware that can carry out instruction relevant by program has come, described program can be stored in computer read/write memory medium, here the alleged storage medium obtained, as: ROM/RAM, magnetic disc, CD etc.
The foregoing is only preferred embodiment of the present invention, be not intended to limit protection scope of the present invention.All any amendments done within the spirit and principles in the present invention, equivalent replacement, improvement etc., be all included in protection scope of the present invention.
Claims (10)
1. an image information detecting method, is characterized in that, described method comprises:
Determine the human eye area in the video segment of preset duration each frame of video interior;
Calculate the Optic flow information of human eye area each pixel interior in described video segment in each frame of video relative to the former frame of this frame of video respectively;
According to the Optic flow information calculated, add up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
According to statistics, obtain the human eye state feature of each frame of video in described video segment;
According to obtained human eye state feature, detect in described video segment whether there is phenomenon nictation.
2. method according to claim 1, is characterized in that, the human eye area in the described video segment determining preset duration in each frame of video, comprising:
Human eye area in the video segment determining preset duration according to following steps respectively in each frame of video:
Detect arbitrary frame of video P in described video segment
1in human eye area A
1;
Detect frame of video P
1former frame in human eye area A
2;
Determine frame of video P
1middle A
1and A
2union region be frame of video P
1in human eye area.
3. method according to claim 2, is characterized in that, arbitrary frame of video P in the described video segment of described detection
1in human eye area A
1, comprising:
Detect arbitrary frame of video P in described video segment
1in human face region;
Human eye key point is detected in detected human face region;
According to the human eye key point determination frame of video P detected
1in human eye area A
1.
4. the method according to any one of claim 1-3, it is characterized in that, the Optic flow information that described basis calculates, the quantity of adding up the pixel moved downward in the human eye area in described video segment in each frame of video respectively and the quantity of the pixel moved upward, comprising:
The quantity of the pixel moved downward in the human eye area of adding up in described video segment in each frame of video respectively according to following steps and the quantity of pixel moved upward:
Arbitrary frame of video P in described video segment
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i<-Th
1when, upgrade Num
ufor: current Num
uvalue+1, wherein, Num
ufor representing frame of video P
2in human eye area in the quantity of pixel that moves downward, its initial value is 0, Th
1for the first threshold preset;
At frame of video P
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i> Th
2when, upgrade Num
ofor: current Num
ovalue+1, wherein, Num
ofor representing frame of video P
2in human eye area in the quantity of pixel that moves upward, its initial value is 0, Th
2for the Second Threshold preset.
5. method according to claim 4, is characterized in that, described according to statistics, obtains the human eye state feature of each frame of video in described video segment, comprising:
For the arbitrary frame of video P in described video segment
2, according to statistics, obtain the human eye state feature of this frame of video according to following steps:
At (Num
u-Num
o)/Num
t> Th
3when, determine frame of video P
2human eye state be characterized as: eye closing motion state;
At (Num
u-Num
o)/Num
t<-Th
3when, determine frame of video P
2human eye state be characterized as: eye opening motion state;
In other situations, determine frame of video P
2human eye state be characterized as: stationary state;
Wherein, Num
tfor frame of video P
2in human eye area in the quantity of pixel, Th
3for the 3rd threshold value preset.
6. an image information detecting device, is characterized in that, described device comprises:
Human eye area determination module, for determine preset duration video segment in human eye area in each frame of video;
Optic flow information computing module, for calculating in the human eye area in described video segment in each frame of video each pixel respectively relative to the Optic flow information of the former frame of this frame of video;
Pixel quantity statistical module, for according to the Optic flow information calculated, adds up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
Human eye state feature obtains module, for according to statistics, obtains the human eye state feature of each frame of video in described video segment;
Blink detection module, for according to obtained human eye state feature, detects in described video segment whether there is phenomenon nictation.
7. device according to claim 6, is characterized in that, described human eye area determination module, comprising: the first human eye region detection submodule, the second human eye area detection sub-module and human eye area determination submodule;
Described human eye area determination module, specifically for by each submodule above-mentioned, determines the human eye area in the video segment of preset duration each frame of video interior respectively;
Wherein, described first human eye region detection submodule, for detecting arbitrary frame of video P in described video segment
1in human eye area A
1;
Described second human eye area detection sub-module, for detecting frame of video P
1former frame in human eye area A
2;
Described human eye area determination submodule, for determining frame of video P
1middle A
1and A
2union region be frame of video P
1in human eye area.
8. device according to claim 7, is characterized in that, described first human eye region detection submodule, comprising:
Human face region detecting unit, for detecting arbitrary frame of video P in described video segment
1in human face region;
Human eye critical point detection unit, for detecting human eye key point in detected human face region;
Human eye area determining unit, for the human eye key point determination frame of video P that basis detects
1in human eye area A
1.
9. the device according to any one of claim 6-8, it is characterized in that, described pixel quantity statistical module, specifically for according to following situation, add up the quantity of pixel and the quantity of the pixel moved upward that move downward in the human eye area in described video segment each frame of video interior respectively;
Arbitrary frame of video P in described video segment
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i<-Th
1when, upgrade Num
ufor: current Num
uvalue+1, wherein, Num
ufor representing frame of video P
2in human eye area in the quantity of pixel that moves downward, its initial value is 0, Th
1for the first threshold preset;
At frame of video P
2in human eye area in the vertical direction component H of Optic flow information of arbitrary pixel i
i> Th
2when, upgrade Num
ofor: current Num
ovalue+1, wherein, Num
ofor representing frame of video P
2in human eye area in the quantity of pixel that moves upward, its initial value is 0, Th
2for the Second Threshold preset.
10. device according to claim 9, is characterized in that, described human eye state feature obtains module, specifically for for the arbitrary frame of video P in described video segment
2, according to statistics, obtain the human eye state feature of this frame of video according to following situation:
At (Num
u-Num
o)/Num
t> Th
3when, determine frame of video P
2human eye state be characterized as: eye closing motion state;
At (Num
u-Num
o)/Num
t<-Th
3when, determine frame of video P
2human eye state be characterized as: eye opening motion state;
In other situations, determine frame of video P
2human eye state be characterized as: stationary state;
Wherein, Num
tfor frame of video P
2in human eye area in the quantity of pixel, Th
3for the 3rd threshold value preset.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410838292.6A CN104504378A (en) | 2014-12-29 | 2014-12-29 | Method and device for detecting image information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410838292.6A CN104504378A (en) | 2014-12-29 | 2014-12-29 | Method and device for detecting image information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104504378A true CN104504378A (en) | 2015-04-08 |
Family
ID=52945774
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410838292.6A Pending CN104504378A (en) | 2014-12-29 | 2014-12-29 | Method and device for detecting image information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104504378A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778611A (en) * | 2016-12-16 | 2017-05-31 | 天津牧瞳星科技有限公司 | Method for tracking blink activity on line |
| CN110223322A (en) * | 2019-05-31 | 2019-09-10 | 腾讯科技（深圳）有限公司 | Image-recognizing method, device, computer equipment and storage medium |
| CN111091058A (en) * | 2019-11-18 | 2020-05-01 | 京东方科技集团股份有限公司 | Eye state detection method, device, equipment and storage medium |
| CN113516017A (en) * | 2021-04-22 | 2021-10-19 | 平安科技（深圳）有限公司 | Method and device for supervising medicine taking process, terminal equipment and storage medium |
| CN115937958A (en) * | 2022-12-01 | 2023-04-07 | 北京惠朗时代科技有限公司 | Blink detection method, device, equipment and storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005013626A (en) * | 2003-06-27 | 2005-01-20 | Nissan Motor Co Ltd | Awakefullness detector |
| CN101877051A (en) * | 2009-10-30 | 2010-11-03 | 江苏大学 | Driver attention state monitoring method and device |
| CN101908140A (en) * | 2010-07-29 | 2010-12-08 | 中山大学 | Biopsy method for use in human face identification |
-
2014
- 2014-12-29 CN CN201410838292.6A patent/CN104504378A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005013626A (en) * | 2003-06-27 | 2005-01-20 | Nissan Motor Co Ltd | Awakefullness detector |
| CN101877051A (en) * | 2009-10-30 | 2010-11-03 | 江苏大学 | Driver attention state monitoring method and device |
| CN101908140A (en) * | 2010-07-29 | 2010-12-08 | 中山大学 | Biopsy method for use in human face identification |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778611A (en) * | 2016-12-16 | 2017-05-31 | 天津牧瞳星科技有限公司 | Method for tracking blink activity on line |
| CN110223322A (en) * | 2019-05-31 | 2019-09-10 | 腾讯科技（深圳）有限公司 | Image-recognizing method, device, computer equipment and storage medium |
| CN110223322B (en) * | 2019-05-31 | 2021-12-14 | 腾讯科技（深圳）有限公司 | Image recognition method and device, computer equipment and storage medium |
| CN111091058A (en) * | 2019-11-18 | 2020-05-01 | 京东方科技集团股份有限公司 | Eye state detection method, device, equipment and storage medium |
| CN111091058B (en) * | 2019-11-18 | 2024-05-17 | 京东方科技集团股份有限公司 | Eye state detection method, device, equipment and storage medium |
| CN113516017A (en) * | 2021-04-22 | 2021-10-19 | 平安科技（深圳）有限公司 | Method and device for supervising medicine taking process, terminal equipment and storage medium |
| CN113516017B (en) * | 2021-04-22 | 2023-07-11 | 平安科技（深圳）有限公司 | Supervision method and device for medicine taking process, terminal equipment and storage medium |
| CN115937958A (en) * | 2022-12-01 | 2023-04-07 | 北京惠朗时代科技有限公司 | Blink detection method, device, equipment and storage medium |
| CN115937958B (en) * | 2022-12-01 | 2023-12-15 | 北京惠朗时代科技有限公司 | Blink detection method, blink detection device, blink detection equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2013200807B2 (en) | Method and portable terminal for correcting gaze direction of user in image | |
| US10733783B2 (en) | Motion smoothing for re-projected frames | |
| CN112823328B (en) | Method for performing an internal and/or external calibration of a camera system | |
| CN104504378A (en) | Method and device for detecting image information | |
| KR102212209B1 (en) | Method, apparatus and computer readable recording medium for eye gaze tracking | |
| JP2021504856A (en) | Forward collision control methods and devices, electronics, programs and media | |
| CN105940430B (en) | Personnel's method of counting and its device | |
| CN110959160A (en) | Gesture recognition method, device and equipment | |
| CN109711304A (en) | A kind of man face characteristic point positioning method and device | |
| CN104899563A (en) | Two-dimensional face key feature point positioning method and system | |
| CN102194443A (en) | Display method and system for window of video picture in picture and video processing equipment | |
| Patel et al. | Moving object detection with moving background using optic flow | |
| CN102473282A (en) | External light glare assessment device, line of sight detection device and external light glare assessment method | |
| US10803604B1 (en) | Layered motion representation and extraction in monocular still camera videos | |
| JP6221292B2 (en) | Concentration determination program, concentration determination device, and concentration determination method | |
| CN113920167A (en) | Image processing method, device, storage medium and computer system | |
| CN103049748B (en) | Behavior monitoring method and device | |
| CN111382705A (en) | Reverse behavior detection method and device, electronic equipment and readable storage medium | |
| JP2014170978A (en) | Information processing device, information processing method, and information processing program | |
| Shahid et al. | Eye-gaze and augmented reality framework for driver assistance | |
| GB2467643A (en) | Improved detection of people in real world videos and images. | |
| WO2021239000A1 (en) | Method and apparatus for identifying motion blur image, and electronic device and payment device | |
| CN111618856B (en) | Robot control method and system based on visual excitation points and robot | |
| CN106485713B (en) | Video foreground detection method | |
| KR101909326B1 (en) | User interface control method and system using triangular mesh model according to the change in facial motion |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150408 |
|
| RJ01 | Rejection of invention patent application after publication |