WO2012000408A1 - Eye detecting method and eye detecting device - Google Patents

Eye detecting method and eye detecting device Download PDF

Info

Publication number
WO2012000408A1
WO2012000408A1 PCT/CN2011/076339 CN2011076339W WO2012000408A1 WO 2012000408 A1 WO2012000408 A1 WO 2012000408A1 CN 2011076339 W CN2011076339 W CN 2011076339W WO 2012000408 A1 WO2012000408 A1 WO 2012000408A1
Authority
WO
WIPO (PCT)
Prior art keywords
eye
image
region
detecting
face
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2011/076339
Other languages
French (fr)
Inventor
Keyu Zhang
Changyan Wang
Jin Xu
Zhigang Wang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BYD Co Ltd
Original Assignee
BYD Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BYD Co Ltd filed Critical BYD Co Ltd
Publication of WO2012000408A1 publication Critical patent/WO2012000408A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18Eye characteristics, e.g. of the iris
    • G06V40/19Sensors therefor

Definitions

  • the present disclosure relates to an object detecting field, and more particularly, to an eye detecting method and eye detecting device.
  • a research proves that an eye status may reflect a fatigue of a driver reliably, and an eye detection is a premise for judging the eye status. Therefore how to detect the eye correctly and quickly is key to achieve fatigue monitoring in a driving process.
  • a detecting technology includes a method based on study, template matching or characteristics.
  • the method based on study includes, for example, AdaBoost (Adaptive Boosting) algorithm, neural network, and principal component analysis.
  • AdaBoost Adaptive Boosting
  • the algorithms are with high accuracy, but a computation is complicated, which limits an application in a real-time system.
  • the method based on the template matching firstly obtains a template of a right eye and a left eye and then searches in a whole image for matching. Because of an influence of light changing, angle rotation etc., this method has low accuracy and the computation for this method is complicated.
  • the method based on the characteristics includes HIS (Hue Saturation Intensity, representing tone, color saturation and brightness) iris detection, gray projection etc..
  • HIS Human Saturation Intensity, representing tone, color saturation and brightness
  • iris detection is fast in speed, but this method should be performed in a color space and is not suitable for an infrared imaging system.
  • the gray projection method will be invalid when a face is deflected.
  • a target image classifier used by the AdaBoost algorithm may scale up or down.
  • the AdaBoost algorithm comprises: judging whether a size of a scaled up target image classifier is larger than the size of the target image, if yes, terminating the detection, otherwise scaling up the target image classifier to detect the target image until finding a match between the target image classifier and the target image.
  • the target images with different sizes are detected by scaling up the target image classifier continuously until the size of the target image classifier is larger than the size of the target image. Therefore, the larger the size of the image is, the larger the amount of the target image classifier is, and the longer the time taken in reviewing the image each time is. Therefore the computation of the AdaBoost algorithm is complex.
  • the present disclosure is directed to solve at least one of the above mentioned problems and to provide an eye detecting method and an eye detecting device.
  • an eye detecting method comprising the steps of: A) detecting whether there is a face region in a N th image, in which N > 1 ;
  • an eye detecting device is provided.
  • the eye detecting device comprises: a face detecting unit, configured to detect whether there is a face region in an image; an eye detecting unit, configured to detect whether there is an eye region in an image and recording a detecting result of the image; a controlling unit, connected with the face detecting unit and the eye detecting unit, and configured to control the eye detecting unit to detect whether there is a first eye region in a face region in a N th image and record a detecting result of the N th image if the face detecting unit detects that there is the face region in the N th image; and an area determining unit, connected with the controlling unit and configured to determine an area in a (N+l) th image corresponding to and larger than the first eye region as a pending eye region in the (N+l) th image according to a location and size information included in the detecting result of the N th image, wherein the controlling unit is further configured to control the eye detecting unit to detect whether there is a second eye region in the pending eye region in the (N
  • the present disclosure when there is the eye region in the N th image, it is not necessary to search the whole (N+l) th image but only to search the area in the (N+l) th image corresponding to the eye region in the N th image. Therefore, a time needed for detecting the eye region is decreased and the eye detecting may be performed quickly, which is suitable for a real-time image system.
  • Fig. 1 is a flow chart showing an eye detecting method according to a conventional AdaBoost algorithm
  • Fig. 2 is a flow chart showing an eye detecting method according to an embodiment of the present disclosure
  • Fig. 3 A is a flow chart showing a face region detecting method according to an embodiment of the present disclosure
  • Fig. 3B is a flow chart showing a face region detecting method according to an embodiment of the present disclosure
  • Fig. 4A is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure
  • Fig. 4B is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure
  • Fig. 4C is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure.
  • Fig. 4D is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure.
  • Fig. 5 A is a schematic block diagram of showing an eye detecting device according to one an embodiment of the present disclosure
  • Fig. 5B is a schematic block diagram of showing an eye detecting device according to one an embodiment of the present disclosure
  • Fig. 5C is a schematic block diagram of showing an eye detecting device according to one an embodiment of the present disclosure.
  • Fig. 6 is a schematic block diagram showing a face detecting unit according to an embodiment of the present disclosure.
  • Fig. 7 is a schematic block diagram showing an eye detecting unit according to an embodiment of the present disclosure.
  • an eye detecting method comprises the following steps.
  • Step S202) a N th image is detected to find whether there is a face region in the N th image, in which N > 1.
  • Step S204 if there is the face region in the N th image, it is detected whether there is a first eye region in the N th image and a detecting result of the N th image is obtained.
  • Step S206 the detecting result of the N th image is recorded and a (N+l) th image is received.
  • Step S208) if the detecting result of the N th image indicates there is the first eye region in the
  • face samples and eye samples are obtained by sampling the image of objects at different angles and by putting an infrared camera in a different light environment.
  • the different light environment comprises but not limited to the environment illuminated by a fluorescent light, an environment without a visible light, at a side of an instrument in a vehicle under different lights.
  • the objects sampled are people with different heights, wearing or not wearing glasses and with different skin colors. Therefore, a sufficient face database is obtained.
  • An eye database is obtained by extracting an eye part from each image in the face database.
  • the face database, the eye database and a database without the face sample and eye sample are established.
  • the face classifier and the eye classifier are obtained by calculating a characteristic value of all the samples in the database and performing selection on the characteristic values.
  • a procedure of obtaining the face classifier comprises the following steps: selecting in the face database and the database without the face sample and eye sample according to a selected group of characteristic values to obtain a first selecting result; adding one or more characteristic values and selecting the image in the first selecting result according the characteristic values with the one or more characteristic values added to obtain the second selecting result; repeating adding and selecting until a selecting error for the images including the face and the images not including the face meets a requirement; obtaining the face classifier including the characteristic value and a parameter.
  • a procedure of obtaining the eye classifier is similar with the procedure of obtaining the face classifier, which will not be described in detail here.
  • the detecting result of the N th image is recorded and the (N+l) th image is received.
  • Step S204) comprises extracting the face region and detecting the first eye region in the face region.
  • Fig. 3 A is a flow chart showing a face region detecting method according to an embodiment of the present disclosure.
  • step S202 comprises the following steps:
  • Step S302 detecting whether there is the face region in the N th image by using a face classifier of a preset size
  • Step S304 terminating detecting whether there is the face region in the N th image if the face region in the N th image is detected;
  • Step S306 if the face region in the N th image is not detected, enlarging a size of the face classifier and detecting the face region in the N th image;
  • Step S308) if the face region in the N th image is detected, going to step S304, otherwise going to step S310;
  • Step S310 comparing the size of the face classifier with the size of the N th image, if the size of the face classifier is the same as or larger than the size of the N th image, going to step S304, otherwise going to step S306.
  • the preset size of the face classifier is 40*40.
  • step S202 comprises the following steps:
  • Step S312) detecting whether there is the face region in the N th image by using a face classifier of a preset size
  • Step S314) terminating detecting whether there is the face region in the N th image if the face region in the N th image is detected;
  • Step S316 if the face region in the N th image is not detected, decreasing a size of the face classifier and detecting whether there is the face region in the N th image.
  • Step S318) if the face region in the N th image is detected, going to step S304, otherwise going to step S320.
  • Step S320 comparing the size of the face classifier with the size of 40*40, if the size of the face classifier is the same as or small than the size of 40*40, going to step S314, else going to step S316.
  • the preset size of the face classifier is the same as the size ofthe N th image.
  • Fig. 4A-4D is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure.
  • the step S204 comprises the following steps:
  • Step S402 detecting whether there is the first eye region in the N th image
  • Step S404 terminating detecting whether there is the first eye region in the N th image if the first eye region in the N th image is detected;
  • Step S406 if the first eye region in the N th image is not detected, enlarging a size of the eye classifier and detecting whether there is the first eye region in the N th image;
  • Step S408 if the first eye region in the N th image is not detected, going to step S404, otherwise going to step S410;
  • Step S410 comparing the size of the eye classifier with the size of the N th image, if the size of the eye classifier is the same as or larger than the size of the N th image, going to step S404, else going to step S406.
  • the preset size of the eye classifier is 40*40.
  • the step S204 comprises the following steps:
  • Step 412 detecting whether there is the first eye region in the N th image
  • Step 414) terminating detecting whether there is the first eye region in the N th image if the first eye region in the N th image is detected;
  • Step S416) if the first eye region in the N th image is not detected, decreasing a size of the eye classifier and detecting whether there is the first eye region in the N th image;
  • Step S418) if the first eye region in the N th image is not detected, going to step S414, else going to step S420;
  • Step S420 comparing the size of the eye classifier with the size of 40*40, if the size of the eye classifier is the same as or smaller than the size of 40*40, going to step S414, else going to step S416.
  • the preset size of the eye classifier is the same as the size of the first eye region.
  • the step S212 comprises the following steps:
  • Step S422 detecting whether there is the second eye region is detected in the (N+l) th image; Step S424), terminating detecting whether there is the second eye region in the (N+l) th image if the second eye region in the (N+l) th image is detected; Step S426), if the second eye region in the (N+l) th image is not detected, enlarging a size of the eye classifier and detecting whether there is the second eye region in the (N+l) th image;
  • Step S428) if the second eye region in the (N+l) th image is not detected, going to step S404, otherwise going to step S430;
  • Step S430 comparing the size of the eye classifier with the size of the (N+l) th image, if the size of the eye classifier is the same as or larger than the size of the (N+l) th image, going to step S424, otherwise going to step S426.
  • the preset size of the eye classifier is 40*40.
  • the step S212 comprises the following steps:
  • Step 432) detecting whether there is the second eye region in the (N+l) th image
  • Step 434) terminating detecting whether there is the second eye region in the (N+l) th image if the second eye region in the (N+l) th image is detected;
  • Step S436 if the second eye region in the (N+l) th image is not detected, decreasing a size of the eye classifier and detecting whether there is the second eye region in the (N+l) th image;
  • Step S438) if the second eye region in the (N+l) th image is not detected, going to step S414, otherwise going to step S440;
  • Step S440 comparing the size of the eye classifier with the size of 40*40, if the size of the eye classifier is the same as or smaller than the size of 40*40, going to step S434, otherwise going to step S436.
  • the preset size of the eye classifier is the same as the size of the second eye region.
  • the method of the present disclosure when there is the eye region in the N th image, it is not necessary to search the whole (N+l) th image but only to search the pending eye region in the (N+l) th image corresponding to and larger than the first eye region in the N th image. Therefore, a time needed for detecting the eye region is reduced and the eye detecting may be performed quickly, which is suitable for a real-time image system.
  • an eye detection device comprises a face detecting unit 501, an eye detecting unit 502, a controlling unit 503 and an area determining unit 504.
  • the face detecting unit 501 is configured to detect whether there is a face region in an image.
  • the eye detecting unit 502 is configured to detect whether there is an eye region in an image and recording a detecting result of the image.
  • the controlling unit 503 is connected with the face detecting unit 501 and the eye detecting unit 502, and is configured to control the eye detecting unit 502 to detect whether there is a first eye region in a face region in a N th image and record a detecting result of the N th image.
  • the area determining unit 504 is connected with the controlling unit 503 and is configured to determine an area in a (N+l) th image corresponding to and larger than the first eye region as an pending eye region in the (N+l) th image according to a location and size information included in the detecting result of the N th image.
  • the controlling unit 503 is further configured to control the eye detecting unit 502 to detect whether there is a second eye region in the pending eye region in the (N+l) th image.
  • the first eye region and the second eye region are a rectangular region.
  • the eye detecting device 500 may further comprise a face extracting unit.
  • the face extracting unit is connected with the face detecting unit 501 and the eye detecting unit 502 and is configured to extract the face region in the image.
  • the eye detecting unit 502 detects the first eye region in the face region.
  • the eye detecting unit 502 may further comprise a first eye detecting unit 510 and a second eye detecting unit 508.
  • the first eye detecting unit 510 is connected with the face extracting unit 506 and configured to detect whether there is the first eye region in the face region.
  • the second eye detecting unit 508 is connected with the area determining unit 504 and is configured to detect whether there is the second eye region in the pending eye region.
  • Fig. 6 is a schematic block diagram showing a face detecting unit according to an embodiment of the present disclosure.
  • the face detecting unit 501 comprises a detecting module 5011, a face classifier size changing module 5012 and a first sub-controlling module 5013.
  • the detecting module 5011 is configured to detect the face region in the N th image.
  • the face classifier size change module 5012 is configured to enlarge or decrease the size of the face classifier.
  • the first sub-controlling module 5013 is connected with the detecting module 5011.
  • the face classifier size changing module 5012 and is configured to control to enlarge a size of the face classifier until the size of the face classifier is the same as or larger than a size of the N th image if the face region in the N th image is not detected.
  • the face classifier is with a preset size of 40*40.
  • the face classifier size changing module 5012 is configured to control to decrease a size of the face classifier until the size of the face classifier is the same as or smaller than a size of 40*40 if the face region in the N th image is not detected.
  • the face classifier is with a preset size that is the same as the size of the N th image.
  • Fig. 7 is a schematic block diagram showing an eye detecting unit according to an embodiment of the present disclosure.
  • the eye detecting unit 502 comprises a detecting module 5021, an eye classifier size changing module 5022 and a second sub-controlling module 5023.
  • the detecting module 5021 is configured to detect the eye region in an image.
  • the eye classifier size changing module 5022 is configured to enlarge or decreases the eye classifier.
  • the second sub-controlling module 5023 is connected with the detecting module 5021.
  • the eye classifier size changing module 5022 and is configured to control to enlarge a size of the eye classifier until the size of the eye classifier is the same as a size of the image if the eye region in the image is not detected.
  • the face classifier is with a preset size of 40*40.
  • the eye classifier size changing module 5012 is configured to control to decrease a size of the eye classifier until the size of the eye classifier is the same as or smaller than a size of 40*40 if the eye region in the N th image is not detected. And the eye classifier is with a preset size that is the same as the size ofthe N th image.
  • the device of the present disclosure when there is the eye region in the N th image, it is not necessary to search the whole (N+l) th image but only to search the pending eye region in the (N+l) th image corresponding to or larger than the eye region in the N th image. Therefore, a time needed for detecting the eye region is decreased and the eye detecting may be performed quickly, which is suitable for a real-time image system.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Ophthalmology & Optometry (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

An eye detecting method is provided. The method comprises: detecting whether there is a face region in a N th image (S202); if yes, detecting whether there is a first eye region in the N th image and obtaining a detecting result of the N th image (S204); recording the detecting result of the N th image and receiving a (N+1) th image (S206); if the detecting result of the N th image indicates there is the first eye region in the N th image (S208, yes), determining an area in the (N+1) th image corresponding to and larger than the first eye region as a pending eye region in the (N+1) th image (S210), otherwise (S208, no) making N=N+1 and returning to step S202; and detecting whether there is a second eye region in the pending eye region in the (N+1) th image (S212), making N=N+1 and returning to step S206.

Description

EYE DETECTING METHOD AND EYE DETECTING DEVICE
CROSS-REFERENCE TO RELATED APPLICATION
This application claims priority to, and benefits of Chinese Patent Application Serial No. 201010218870.8, filed with the State Intellectual Property Office of P. R. C. on June 29, 2010, the entire contents of which are incorporated herein by reference.
FIELD
The present disclosure relates to an object detecting field, and more particularly, to an eye detecting method and eye detecting device.
BACKGROUND
A research proves that an eye status may reflect a fatigue of a driver reliably, and an eye detection is a premise for judging the eye status. Therefore how to detect the eye correctly and quickly is key to achieve fatigue monitoring in a driving process.
Many researches have been done in a field of eye detection, and a detecting technology includes a method based on study, template matching or characteristics. The method based on study includes, for example, AdaBoost (Adaptive Boosting) algorithm, neural network, and principal component analysis. The algorithms are with high accuracy, but a computation is complicated, which limits an application in a real-time system. The method based on the template matching firstly obtains a template of a right eye and a left eye and then searches in a whole image for matching. Because of an influence of light changing, angle rotation etc., this method has low accuracy and the computation for this method is complicated. The method based on the characteristics includes HIS (Hue Saturation Intensity, representing tone, color saturation and brightness) iris detection, gray projection etc.. The iris detection is fast in speed, but this method should be performed in a color space and is not suitable for an infrared imaging system. The gray projection method will be invalid when a face is deflected. There is also an eye detecting method based on a whole face characteristic, eye gradient etc., but this method is with low accuracy.
In a real image detection, a size of an image to be detected and an amount of matched target objects are unknown in advance. A target image classifier used by the AdaBoost algorithm may scale up or down. Referring to Fig.1, the AdaBoost algorithm comprises: judging whether a size of a scaled up target image classifier is larger than the size of the target image, if yes, terminating the detection, otherwise scaling up the target image classifier to detect the target image until finding a match between the target image classifier and the target image. According to the AdaBoost algorithm, the target images with different sizes are detected by scaling up the target image classifier continuously until the size of the target image classifier is larger than the size of the target image. Therefore, the larger the size of the image is, the larger the amount of the target image classifier is, and the longer the time taken in reviewing the image each time is. Therefore the computation of the AdaBoost algorithm is complex.
SUMMARY
The present disclosure is directed to solve at least one of the above mentioned problems and to provide an eye detecting method and an eye detecting device.
According to an aspect of the present disclosure, an eye detecting method is provided, comprising the steps of: A) detecting whether there is a face region in a Nth image, in which N > 1 ;
B) if there is the face region in the Nth image, detecting whether there is a first eye region in the Nth image and obtaining a detecting result of the Nth image; C) recording the detecting result of the Nth image and receiving a (N+l)th image, otherwise making N=N+1 and returning to step A);
D) if the detecting result of the Nth image indicates there is the first eye region in the Nth image, determining an area in the (N+l)th image corresponding to and larger than the first eye region as a pending eye region in the (N+l)th image according to a location and size information included in the detecting result of the Nth image; and E) detecting whether there is a second eye region in the pending eye region in the (N+l)th image making N=N+1 and returning to step C) .
According to another aspect of the present disclosure, an eye detecting device is provided.
The eye detecting device comprises: a face detecting unit, configured to detect whether there is a face region in an image; an eye detecting unit, configured to detect whether there is an eye region in an image and recording a detecting result of the image; a controlling unit, connected with the face detecting unit and the eye detecting unit, and configured to control the eye detecting unit to detect whether there is a first eye region in a face region in a Nth image and record a detecting result of the Nth image if the face detecting unit detects that there is the face region in the Nth image; and an area determining unit, connected with the controlling unit and configured to determine an area in a (N+l)th image corresponding to and larger than the first eye region as a pending eye region in the (N+l)th image according to a location and size information included in the detecting result of the Nth image, wherein the controlling unit is further configured to control the eye detecting unit to detect whether there is a second eye region in the pending eye region in the (N+l)th image.
According to the present disclosure, when there is the eye region in the Nth image, it is not necessary to search the whole (N+l) th image but only to search the area in the (N+l) th image corresponding to the eye region in the Nth image. Therefore, a time needed for detecting the eye region is decreased and the eye detecting may be performed quickly, which is suitable for a real-time image system.
Additional aspects and advantages of the embodiments of present disclosure will be given in part in the following descriptions, become apparent in part from the following descriptions, or be learned from the practice of the embodiments of the present disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
The above and other aspects and advantages of the disclosure will become apparent and more readily appreciated from the following detailed descriptions taken in conjunction with the accompanying drawings, in which:
Fig. 1 is a flow chart showing an eye detecting method according to a conventional AdaBoost algorithm;
Fig. 2 is a flow chart showing an eye detecting method according to an embodiment of the present disclosure;
Fig. 3 A is a flow chart showing a face region detecting method according to an embodiment of the present disclosure;
Fig. 3B is a flow chart showing a face region detecting method according to an embodiment of the present disclosure;
Fig. 4A is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure; Fig. 4B is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure;
Fig. 4C is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure;
Fig. 4D is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure;
Fig. 5 A is a schematic block diagram of showing an eye detecting device according to one an embodiment of the present disclosure;
Fig. 5B is a schematic block diagram of showing an eye detecting device according to one an embodiment of the present disclosure;
Fig. 5C is a schematic block diagram of showing an eye detecting device according to one an embodiment of the present disclosure;
Fig. 6 is a schematic block diagram showing a face detecting unit according to an embodiment of the present disclosure; and
Fig. 7 is a schematic block diagram showing an eye detecting unit according to an embodiment of the present disclosure.
DETAILED DESCRIPTION OF THE EMBODIMENTS
Reference will be made in detail to embodiments of the present disclosure. The embodiments described herein with reference to drawings are explanatory, illustrative, and used to generally understand the present disclosure. The embodiments shall not be construed to limit the present disclosure.
Referring to Fig. 2, an eye detecting method is provided. The method comprises the following steps.
Step S202), a Nth image is detected to find whether there is a face region in the Nth image, in which N > 1.
Step S204), if there is the face region in the Nth image, it is detected whether there is a first eye region in the Nth image and a detecting result of the Nth image is obtained.
Step S206), the detecting result of the Nth image is recorded and a (N+l)th image is received. Step S208), if the detecting result of the Nth image indicates there is the first eye region in the
Nth image, going to S210, an area in the (N+l)th image corresponding to and larger than the first eye region is determined as an pending eye region in the (N+l)th image according to a location and size information included in the detecting result of the Nth image, otherwise the method makes N=N+1 and returns to step S202.
Step S212), detecting whether there is a second eye region in the pending eye region in the (N+l)th image, and the method makes N=N+1 and returns to step S206.
In some embodiment of the present disclosure, face samples and eye samples are obtained by sampling the image of objects at different angles and by putting an infrared camera in a different light environment. The different light environment comprises but not limited to the environment illuminated by a fluorescent light, an environment without a visible light, at a side of an instrument in a vehicle under different lights. The objects sampled are people with different heights, wearing or not wearing glasses and with different skin colors. Therefore, a sufficient face database is obtained. An eye database is obtained by extracting an eye part from each image in the face database. The face database, the eye database and a database without the face sample and eye sample are established. The face classifier and the eye classifier are obtained by calculating a characteristic value of all the samples in the database and performing selection on the characteristic values. A procedure of obtaining the face classifier comprises the following steps: selecting in the face database and the database without the face sample and eye sample according to a selected group of characteristic values to obtain a first selecting result; adding one or more characteristic values and selecting the image in the first selecting result according the characteristic values with the one or more characteristic values added to obtain the second selecting result; repeating adding and selecting until a selecting error for the images including the face and the images not including the face meets a requirement; obtaining the face classifier including the characteristic value and a parameter. A procedure of obtaining the eye classifier is similar with the procedure of obtaining the face classifier, which will not be described in detail here.
In an embodiment of the present disclosure, if there is no face region in the Nth image, the detecting result of the Nth image is recorded and the (N+l)th image is received.
In an embodiment of the present disclosure, Step S204) comprises extracting the face region and detecting the first eye region in the face region.
Fig. 3 A is a flow chart showing a face region detecting method according to an embodiment of the present disclosure. In an embodiment of the present disclosure, step S202) comprises the following steps:
Step S302), detecting whether there is the face region in the Nth image by using a face classifier of a preset size;
Step S304), terminating detecting whether there is the face region in the Nth image if the face region in the Nth image is detected;
Step S306), if the face region in the Nth image is not detected, enlarging a size of the face classifier and detecting the face region in the Nth image;
Step S308), if the face region in the Nth image is detected, going to step S304, otherwise going to step S310; and
Step S310), comparing the size of the face classifier with the size of the Nth image, if the size of the face classifier is the same as or larger than the size of the Nth image, going to step S304, otherwise going to step S306.
In an embodiment of the present disclosure, the preset size of the face classifier is 40*40.
In an embodiment of the present disclosure, as shown in Fig. 3, step S202) comprises the following steps:
Step S312), detecting whether there is the face region in the Nth image by using a face classifier of a preset size;
Step S314), terminating detecting whether there is the face region in the Nth image if the face region in the Nth image is detected;
Step S316), if the face region in the Nth image is not detected, decreasing a size of the face classifier and detecting whether there is the face region in the Nth image.
Step S318), if the face region in the Nth image is detected, going to step S304, otherwise going to step S320.
Step S320), comparing the size of the face classifier with the size of 40*40, if the size of the face classifier is the same as or small than the size of 40*40, going to step S314, else going to step S316.
In an embodiment of the present disclosure, the preset size of the face classifier is the same as the size ofthe Nth image.
Fig. 4A-4D is a flow chart showing an eye region detecting method according to an embodiment of the present disclosure.
In an embodiment of the present disclosure, as shown in Fig. 4A, the step S204 comprises the following steps:
Step S402), detecting whether there is the first eye region in the Nth image;
Step S404), terminating detecting whether there is the first eye region in the Nth image if the first eye region in the Nth image is detected;
Step S406), if the first eye region in the Nth image is not detected, enlarging a size of the eye classifier and detecting whether there is the first eye region in the Nth image;
Step S408), if the first eye region in the Nth image is not detected, going to step S404, otherwise going to step S410; and
Step S410), comparing the size of the eye classifier with the size of the Nth image, if the size of the eye classifier is the same as or larger than the size of the Nth image, going to step S404, else going to step S406.
In an embodiment of the present disclosure, the preset size of the eye classifier is 40*40. In an embodiment of the present disclosure, as shown in Fig. 4B, the step S204 comprises the following steps:
Step 412), detecting whether there is the first eye region in the Nth image;
Step 414), terminating detecting whether there is the first eye region in the Nth image if the first eye region in the Nth image is detected;
Step S416), if the first eye region in the Nth image is not detected, decreasing a size of the eye classifier and detecting whether there is the first eye region in the Nth image;
Step S418), if the first eye region in the Nth image is not detected, going to step S414, else going to step S420; and
Step S420), comparing the size of the eye classifier with the size of 40*40, if the size of the eye classifier is the same as or smaller than the size of 40*40, going to step S414, else going to step S416.
In an embodiment of the present disclosure, the preset size of the eye classifier is the same as the size of the first eye region.
In an embodiment of the present disclosure, as shown in Fig. 4C, the step S212 comprises the following steps:
Step S422), detecting whether there is the second eye region is detected in the (N+l)th image; Step S424), terminating detecting whether there is the second eye region in the (N+l)th image if the second eye region in the (N+l)th image is detected; Step S426), if the second eye region in the (N+l)th image is not detected, enlarging a size of the eye classifier and detecting whether there is the second eye region in the (N+l)th image;
Step S428), if the second eye region in the (N+l)th image is not detected, going to step S404, otherwise going to step S430; and
Step S430), comparing the size of the eye classifier with the size of the (N+l)th image, if the size of the eye classifier is the same as or larger than the size of the (N+l)th image, going to step S424, otherwise going to step S426.
In an embodiment of the present disclosure, the preset size of the eye classifier is 40*40.
In an embodiment of the present disclosure, as shown in Fig. 4D, the step S212 comprises the following steps:
Step 432), detecting whether there is the second eye region in the (N+l)th image;
Step 434), terminating detecting whether there is the second eye region in the (N+l)th image if the second eye region in the (N+l)th image is detected;
Step S436), if the second eye region in the (N+l)th image is not detected, decreasing a size of the eye classifier and detecting whether there is the second eye region in the (N+l)th image;
Step S438), if the second eye region in the (N+l)th image is not detected, going to step S414, otherwise going to step S440; and
Step S440), comparing the size of the eye classifier with the size of 40*40, if the size of the eye classifier is the same as or smaller than the size of 40*40, going to step S434, otherwise going to step S436.
In an embodiment of the present disclosure, the preset size of the eye classifier is the same as the size of the second eye region.
According to the method of the present disclosure, when there is the eye region in the Nth image, it is not necessary to search the whole (N+l)th image but only to search the pending eye region in the (N+l)th image corresponding to and larger than the first eye region in the Nth image. Therefore, a time needed for detecting the eye region is reduced and the eye detecting may be performed quickly, which is suitable for a real-time image system.
In an embodiment of the present disclosure, an eye detection device is provided. Referring to Fig. 5 A, the eye detecting device 500 comprises a face detecting unit 501, an eye detecting unit 502, a controlling unit 503 and an area determining unit 504. The face detecting unit 501 is configured to detect whether there is a face region in an image. The eye detecting unit 502 is configured to detect whether there is an eye region in an image and recording a detecting result of the image. The controlling unit 503 is connected with the face detecting unit 501 and the eye detecting unit 502, and is configured to control the eye detecting unit 502 to detect whether there is a first eye region in a face region in a Nth image and record a detecting result of the Nth image.
The area determining unit 504 is connected with the controlling unit 503 and is configured to determine an area in a (N+l)th image corresponding to and larger than the first eye region as an pending eye region in the (N+l)th image according to a location and size information included in the detecting result of the Nth image. In an embodiment of the present disclosure, the controlling unit 503 is further configured to control the eye detecting unit 502 to detect whether there is a second eye region in the pending eye region in the (N+l)th image. In an embodiment of the present disclosure, the first eye region and the second eye region are a rectangular region.
In an embodiment of the present disclosure, the eye detecting device 500 may further comprise a face extracting unit. Referring to Fig. 5B, the face extracting unit is connected with the face detecting unit 501 and the eye detecting unit 502 and is configured to extract the face region in the image. The eye detecting unit 502 detects the first eye region in the face region.
In an embodiment of the present disclosure, the eye detecting unit 502 may further comprise a first eye detecting unit 510 and a second eye detecting unit 508. Referring to Fig. 5C, the first eye detecting unit 510 is connected with the face extracting unit 506 and configured to detect whether there is the first eye region in the face region. The second eye detecting unit 508 is connected with the area determining unit 504 and is configured to detect whether there is the second eye region in the pending eye region.
Fig. 6 is a schematic block diagram showing a face detecting unit according to an embodiment of the present disclosure.
In an embodiment of the present disclosure, as shown in Fig. 6, the face detecting unit 501 comprises a detecting module 5011, a face classifier size changing module 5012 and a first sub-controlling module 5013. The detecting module 5011 is configured to detect the face region in the Nth image. The face classifier size change module 5012 is configured to enlarge or decrease the size of the face classifier. The first sub-controlling module 5013 is connected with the detecting module 5011. In an embodiment, the face classifier size changing module 5012, and is configured to control to enlarge a size of the face classifier until the size of the face classifier is the same as or larger than a size of the Nth image if the face region in the Nth image is not detected. And the face classifier is with a preset size of 40*40. In an embodiment, the face classifier size changing module 5012 is configured to control to decrease a size of the face classifier until the size of the face classifier is the same as or smaller than a size of 40*40 if the face region in the Nth image is not detected. And the face classifier is with a preset size that is the same as the size of the Nth image.
Fig. 7 is a schematic block diagram showing an eye detecting unit according to an embodiment of the present disclosure.
In an embodiment of the present disclosure, as shown in Fig. 7, the eye detecting unit 502 comprises a detecting module 5021, an eye classifier size changing module 5022 and a second sub-controlling module 5023. The detecting module 5021 is configured to detect the eye region in an image. The eye classifier size changing module 5022 is configured to enlarge or decreases the eye classifier. The second sub-controlling module 5023 is connected with the detecting module 5021. In an embodiment, the eye classifier size changing module 5022, and is configured to control to enlarge a size of the eye classifier until the size of the eye classifier is the same as a size of the image if the eye region in the image is not detected. And the face classifier is with a preset size of 40*40. In an embodiment, the eye classifier size changing module 5012, and is configured to control to decrease a size of the eye classifier until the size of the eye classifier is the same as or smaller than a size of 40*40 if the eye region in the Nth image is not detected. And the eye classifier is with a preset size that is the same as the size ofthe Nth image.
According to the device of the present disclosure, when there is the eye region in the Nth image, it is not necessary to search the whole (N+l)th image but only to search the pending eye region in the (N+l) th image corresponding to or larger than the eye region in the Nth image. Therefore, a time needed for detecting the eye region is decreased and the eye detecting may be performed quickly, which is suitable for a real-time image system.
Although explanatory embodiments have been shown and described, it would be appreciated by those skilled in the art that changes, alternatives, and modifications can be made in the embodiments without departing from spirit and principles of the disclosure. Such changes, alternatives, and modifications all fall into the scope of the claims and their equivalents.

Claims

WHAT IS CLAIMED IS:
1. An eye detecting method, comprising the steps of:
A) detecting whether there is a face region in a Nth image, in which N > 1 ;
B) if there is the face region in the Nth image, detecting whether there is a first eye region in the Nth image and obtaining a detecting result of the Nth image;
C) recording the detecting result of the Nth image and receiving a (N+l)th image;
D) if the detecting result of the Nth image indicates there is the first eye region in the Nth image, determining an area in the (N+l)th image corresponding to and larger than the first eye region as a pending eye region in the (N+l)th image according to a location and size information included in the detecting result of the Nth image, otherwise making N=N+1 and returning to step A); and
E) detecting whether there is a second eye region in the pending eye region in the (N+l)th image, making N=N+1 and returning to step C).
2. The eye detecting method of claim 1, further comprising the step of:
if there is no face region in the Nth image, proceeding to step (C).
3. The eye detecting method of claim 1, wherein step B) comprises:
extracting the face region and detecting whether there is the first eye region in the face region.
4. The eye detecting method of claim 1, wherein step A) comprises the steps of:
al) detecting whether there is the face region in the Nth image by using a face classifier of a preset size;
a2) terminating detecting whether there is the face region in the Nth image if the face region in the Nth image is detected;
a3) if the face region in the Nth image is not detected, enlarging a size of the face classifier and detecting whether there is the face region in the Nth image; and
a4) if the face region in the Nth image is not detected, repeating step a3) until the size of the face classifier is the same as or larger than a size of the Nth image, otherwise terminating detecting whether there is the face region in the Nth image.
5. The eye detecting method of claim 1, wherein step A) comprises the steps of: al) detecting whether there is the face region in the Nth image by using a face classifier of a preset size;
a2) terminating detecting whether there is the face region in the Nth image if the face region in the Nth image is detected;
a3) if the face region in the Nth image is not detected, decreasing a size of the face classifier and detecting whether there is the face region in the Nth image; and
a4) if the face region in the Nth image is not detected, repeating step a3) until the size of the face classifier is or is smaller than 40*40 pixels, otherwise terminating detecting whether there is the face region in the Nth image.
6. The eye detecting method of claim 1, wherein detecting whether there is the first eye region comprises the steps of:
al) detecting whether there is the first eye region in the Nth image by using an eye classifier of a preset size;
a2) terminating detecting whether there is the first eye region in the Nth image if the first eye region in the Nth image is detected;
a3) if the first eye region in the Nth image is not detected, enlarging a size of the eye classifier and detecting whether there is the first eye region in the Nth image; and
a4) if the first eye region in the Nth image is not detected, repeating step a3) until the size of the eye classifier is the same as or larger than a size of the Nth image, otherwise terminating detecting whether there is the first eye region in the Nth image.
7. The eye detecting method of claim 1, wherein detecting whether there is the first eye region comprises the steps of:
al) detecting whether there is the first eye region in the Nth image by using an eye classifier of a preset size;
a2) terminating detecting whether there is the first eye region in the Nth image if the first eye region in the Nth image is detected;
a3) if the first eye region in the Nth image is not detected, decreasing a size of the eye classifier and detecting whether there is the first eye region in the Nth image; and a4) if the first eye region in the Nth image is not detected, repeating step a3) until the size of the eye classifier is or is smaller than 40*40 pixels, otherwise terminating detecting whether there is the first eye region in the Nth image.
8. The eye detecting method of claim 1, wherein the second eye region comprises the steps of: al) detecting whether there is the second eye region in the (N+l)th image by using an eye classifier of a preset size;
a2) terminating detecting whether there is the second eye region in the (N+l)th image if the second eye region in the (N+l)th image is detected;
a3) if the second eye region in the (N+l)th image is not detected, enlarging a size of the eye classifier and detecting whether there is the second eye region in the (N+l)th image; and a4) if the second eye region in the (N+l)th image is not detected, repeating step a3) until the size of the eye classifier is the same as or larger than a size of the (N+l)th image, otherwise terminating detecting whether there is the second eye region in the (N+l)th image.
9. The eye detecting method of claim 1, wherein detecting the second eye region comprises the steps of:
al) detecting whether there is the second eye region in the (N+l)th image by using an eye classifier of a preset size;
a2) terminating detecting whether there is the second eye region in the (N+l)th image if the second eye region in the (N+l)th image is detected;
a3) if the second eye region in the (N+l)th image is not detected, decreasing a size of the eye classifier and detecting whether there is the second eye region in the (N+l)th image; and a4) if the second eye region in the (N+l)th image is not detected, repeating step a3) until the size of the eye classifier is or is smaller than 40*40 pixels, otherwise terminating detecting whether there is the second eye region in the (N+l)th image.
10. An eye detecting device, comprising:
a face detecting unit, configured to detect whether there is a face region in an image; an eye detecting unit, configured to detect whether there is an eye region in an image and recording a detecting result of the image;
a controlling unit, connected with the face detecting unit and the eye detecting unit, and configured to control the eye detecting unit to detect whether there is a first eye region in a face region in a Nth image and record a detecting result of the Nth image if the face detecting unit detects that there is the face region in the Nth image; and
an area determining unit, connected with the controlling unit and the eye detecting unit and configured to determine an area in a (N+l)th image corresponding to and larger than the first eye region as a pending eye region in the (N+l)th image according to a location and size information included in the detecting result of the Nth image,
wherein the controlling unit is further configured to control the eye detecting unit to detect whether there is a second eye region in the pending eye region in the (N+l)th image.
11. The eye detecting device of claim 8 further comprising a face extracting unit, connected with the face detecting unit and the eye detecting unit and configured to extract the face region in the image.
12. The eye detecting device of claim 11, wherein the eye detecting comprises:
a first eye detecting unit, connected with the face extracting unit and configured to detect whether there is the first eye region in the face region; and
a second eye detecting unit, connected with the area determining unit and configured to detect whether there is the second region in the pending eye region.
13. The eye detecting device of claim 8, wherein the face detecting unit comprises:
a detecting module, configured to detect whether there is the face region in the Nth image by using the face classifier of a preset size;
a face classifier enlarging module, configured to enlarge the face classifier of a preset size; and
a first sub-controlling module, connected with the detecting module and the face classifier enlarging module, and configured to control to enlarging a size of the face classifier and to detect whether there is the face region in the Nth image until the size of the face classifier is the same as or larger than a size of the Nth image if the face region in the Nth image is not detected.
14. The eye detecting device of claim 8, wherein the face detecting unit comprises:
a detecting module, configured to detect whether there is the face region in the Nth image by using the face classifier of a preset size;
a face classifier decreasing module, configured to decrease the face classifier of a preset size; and
a first sub-controlling module, connected with the detecting module and the face classifier decreasing module, and configured to control to decrease a size of the face classifier and to detect whether there is the face region in the Nth image until the size of the face classifier is or smaller than 40*40 pixels if the face region in the Nth image is not detected.
15. The eye detecting device of claim 8, wherein the eye detecting unit comprises:
a detecting module, configured to detect whether there is the eye region in an image by using the eye classifier of a preset size;
an eye classifier enlarging module, configured to enlarge the eye classifier of a preset size; and
a second sub-controlling module, connected with the detecting module and the eye classifier enlarging module, and configured to control to enlarging a size of the eye classifier and to detect whether there is the eye region in the image until the size of the eye classifier is the same as or larger than a size of the image if the eye region in the image is not detected.
16. The eye detecting device of claim 8, wherein the eye detecting unit comprises:
a detecting module, configured to detect whether there is the eye region in an image by using the eye classifier of a preset size;
an eye classifier decreasing module, configured to decrease the eye classifier of a preset size; and
a second sub-controlling module, connected with the detecting module and the eye classifier decreasing module, and configured to control to decrease a size of the eye classifier and to detect whether there is the eye region in the image until the size of the eye classifier is or less than 40*40 pixels if the eye region in the image is not detected.
PCT/CN2011/076339 2010-06-29 2011-06-24 Eye detecting method and eye detecting device Ceased WO2012000408A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201010218870.8A CN102314589B (en) 2010-06-29 2010-06-29 Fast human-eye positioning method and device
CN201010218870.8 2010-06-29

Publications (1)

Publication Number Publication Date
WO2012000408A1 true WO2012000408A1 (en) 2012-01-05

Family

ID=45401392

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2011/076339 Ceased WO2012000408A1 (en) 2010-06-29 2011-06-24 Eye detecting method and eye detecting device

Country Status (2)

Country Link
CN (1) CN102314589B (en)
WO (1) WO2012000408A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106709420A (en) * 2016-11-21 2017-05-24 厦门瑞为信息技术有限公司 Method for monitoring driving behaviors of driver of commercial vehicle
CN107292251A (en) * 2017-06-09 2017-10-24 湖北天业云商网络科技有限公司 A kind of Driver Fatigue Detection and system based on human eye state

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102339377B (en) * 2010-07-21 2013-12-04 比亚迪股份有限公司 Quick human-eye positioning method and device
CN103942542A (en) * 2014-04-18 2014-07-23 重庆卓美华视光电有限公司 Human eye tracking method and device
CN104573660A (en) * 2015-01-13 2015-04-29 青岛大学 Method for precisely positioning human eyes by SIFT point descriptor
CN105740848B (en) * 2016-03-02 2019-05-17 武汉虹识技术有限公司 A kind of fast human-eye positioning method based on confidence level
CN106295511B (en) * 2016-07-26 2019-05-21 北京小米移动软件有限公司 Face tracking method and device
CN106709431A (en) * 2016-12-02 2017-05-24 厦门中控生物识别信息技术有限公司 Iris recognition method and device
CN107622252B (en) * 2017-09-29 2022-02-22 百度在线网络技术(北京)有限公司 Information generation method and device
CN107766840A (en) * 2017-11-09 2018-03-06 杭州有盾网络科技有限公司 A kind of method, apparatus of blink detection, equipment and computer-readable recording medium
CN108985245A (en) * 2018-07-25 2018-12-11 深圳市飞瑞斯科技有限公司 Determination method, apparatus, computer equipment and the storage medium of eye locations
CN110929672B (en) * 2019-11-28 2024-03-01 联想(北京)有限公司 Pupil positioning method and electronic equipment

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1475961A (en) * 2003-07-14 2004-02-18 中国科学院计算技术研究所 Human Eye Positioning Method Based on GaborEye Model
CN1940967A (en) * 2005-09-26 2007-04-04 富士胶片株式会社 Method, apparatus, and program for dividing images
CN101216887A (en) * 2008-01-04 2008-07-09 浙江大学 Computer Automatic Discrimination Method of Photographic Face and Live Human Face

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100809431B1 (en) * 2006-11-21 2008-03-05 한국전자통신연구원 Robust Eye Detection Method and Apparatus
CN101739548B (en) * 2009-02-11 2012-07-25 北京智安邦科技有限公司 Eye detection method and system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1475961A (en) * 2003-07-14 2004-02-18 中国科学院计算技术研究所 Human Eye Positioning Method Based on GaborEye Model
CN1940967A (en) * 2005-09-26 2007-04-04 富士胶片株式会社 Method, apparatus, and program for dividing images
CN101216887A (en) * 2008-01-04 2008-07-09 浙江大学 Computer Automatic Discrimination Method of Photographic Face and Live Human Face

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106709420A (en) * 2016-11-21 2017-05-24 厦门瑞为信息技术有限公司 Method for monitoring driving behaviors of driver of commercial vehicle
CN106709420B (en) * 2016-11-21 2020-07-10 厦门瑞为信息技术有限公司 Method for monitoring driving behavior of commercial vehicle driver
CN107292251A (en) * 2017-06-09 2017-10-24 湖北天业云商网络科技有限公司 A kind of Driver Fatigue Detection and system based on human eye state
CN107292251B (en) * 2017-06-09 2020-08-28 湖北天业云商网络科技有限公司 Driver fatigue detection method and system based on human eye state

Also Published As

Publication number Publication date
CN102314589A (en) 2012-01-11
CN102314589B (en) 2014-09-03

Similar Documents

Publication Publication Date Title
WO2012000408A1 (en) Eye detecting method and eye detecting device
CN102426649B (en) Simple steel seal digital automatic identification method with high accuracy rate
JP4505362B2 (en) Red-eye detection apparatus and method, and program
US9704017B2 (en) Image processing device, program, image processing method, computer-readable medium, and image processing system
US8594435B2 (en) Image processing device and method, and program therefor
JP4877374B2 (en) Image processing apparatus and program
US9064178B2 (en) Edge detection apparatus, program and method for edge detection
WO2022105655A1 (en) Image processing method, image processing apparatus, electronic device, and computer readable storage medium
EP3905104B1 (en) Living body detection method and device
RU2008143205A (en) EFFICIENT CODING OF MANY SPECIES
EP2959454A1 (en) Method, system and software module for foreground extraction
US20100322510A1 (en) Sky detection system used in image extraction device and method using sky detection system
US8055016B2 (en) Apparatus and method for normalizing face image used for detecting drowsy driving
EP2238744A1 (en) Automatic redeye detection
US8655060B2 (en) Night-scene light source detecting device and night-scene light source detecting method
JP2011165170A (en) Object detection device and program
EP3137896A1 (en) Method and apparatus for processing block to be processed of urine sediment image
Fernández-Caballero et al. Display text segmentation after learning best-fitted OCR binarization parameters
CN115393708B (en) Multi-level collaboration-based multi-source remote sensing image fusion and target perception method and device
CN117351556A (en) Gesture recognition method, device, vehicle and storage medium
CN119048507A (en) Mobile phone volume key flaw detection method
US8538142B2 (en) Face-detection processing methods, image processing devices, and articles of manufacture
CN102004537B (en) System power-on and power-off control device and method
JP6855175B2 (en) Image processing equipment, image processing methods and programs
JP2018072884A (en) Information processing device, information processing method and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11800153

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11800153

Country of ref document: EP

Kind code of ref document: A1