CN109376624A - A kind of modification method and device of eye closing photo - Google Patents

A kind of modification method and device of eye closing photo Download PDF

Info

Publication number
CN109376624A
CN109376624A CN201811171136.3A CN201811171136A CN109376624A CN 109376624 A CN109376624 A CN 109376624A CN 201811171136 A CN201811171136 A CN 201811171136A CN 109376624 A CN109376624 A CN 109376624A
Authority
CN
China
Prior art keywords
photo
eye
face
human eye
human
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811171136.3A
Other languages
Chinese (zh)
Inventor
何健
柳瑞超
梅芬
殷飞
杨建军
杜玉强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics China R&D Center
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics China R&D Center
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics China R&D Center, Samsung Electronics Co Ltd filed Critical Samsung Electronics China R&D Center
Priority to CN201811171136.3A priority Critical patent/CN109376624A/en
Publication of CN109376624A publication Critical patent/CN109376624A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168Feature extraction; Face representation
    • G06V40/171Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/90Dynamic range modification of images or parts thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172Classification, e.g. identification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person
    • G06T2207/30201Face

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

This application discloses a kind of modification methods of eye closing photo, comprising: human eye area is identified in current photo, and detecting is to open eyes or close one's eyes;When testing result is to close one's eyes, the face where the human eye area identified, and the most matched eye opening human face photo of the face inquired and determined in the terminal are determined;Using the eye opening human face photo inquired, the human eye area identified in current photo is corrected.Using the application, the amendment of eye closing photo can be carried out automatically.

Description

A kind of modification method and device of eye closing photo
Technical field
This application involves image processing techniques, in particular to a kind of the modification method and device of eye closing photo.
Background technique
During daily take pictures, the case where often will appear eye closing.For eye closing photo, image procossing skill can be passed through Art is modified to eye opening photo.Specifically, existing eye closing photo modification method mainly includes following processing:
1. user chooses an eye opening photo;
2. user chooses eye areas and copies on eye closing photo;
3. the various corrected parameters of user setting;
4. system generates new photo according to the Parameter fusion of setting after user's confirmation.
Above-mentioned eye closing photo modification method needs user constantly to participate in, not smart enough.Meanwhile in the above-mentioned methods, it uses Family is difficult to select to be more suitable for the eye opening photo that current eye closing photo is merged in replacement in the picture library of magnanimity, and be arranged Also it is easy to appear mistakes to cause generation photo not beautiful enough for corrected parameter.
Summary of the invention
The application provides a kind of modification method of eye closing photo, can realize the amendment of eye closing photo automatically.
To achieve the above object, the application adopts the following technical scheme that
A kind of modification method of eye closing photo, comprising:
Human eye area is identified in current photo, and detecting is to open eyes or close one's eyes;
When testing result is to close one's eyes, determine the face where the human eye area that identifies, and in the terminal inquiry with really The most matched eye opening human face photo of the face made;Using the eye opening human face photo inquired, corrects and identified in current photo Human eye area.
Preferably, described identify that human eye area includes: in current photo
It identifies away the human face region in photo, positions facial key feature points, and crucial special according to the face oriented Eye feature point in sign point, orients human eye area.
Preferably, the detection is the mode opened eyes or closed one's eyes are as follows: pass through deep neural network CNN training and/or SVM Obtained detection of classifier opens eyes or closes one's eyes, and/or, the connected region judgement based on threshold value is eye opening or closes one's eyes, and/ Or, the approximate judgement determination using human eye area height is to open eyes or close one's eyes.
It is shone preferably, being inquired in systems according to preset matching rule with the most matched eye opening face of face determined Piece.
Preferably, the matching rule includes: the higher matching degree of face consistency in alternative photo with current photo In higher, alternative photo in the photo higher, alternative with the higher matching of expression consistency of current photo with current photo Higher, the alternative photo of the higher matching degree of face head pose consistency in human eye area corresponding with the face of current photo In the higher, and/or alternative photo of the more consistent matching degree of size more with the actinic more consistent matching degree of the face of current photo It is high.
Preferably, the human eye area packet identified in the eye opening human face photo amendment current photo that the utilization inquires It includes:
Human eye area is intercepted in the eye opening human face photo inquired;It is abscissused using the human eye angle deemphasis in current photo The angle of the human eye area taken;Utilize the illumination for the human eye area that the illumination deemphasis in current photo is just intercepting;Utilize interception Human eye area after the correction of human eye area in picture and current photo is merged.
Preferably, the illumination for the human eye area that the illumination deemphasis in the current photo is just intercepting includes: to be become using linear Change the corresponding correction of realization.
Preferably, when the testing result is to close one's eyes, it, should before determining the face where the human eye area identified Method further comprises:
It reminds user's current photo that there are eye closing situations, if receiving the revision directive of user, continues to execute described true Surely the processing for the personnel area identified;Otherwise, terminate this process.
A kind of correcting device of eye closing photo, comprising: eye recognition module, human eye state detection module, human eye match mould Block and eye closing correction module;
The eye recognition module, for identifying human eye area in current photo;
The human eye state detection module is eye opening or eye closing for detecting in the human eye area, and in determination After eye closing, the human eye matching module is notified;
The human eye matching module, for after receiving notice, determining the face where the human eye area identified, and The most matched eye opening human face photo of the face inquired and determined in systems;
The eye closing correction module identifies in amendment current photo for utilizing the eye opening human face photo inquired Human eye area.
As seen from the above technical solution, in the application, identify human eye area in current photo, and detect be open eyes or It closes one's eyes;When testing result is to close one's eyes, the face where the human eye area identified is determined, and inquire and determine in systems The most matched eye opening human face photo of face;Using the eye opening human face photo inquired, the people identified in current photo is corrected Vitrea eye domain.By above-mentioned processing, the amendment of eye closing photo can be realized automatically.
Detailed description of the invention
Fig. 1 is the basic structure schematic diagram of eye closing photo correcting device in the application;
Fig. 2 is the basic procedure schematic diagram of eye closing photo modification method in the application;
Fig. 3 is eye recognition schematic diagram;
Fig. 4 is human eye state detection schematic diagram;
Fig. 5 is the flow diagram that inquiry most matches photo;
Fig. 6 is training stage and the processing stage flow diagram of micro- expression classifier;
Fig. 7 is the network model schematic diagram of micro- expression classifier;
Fig. 8 is the schematic diagram of eye closing correcting process;
Fig. 9 is the schematic diagram of eye closing photo amendment front and back in scene one;
Figure 10 is the schematic diagram of eye closing photo amendment front and back in scene two;
Figure 11 is the schematic diagram of eye closing photo amendment front and back in scene three.
Specific embodiment
In order to which the purpose, technological means and advantage of the application is more clearly understood, the application is done below in conjunction with attached drawing It is further described.
The application provides the modification method and device of a kind of eye closing photo.Fig. 1 is the basic structure schematic diagram of correcting device, As shown in Figure 1, the device includes: eye recognition module, human eye state detection module, human eye matching module and amendment mould of closing one's eyes Block.
Wherein, eye recognition module, for identifying human eye area in current photo.Human eye state detection module, is used for Detection is to open eyes or close one's eyes, and after determining eye closing, notify human eye matching module in human eye area.Human eye matching module, For after receiving notice, determining the face where the human eye area identified, and the people for inquiring and determining in systems The most matched eye opening human face photo of face.Eye closing correction module, for correcting current photo using the eye opening human face photo inquired In the human eye area that identifies.
Fig. 2 is the modification method basic procedure schematic diagram of eye closing photo in the application.Below in conjunction with the device of Fig. 1, introduce Modification method specifically includes:
Step 201, human eye area is identified in current photo.
Eye recognition module identifies the human eye area in current photo.It mainly include recognition of face, face alignment and people Vitrea eye domain determines.Recognition of face and face alignment techniques comparative maturity have many open source libraries to support, such as dlib, opencv Deng.The specific mode for determining human eye area can be with are as follows: (1) identifies the region of face in current photo;(2) face positioned in alignment Facial key feature points out, such as eyes, nose, the corners of the mouth point, eyebrow and each component outline point of face;(3) basis is oriented Facial key features point in eye feature point, orient eye areas.Detailed process refers to Fig. 3 eye recognition schematic diagram.
Step 202, detection is to open eyes or close one's eyes, and continue to execute step 203 after determining eye closing in human eye area.
Human eye state detection module is used to detect that the state of human eye to be to close one's eyes or open eyes.Human eye state detection specifically may be used So that one of with the following methods or multiple combinations: (1) one classifier of deep neural network CNN training utilizes classifier Carry out state-detection;(2) SVM (Support Vector Machine) training classifier carries out state inspection using the classifier It surveys;(3) the connected region judgement based on threshold value, carries out state-detection;(4) it by the approximate judgement of human eye area height, carries out State-detection.The integrated mode of aforesaid way can be used during actual development, the accuracy rate of monitoring state is higher.Stream in detail Journey refers to Fig. 4 human eye state detection schematic diagram.
Step 203, it reminds user's current photo there are eye closing situation, prompts the user to indicate whether to need to correct, if so, Step 204 is executed, otherwise, executes step 208.
The processing of this step is optional processing, can not include this step, directly execution step 204 in practical applications.
Step 204, the face where the human eye area identified is determined.
Step 205, the most matched eye opening human face photo of the face inquired and determined in systems.
Human eye matching module is used for from the terminal (such as face database (eye opening comprising various standards that picture library or system are included Photo) etc.) in find out and the most matched alternative photo of eye closing photo, for replacing the eye closing region in fusion current photo.? It can be carried out according to the matching rule of setting when inquiry most matching photo.Wherein, matching priority can be according to following rule: (1) alternative photo must be opened eyes;(2) alternative human face photo and the face consistency currently closed one's eyes are higher, it is determined that It is higher with spending;(3) alternative face and the micro- expression consistency of the face currently closed one's eyes are higher, it is determined that matching degree is higher;(4) standby The face of choosing and the face head pose consistency currently closed one's eyes are higher, it is determined that matching degree is higher;(5) alternative face and work as The corresponding human eye area size of the face of preceding eye closing is more consistent, it is determined that matching degree is higher;(6) alternative face and current eye closing Face it is actinic more consistent, it is determined that matching degree is higher.Detailed process can refer to Fig. 5.Wherein, micro- expression classifier instruction The detail flowchart for practicing stage and processing stage may refer to Fig. 6.The network model of micro- expression classifier is based on MobileNet's Optimization, detailed model is referring to Fig. 7.
Step 206, using the eye opening human face photo inquired, the human eye area identified in current photo is corrected.
Eye closing correction module goes to the eye closing region in amendment current photo with the eye opening photo recommended in human eye matching module. Key step can carry out as follows: (1) human eye area in the eye opening photo that interception is recommended;(2) it uses in eye closing photo and corresponds to people Angle of the angle deemphasis of eye just when the human eye area of preceding interception;(3) with the illumination deemphasis in eye closing photo just when preceding interception The illumination of human eye area, such as simply usable linear transformation;(4) picture with the eye opening Area generation after correction and eye closing Corresponding eye closing region is merged in photo, can specifically use existing amalgamation mode, such as common graph cut scheme Effect.Detailed process closes one's eyes with reference to Fig. 8 and corrects schematic diagram.
Step 207, eye opening photo is generated.
Step 208, the eye opening photo of generation is stored.
The example of several amendment eye closing photos is given below.
Scene 1: often occur the situation of some people eye closing when group takes pictures, be adapted to and opened automatically by the present processes Eye, as shown in Figure 9.
Scene 2: all eye closing photos are corrected to opening eyes by one key of the present processes in picture library app, certain also may be selected A little eye closing photo amendments, as shown in Figure 10.
Scene 3: when camera app takes pictures, eye closing situation is detected automatically, and user is reminded to be adapted to eye opening, such as Figure 11 It is shown.
As above the eye closing photo modification method and device for passing through the application, can easily correct eye closing photo, pole automatically The earth promotes user experience.
The foregoing is merely illustrative of the preferred embodiments of the present invention, is not intended to limit the invention, all in essence of the invention Within mind and principle, any modification, equivalent substitution, improvement and etc. done be should be included within the scope of the present invention.

Claims (9)

1. a kind of modification method of eye closing photo characterized by comprising
Human eye area is identified in current photo, and detecting is to open eyes or close one's eyes;
When testing result is to close one's eyes, the face where the human eye area identified is determined, and inquire and determine in the terminal The most matched eye opening human face photo of face;Using the eye opening human face photo inquired, the people identified in current photo is corrected Vitrea eye domain.
2. the method according to claim 1, wherein described identify that human eye area includes: in current photo
It identifies away the human face region in photo, positions facial key feature points, and according to the facial key features point oriented In eye feature point, orient human eye area.
3. the method according to claim 1, wherein the detection is the mode opened eyes or closed one's eyes are as follows: pass through The detection of classifier that deep neural network CNN training and/or SVM are obtained opens eyes or closes one's eyes, and/or, the connection based on threshold value Region decision be open eyes or close one's eyes, and/or, approximate judgement determinations using human eye area height is to open eyes or eye closing.
4. the method according to claim 1, wherein inquiring and determining according to preset matching rule in systems The most matched eye opening human face photo of face out.
5. according to the method described in claim 4, it is characterized in that, the matching rule includes: in alternative photo and current In higher, the alternative photo of the higher matching degree of face consistency of photo more with the higher matching of expression consistency of current photo In high, alternative photo in the photo higher, alternative with the higher matching degree of face head pose consistency of current photo with work as In the higher, and/or alternative photo of the more consistent matching degree of the corresponding human eye area size of the face of preceding photo with current photo The actinic more consistent matching degree of face is higher.
6. the method according to claim 1, wherein the eye opening human face photo amendment that the utilization inquires is current The human eye area identified in photo includes:
Human eye area is intercepted in the eye opening human face photo inquired;It is just being intercepted using the human eye angle deemphasis in current photo The angle of human eye area;Utilize the illumination for the human eye area that the illumination deemphasis in current photo is just intercepting;Utilize the human eye of interception Human eye area after the correction in region in picture and current photo is merged.
7. according to the method described in claim 6, it is characterized in that, the human eye that the illumination deemphasis in the current photo is just intercepting The illumination in region includes: to realize corresponding correction using linear transformation.
8. according to claim 1 to any method in 7, which is characterized in that when the testing result is to close one's eyes, true Surely before the face where the human eye area identified, this method further comprises:
It reminds user's current photo that there are eye closing situations, if receiving the revision directive of user, continues to execute the determining knowledge Not Chu personnel area processing;Otherwise, terminate this process.
9. a kind of correcting device of eye closing photo, which is characterized in that the device includes: eye recognition module, human eye state detection Module, human eye matching module and eye closing correction module;
The eye recognition module, for identifying human eye area in current photo;
The human eye state detection module is to open eyes or close one's eyes, and close one's eyes determining for detecting in the human eye area Afterwards, the human eye matching module is notified;
The human eye matching module, for determining the face where the human eye area identified, and be after receiving notice The most matched eye opening human face photo of the face inquired and determined in system;
The eye closing correction module, for correcting the human eye identified in current photo using the eye opening human face photo inquired Region.
CN201811171136.3A 2018-10-09 2018-10-09 A kind of modification method and device of eye closing photo Pending CN109376624A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811171136.3A CN109376624A (en) 2018-10-09 2018-10-09 A kind of modification method and device of eye closing photo

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811171136.3A CN109376624A (en) 2018-10-09 2018-10-09 A kind of modification method and device of eye closing photo

Publications (1)

Publication Number Publication Date
CN109376624A true CN109376624A (en) 2019-02-22

Family

ID=65403206

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811171136.3A Pending CN109376624A (en) 2018-10-09 2018-10-09 A kind of modification method and device of eye closing photo

Country Status (1)

Country Link
CN (1) CN109376624A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110348331A (en) * 2019-06-24 2019-10-18 深圳和而泰家居在线网络科技有限公司 Face identification method and electronic equipment
CN111340688A (en) * 2020-02-24 2020-06-26 网易(杭州)网络有限公司 Method and device for generating eye closing image
CN112036311A (en) * 2020-08-31 2020-12-04 北京字节跳动网络技术有限公司 Image processing method and device based on eye state detection and storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106934329A (en) * 2015-12-31 2017-07-07 掌赢信息科技(上海)有限公司 One kind blink recognition methods and electronic equipment
CN107833197A (en) * 2017-10-31 2018-03-23 广东欧珀移动通信有限公司 Method, apparatus, computer-readable recording medium and the electronic equipment of image procossing
CN107911608A (en) * 2017-11-30 2018-04-13 西安科锐盛创新科技有限公司 The method of anti-shooting of closing one's eyes
CN108259766A (en) * 2018-03-29 2018-07-06 宁波大学 A kind of mobile intelligent terminal is taken pictures image pickup processing method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106934329A (en) * 2015-12-31 2017-07-07 掌赢信息科技(上海)有限公司 One kind blink recognition methods and electronic equipment
CN107833197A (en) * 2017-10-31 2018-03-23 广东欧珀移动通信有限公司 Method, apparatus, computer-readable recording medium and the electronic equipment of image procossing
CN107911608A (en) * 2017-11-30 2018-04-13 西安科锐盛创新科技有限公司 The method of anti-shooting of closing one's eyes
CN108259766A (en) * 2018-03-29 2018-07-06 宁波大学 A kind of mobile intelligent terminal is taken pictures image pickup processing method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
胡德文等: "《生物特征识别技术与方法》", 31 December 2013 *

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110348331A (en) * 2019-06-24 2019-10-18 深圳和而泰家居在线网络科技有限公司 Face identification method and electronic equipment
CN110348331B (en) * 2019-06-24 2022-01-14 深圳数联天下智能科技有限公司 Face recognition method and electronic equipment
CN111340688A (en) * 2020-02-24 2020-06-26 网易(杭州)网络有限公司 Method and device for generating eye closing image
CN111340688B (en) * 2020-02-24 2023-08-11 网易(杭州)网络有限公司 Method and device for generating closed-eye image
CN112036311A (en) * 2020-08-31 2020-12-04 北京字节跳动网络技术有限公司 Image processing method and device based on eye state detection and storage medium
US11842569B2 (en) 2020-08-31 2023-12-12 Beijing Bytedance Network Technology Co., Ltd. Eye state detection-based image processing method and apparatus, and storage medium

Similar Documents

Publication Publication Date Title
WO2019232866A1 (en) Human eye model training method, human eye recognition method, apparatus, device and medium
CN109376624A (en) A kind of modification method and device of eye closing photo
US20190228276A1 (en) License plate reader using optical character recognition on plural detected regions
KR101159830B1 (en) Red eye false positive filtering using face location and orientation
US8620036B2 (en) System and method for controlling image quality
CN110705357A (en) Face recognition method and face recognition device
CN109271875B (en) A kind of fatigue detection method based on supercilium and eye key point information
CN104573617B (en) A kind of camera shooting control method
CN107438854A (en) The system and method that the image captured using mobile device performs the user authentication based on fingerprint
CN108304819B (en) Gesture recognition system and method, and storage medium
JP2015512190A (en) Method and apparatus for operator absent image capture
CN104951773A (en) Real-time face recognizing and monitoring system
CN109598229A (en) Monitoring system and its method based on action recognition
CN109583348A (en) A kind of face identification method, device, equipment and system
CN105022999A (en) Man code company real-time acquisition system
US11062126B1 (en) Human face detection method
CN108764278A (en) A kind of the self study industrial intelligent detecting system and method for view-based access control model
CN107483813A (en) A kind of method, apparatus and storage device that recorded broadcast is tracked according to gesture
CN109241907A (en) Mask method, device and electronic equipment
CN111881740A (en) Face recognition method, face recognition device, electronic equipment and medium
CN109298783A (en) Mark monitoring method, device and electronic equipment based on Expression Recognition
CN104156689B (en) Method and device for positioning feature information of target object
Tin Robust Algorithm for face detection in color images
CN110210401B (en) Intelligent target detection method under weak light
Sithara et al. A survey on face recognition technique

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20190222

RJ01 Rejection of invention patent application after publication