JP4349019B2 - Image correction device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a technique for correcting a captured image, particularly an image having a person as a subject.
[0002]
[Prior art]
In general, it is said that both men and women increase the impression that the larger pupil size gives them to the other person and the sense of familiarity. For example, a baby gives a cute impression to the other person because the proportion of the eyes to the face size is large, and as a result, the size of the pupil feels large. In addition, as a visual effect, it is said that the face appears to be a small face when the pupil is large.
[0003]
On the other hand, as a conventional technique, there is a technique for simulating makeup by performing image processing on a face image of a subject person (see Patent Document 1). Patent Document 1 describes a technique for detecting human face parts (eyes, pupils, eyelids, eyebrows, mouth, nose, cheeks, skin, hair, etc.) and changing the shape, size, and color of these parts. Has been.
[0004]
Further, as conventional techniques, a technique for detecting the size of a pupil of a subject from an image (see Patent Document 2), a technique for detecting the position of a pupil of a subject from an image (see Patent Document 3 and Non-Patent Document 1), etc. There is.
[0005]
[Patent Document 1]
JP 2000-151985
[Patent Document 2]
JP 09-028672 A
[Patent Document 3]
JP 2000-339476 A
[Non-Patent Document 1]
Mayumi Yuasa, Kazuhiro Fukui, Osamu Yamaguchi, “Highly accurate pupil detection based on minimizing the integrated energy of patterns and edges”, IEICE Technical Report, 2000, Vol. 100, no. 134, p. 79-84
[0006]
[Problems to be solved by the invention]
However, neither of Patent Documents 1 and 2 describes a specific process for enlarging and outputting the pupil size. For example, even if the pupil and the size of the pupil are accurately detected by the technique described in Patent Document 2, the output image becomes unnatural simply by enlarging the size.
[0007]
In this way, no specific technique has been proposed so far, focusing on image correction for increasing the size of the pupil. For this reason, simply applying Patent Documents 1 and 2 cannot enlarge the size of the pupil and obtain a natural face image.
[0008]
In the present invention, an image correction apparatus or program that solves such problems, naturally expands the size of the pupil included in the face image, and generates a face image that gives a good impression to the viewer of the image is provided. The purpose is to provide.
[0009]
[Means for Solving the Problems]
In order to solve the above problems, the present invention has the following configuration. A first aspect of the present invention is an image correction apparatus that includes a pupil specifying unit, a contour specifying unit, an enlarging unit, a deleting unit, and a combining unit.
[0010]
The pupil specifying means specifies a pupil region of a person who is a subject in the image.
[0011]
The contour specifying means specifies the contours of the eyes of the person who is the subject in the image. The outline of the eye indicates the outline of the area surrounded by the upper eyelid and the lower eyelid. In other words, the eye outline indicates the boundary between the eyelid covering the eyeball and the eyeball surface not covered by the eyelid.
[0012]
The enlarging means enlarges the image of the pupil area specified by the pupil specifying means.
[0013]
The deletion unit deletes a portion that protrudes from the contour specified by the contour specifying unit in the image of the pupil region enlarged by the expansion unit.
[0014]
The synthesizing unit obtains an image in which the pupil is enlarged, using the image of the pupil region in which the portion protruding from the outline is deleted by the deleting unit. For example, when a part of the image including the pupil region is extracted from the original image and becomes a processing target of the enlargement unit or the deletion unit, the synthesis unit combines the image output from the deletion unit with the original position of the original image. .
[0015]
In the first aspect of the present invention, the image of the pupil region specified by the pupil specifying means is enlarged by a predetermined procedure by the enlargement means. In this state, the image of the pupil region may exceed the contour of the eye surrounded by the eyelid of the subject. Therefore, the contour of the eye is specified by the contour means, and the eye contour of the enlarged pupil region image is identified. The part that protrudes is deleted. And the image after correction | amendment is produced | generated using the image of this pupil area | region.
[0016]
For this reason, according to the first aspect of the present invention, the user can automatically operate without manually operating a device such as an image editing device (including an information processing device in which image editing software is installed). Therefore, it is possible to acquire an image that gives a good impression to the viewer with a naturally enlarged pupil.
[0017]
Further, the contour specifying means in the first aspect of the present invention extracts the eye region image that is an image including the human eye region, and specifies the eye contour by masking the skin color region from the eye region image. It may be configured to.
[0018]
The enlargement means in the first aspect of the present invention includes an enlargement area specifying means for specifying an enlargement area based on the pupil area, and an enlargement conversion for enlarging and converting the entire image of the pupil area at a predetermined magnification based on the enlargement area. Means may be included.
[0019]
A second aspect of the present invention is an image correction apparatus that includes a pupil specifying unit, a contour extracting unit, a contour operating unit, and a pixel value providing unit.
[0020]
The pupil specifying means specifies a pupil region of a person who is a subject in the image. The contour specifying means specifies the contours of the eyes of the person who is the subject in the image. The contour extracting means extracts the contour of the pupil region. The outline of the pupil region corresponds to the boundary between the black eye region and the white eye region. The contour operating means expands the pupil area by manipulating the contour of the pupil area extracted by the contour extracting means. The pixel value giving means gives a pixel value indicating the pupil to each pixel in the pupil area expanded by the contour operation means. The pixel value indicating the pupil is a pixel value that is the same as or similar to the pixel value in the human pupil region.
[0021]
In the second aspect of the present invention configured as described above, when the pixel area is limited to each pixel in the difference area between the expanded pupil area and the unextended pupil area, a new pixel value is given. The original pupil region image is not manipulated. For this reason, it is possible to obtain an image in which the pupil is naturally enlarged without destroying the video image or the like reflected in the pupil in the original image.
[0022]
Further, the contour operating means in the second aspect of the present invention separates the contours from each other when a plurality of contours located at the boundary with the white-eye region among the contours of the pupil region extracted by the contour extracting means are obtained. The outline of the pupil region may be widened by moving in the direction of movement.
[0023]
In addition, the contour operation means in the second aspect of the present invention includes eye center acquisition means for acquiring two points located at substantially the center of each of the left and right eyes of a person who is a subject in an image, and a straight line including the two points. Movement axis specifying means for specifying as a movement axis, and the contour may be moved according to the movement axis.
[0024]
Further, the contour operating means in the second aspect of the present invention may be configured to move each contour parallel to the movement axis.
[0025]
A third aspect of the present invention is an image correction apparatus, which includes pupil specifying means, approximation means, figure enlargement means, and pixel value provision means. The pupil specifying means specifies a pupil region of a person who is a subject in the image. The approximating means approximates the pupil area specified by the pupil specifying means to a geometric figure. The figure enlarging means enlarges the approximated pupil region by enlarging the geometric figure obtained by the approximating means. The pixel value giving means gives a pixel value indicating the pupil to each pixel in the pupil area expanded by the figure enlarging means. The pixel value indicating the pupil is a pixel value that is the same as or similar to the pixel value in the human pupil region.
[0026]
In the third aspect of the present invention configured as described above, the image of the pupil region is enlarged based on the geometric figure. For this reason, the enlarged pupil region is obtained as a well-defined contour, and an image that gives a better impression to the viewer can be obtained.
[0027]
Further, in the third aspect of the present invention configured as described above, the pixel value is set so as to give a new pixel value limited to each pixel in the difference region between the expanded pupil region and the pupil region before being expanded. When the providing means is configured, the original pupil region image is not operated. For this reason, the image that is reflected in the pupil in the original image is not destroyed, and an image in which the pupil is naturally enlarged can be obtained.
[0028]
Further, the approximation means in the third aspect of the present invention includes an axis acquisition means for acquiring an axis, and a circle center acquisition for acquiring a midpoint position of a portion having the largest axial width as a center of the circle in the pupil region. It may be configured to include means and radius acquisition means for acquiring a half value of the axial width as the radius of the circle when the center of the circle is acquired.
[0029]
The graphic enlarging means in the third aspect of the present invention may be configured to draw a circle using the center and a radius longer than the acquired radius.
[0030]
Further, the graphic enlarging means in the third aspect of the present invention may be configured to draw a trajectory using the center and the radius obtained by extending the acquired radius according to the angle from the axis.
[0031]
Further, the pixel value providing means in the second and third aspects of the present invention may be configured to sample a pixel value from the pupil region and to give the sampled pixel value as a pixel value indicating the pupil. good.
[0032]
Further, the first aspect, the second aspect, and the third aspect of the present invention may be configured to further include a blurring unit that performs a blurring process on a substantially boundary of the enlarged pupil region.
[0033]
According to a fourth aspect of the present invention, a step of specifying a pupil region of a person who is a subject in an image, a step of specifying an outline of a human eye that is a subject in the image, and an image of the specified pupil region Using the enlargement step to enlarge, the step of deleting the portion that protrudes from the specified contour in the image of the pupil region enlarged in the enlargement step, and the image of the pupil region from which the portion that protrudes from the contour is deleted, Is a program that causes the information processing apparatus to execute the step of acquiring an enlarged image.
[0034]
Further, the enlargement step in the fourth aspect of the present invention includes a step of specifying the enlargement region based on the pupil region, and a step of enlarging and converting the entire image of the pupil region based on the enlargement region at a predetermined magnification. It may be configured.
[0035]
Further, the fifth aspect of the present invention is a step of specifying a pupil region of a person who is a subject in the image, a step of specifying a contour of a human eye which is a subject in the image, and extracting a contour of the pupil region A step of expanding the pupil region by manipulating the contour of the pupil region, a step of providing a pixel value indicating the pupil for each pixel of the expanded pupil region, and a step of A program for causing an information processing apparatus to execute an image using an image and obtaining an image with an enlarged pupil.
[0036]
Further, the enlargement step in the fifth aspect of the present invention includes a step of approximating the pupil region to a geometric figure, a step of widening the approximated pupil region by enlarging the obtained geometric figure, and Providing a pixel value indicating the pupil for each pixel in the pupil region.
[0037]
DETAILED DESCRIPTION OF THE INVENTION
[[First Embodiment]]
Next, the image correction apparatus 1a which is the first embodiment of the image correction apparatus will be described with reference to the drawings. In the following description, the person image is an image including at least a part or all of the face of the person including the pupil. Therefore, the person image may include an image of the entire person, or may include an image of only the face of the person or only the upper body. Moreover, you may include the image about a several person. Furthermore, the background may include any pattern such as a scenery (background: including objects that have attracted attention as a subject) or a pattern other than a person.
[0038]
In addition, the following description about the image correction apparatus 1a is an illustration, The structure is not limited to the following description.
[0039]
〔System configuration〕
The image correction apparatus 1a includes a CPU (Central Processing Unit), a main storage device (RAM), an auxiliary storage device, and the like connected via a bus in terms of hardware. The auxiliary storage device is configured using a nonvolatile storage device. The nonvolatile storage device referred to here is a so-called ROM (Read-Only Memory: including EEPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), mask ROM, etc.), FRAM (Ferroelectric RAM). ), Hard disk, etc.
[0040]
FIG. 1 is a diagram illustrating functional blocks of the image correction apparatus 1a. The image correction apparatus 1a is loaded with various programs (OS, applications, etc.) stored in the auxiliary storage device and executed by the CPU, whereby the input unit 2, storage unit 3, area detection unit 4, It functions as a device including the correction processing unit 5a, the output unit 6, and the like. The area detection unit 4 and the correction processing unit 5a are realized by executing an image correction program by the CPU.
[0041]
FIG. 2 is a diagram showing a main process executed by each functional unit of the image correction apparatus 1a and its flow. Hereinafter, the functional units of the image correction apparatus 1a will be described with reference to FIGS.
[0042]
<Input section>
The input unit 2 functions as an interface for inputting data of the original image 7 of the human image (hereinafter referred to as “data of the original image 7”) to the image correction apparatus 1a. The input unit 2 inputs data of the original image 7 from the outside of the image correction apparatus 1a to the image correction apparatus 1a (S1). The input unit 2 may be configured using any existing technique for inputting data of the original image 7 to the image correction apparatus 1a.
[0043]
For example, person image data may be input to the image correction apparatus 1a via a network (for example, a local area network or the Internet). Further, human image data may be input to the image correction apparatus 1a from a digital camera, a personal computer, or the like. Further, human image data recorded on a recording device or a recording medium (for example, various flash memories, a floppy (registered trademark) disk, a CD (Compact Disk), or a DVD (Digital Versatile Disc, Digital Video Disc)) is stored in the image correction apparatus 1a. May be entered. The image correction device 1a is included in various devices (for example, a PDA (Personal Digital Assistant) or a mobile phone) including an imaging device such as a digital camera or an imaging device such as a digital camera. The image may be input to the image correction apparatus 1a before or after being stored. The image correction apparatus 1a may be included in an image output apparatus such as a printer or a display, and human image data input to the image output apparatus as output data may be input to the image correction apparatus 1a.
[0044]
The input unit 2 is configured according to each case described above. The input unit 2 may be configured to be able to respond to a plurality of cases described above.
[0045]
<Storage unit>
The storage unit 3 is configured using a so-called RAM. The storage unit 3 is used when each process is executed by the region detection unit 4 or the correction processing unit 5a. For example, data of the original image 7 to be processed, data of the eye area image 8 as intermediate generation data, data of the pupil enlarged image 9 as output data, and the like are read and written in the storage unit 3. Here, the eye region image 8 is an image included in the original image 7 and includes an image including the eyes of the subject person and the periphery thereof. The enlarged pupil image 9 is an image output from the image correction apparatus 1a, and refers to an image in which the pupil of the person in the original image 7 is enlarged by image correction. For this reason, the original image 7 and the enlarged pupil image 9 are basically the same image except for the pupil and its surroundings.
[0046]
<Area detection unit>
The area detection unit 4 executes area detection processing. Hereinafter, the area detection process will be described.
[0047]
In the region detection process, the data of the eye region image 8 on which the image processing is performed is acquired from the data of the original image 7. In addition, a pupil region to be subjected to processing such as enlargement is detected from the eye region image 8 in the correction processing unit 5a.
[0048]
FIG. 3 is a diagram for explaining the region detection processing. Next, a specific flow of the area detection process will be described.
[0049]
First, in the area detection process, an area including a person's face (hereinafter referred to as “face area”) is detected from the original image 7 (S2: see FIG. 2). Then, the coordinates of a rectangle including the detected face area (hereinafter referred to as “face rectangle”: corresponding to the inner rectangle in FIG. 3B) are obtained. The position and size of the face area of the person who is the subject in the original image 7 are specified by the coordinates of the face rectangle.
[0050]
The detection of the face area may be realized by applying any existing technique. For example, the coordinates of the face rectangle may be obtained by template matching using a reference template corresponding to the outline of the entire face. Further, the coordinates of the face rectangle may be obtained by template matching based on face components (eyes, nose, ears, etc.). Further, the vertex of the hair may be detected by chroma key processing, and the coordinates of the face rectangle may be obtained based on this vertex.
[0051]
In the area detection processing, next, an eye area image 8 is acquired from the face area image (S3). The acquisition of the eye area image 8 may be realized by applying any existing technique.
[0052]
For example, first, the center position of a person's eyes or pupils is detected (corresponding to the cross mark (x mark) in FIG. 3C), and the area of the eye region image 8 based on the detected position (FIG. 3D). Corresponding to the two innermost rectangles) is determined, and the eye region image 8 is acquired. At this time, any existing technique may be applied to the detection of the center position of the person's eyes or pupils.
[0053]
For example, the center position of a human eye or pupil is detected by the techniques described in Patent Document 3 and Non-Patent Document 1. Further, when the eye area image 8 is acquired based on the detected position, it may be acquired by any method. For example, an image surrounded by a specific rectangle centered on the detected position is acquired as the eye region image 8. At this time, the size of the specific rectangle is determined based on, for example, the face width of the person or the distance between the center positions of the left and right eyes.
[0054]
In the region detection process, next, a pupil region is detected from the acquired eye region image 8 (S4 in FIG. 2). The detection of the pupil region may be realized by applying any existing technique. An example is shown below.
[0055]
FIG. 4 is a flowchart showing a flow of processing when detection of a pupil region is performed. FIG. 5 is a diagram showing an eye region image 8 (FIG. 5A) before the pupil region detection is performed and an eye region image 8 (FIG. 5B) after the pupil region extraction is performed. It is.
[0056]
First, a skin color region is extracted from the eye region image 8 as a non-pupil region. Then, the extracted skin color region is masked (S4a). This process is performed by, for example, the Lab space method. By this masking, the skin color area is removed from the eye area image 8, and the outline of the eye including the white eye area and the pupil area is specified.
[0057]
Next, a white-eye region (that is, a white region) is extracted as a non-pupil region from the eye region image 8 subjected to the processing of S4a. Then, masking is performed on the extracted white-eye region (S4b). This process is performed by, for example, the RGB space method. By this masking, the white eye region is removed, and an image of the pupil region is acquired.
[0058]
In the eye area image 8 (corresponding to (b) of FIG. 5) obtained as a result of the execution of the processes of S4a and S4b, an image of the pupil area to be processed such as enlargement remains (S4c). In this way, the pupil region is extracted (S4).
[0059]
<Correction processor>
The correction processing unit 5a performs correction processing. Hereinafter, the correction process will be described. In the correction process, an image correction process for enlarging the pupil area detected by the area detection process is performed in the eye area, that is, in the area between the upper eyelid and the lower eyelid (S5: see FIG. 2).
[0060]
FIG. 6 is a flowchart showing a specific flow of the correction process. FIG. 7 is a functional block diagram illustrating a configuration example of the correction processing unit 5a. The correction processing unit 5 a includes a region specifying unit 10, an enlargement processing unit 11, a blur processing unit 12, and a region correction unit 13.
[0061]
First, in the eye area image 8, the detected pupil area is enlarged (S5a). Here, a specific flow of the pupil region enlargement process performed by the correction processing unit 5a will be described. First, among the detected pupil regions, a line that passes through the point having the maximum value or minimum value in the x-axis direction and is perpendicular to the x-axis, and a line that passes through the maximum value or minimum value in the y-axis direction and is perpendicular to the y-axis. The enclosed rectangle is obtained. The acquisition of the rectangle is executed by the area specifying unit 10. Then, using the center of the rectangle as a reference point, the image of the pupil region is enlarged according to a specific magnification (eg, 110%, 120%). This enlargement is executed by the enlargement processing unit 11.
[0062]
The specific magnification may be set in advance by the user or may be determined dynamically. The specific magnification can be dynamically determined by, for example, a method of determining the size of the eye area and the pupil area (for example, a ratio of the number of pixels) or a ratio of the lateral width of the eye area and the pupil area. There are ways to do it.
[0063]
Processing for realizing enlargement of the pupil region based on another method will be described in the columns of the second embodiment and the third embodiment.
[0064]
Next, the blurring process is performed on the boundary portion of the pupil area enlarged by the process of S5a. This process is executed by the blurring processing unit 12. This boundary portion is obtained in accordance with a certain standard from the area of the enlarged pupil region and the values of the vertical and horizontal widths. For example, the enlarged pupil region is further reduced at a specific magnification (eg, 98%), and a difference portion between the enlarged pupil region and the reduced pupil region is regarded as a boundary portion. In order to omit the reduction process, the boundary portion may be obtained by using the pupil region before enlargement as the reduced pupil region. Further, the blur processing may not be executed when the area of the pupil region, the vertical width, the horizontal width, or the like is smaller than a preset threshold value.
[0065]
In addition, in the blurring process for the boundary portion, any existing correction method is applied as long as the correction is performed to eliminate the unnaturalness of the boundary between the enlarged pupil region and the white eye portion. Also good. For example, the blurring process may be performed by linearly changing the color from the pixel value of the pupil region to the pixel value of the white eye region from the inside to the outside of the boundary portion. Alternatively, a blurring process using a moving average filter or the like may be performed on the boundary portion.
[0066]
Next, the portion of the pupil region image that has been subjected to image processing by the processing of S5b is removed from the upper eyelid and the lower eyelid (S5c). This process is executed by the area correction unit 13. This process is realized, for example, by using a skin color area mask created in the area detection process.
[0067]
Then, the image of the pupil region from which the portion protruding from the process of S5c has been deleted is synthesized at the corresponding position of the original image 7, and the pupil enlarged image 9 is generated (S5d).
[0068]
<Output section>
The output unit 6 functions as an interface for outputting the data of the enlarged pupil image 9 created by the correction process to the outside. The method of using the data output by the output unit 6 depends on the device that acquires this data via the output unit 6. For example, when the image correction apparatus 1a includes a display or is connected to a display, data output by the output unit 6 is performed on the display, and the pupil enlarged image 9 is displayed. For example, when the image correction apparatus 1a is included in a digital camera or various apparatuses including a digital camera, the digital camera is configured to store the data of the enlarged pupil image 9 output from the output unit 6. Further, for example, when the image correction apparatus 1a is included in another image correction apparatus, the other image correction apparatus performs a correction process on the data of the pupil enlarged image 9 or this pupil enlargement. It is configured to record or output the data of the image 9 as a product.
[0069]
[Action / Effect]
In the image correction apparatus 1a, the pupil region is detected from the original image 7 by executing the region detection process. Next, the image of the detected pupil region is enlarged, and the image of the portion that protrudes from the upper eyelid and the lower eyelid (that is, the eye region) is deleted from the enlarged pupil region image. Then, the image of the eye region from which the protruding portion is deleted is combined with the corresponding portion of the original image 7 to generate the enlarged pupil image 9.
[0070]
FIG. 8 is a diagram illustrating an example of an image in which a pupil region is enlarged. FIG. 9 is a diagram illustrating an example of an image before the pupil is enlarged (example of input image: (a)) and an example of an image after the pupil is enlarged (example of output image: (b)). It is.
[0071]
FIG. 8A is a diagram illustrating an example of an image when the blurring process (S5b) and the deletion of the protruding pupil area (S5c) are not performed on the enlarged pupil area. As described above, when the extracted image of the pupil region is simply enlarged, the upper end side and the lower end side of the pupil region particularly protrude beyond the eyelids. For this reason, the enlarged image becomes an unnatural image. On the other hand, FIG. 8B is a diagram illustrating an example of an image when the blurring process (S5b) and the removal of the protruding pupil region (S5c) are performed on the enlarged pupil region. That is, FIG. 8B is an enlarged pupil image 9 generated by the image correction apparatus 1a. According to the image correction apparatus 1a, as shown in FIG. 8B, the image of the pupil region is enlarged without protruding from the upper and lower eyelids. For this reason, the pupil region image generated in this way is used in the processing of S5d, so that the pupil is naturally enlarged as compared with the image (FIG. 9A) input to the image correction apparatus 1a. Thus, it is possible to automatically acquire a person image (FIG. 9B) that gives a good impression to the viewer.
[0072]
Therefore, it is possible to respond to a user's desire to obtain a photograph (FIG. 9B) that gives a good impression to others without performing manual processing using image editing software or the like that requires skilled skills. Become.
[0073]
In addition, recently, camera-equipped mobile terminals (eg, mobile phones, PDAs) have become widespread. When a human image is captured by such a device, the captured image is obtained by transmission processing to another device using a communication function provided in advance in the device, or display on a display unit provided in the device. There are many opportunities to see others directly. In other words, the image captured by such an apparatus has few opportunities to perform image editing using a personal computer, a photo retouch dedicated apparatus, or the like before being touched by another person's eyes. In addition, since the size of the display unit and the input interface provided in such a portable terminal are very limited, it is very difficult to perform image processing on the portable terminal device by human hands. This is a troublesome process for the user.
[0074]
On the other hand, since the image correction apparatus 1a is mounted on such a portable terminal with a camera, the pupil of the captured human image is naturally enlarged by the above-described processing, so that the user can easily make an impression on others. An image can be obtained.
[0075]
In the image correction apparatus 1a, the blurring process is performed on the boundary portion of the pupil region, that is, the boundary portion between the pupil region and the white eye region. For this reason, it becomes possible to acquire the natural enlarged pupil image 9 that does not give the user a sense of incongruity such as the appearance of the pupil being raised.
[0076]
[Modification]
In the above description, when the eye area image 8 is acquired in the area detection process, the eye area image 8 is temporarily stored in the storage unit 3, and the process related to pupil enlargement is performed on the eye area image 8. It was. Then, the eye region image 8 is synthesized with the corresponding part of the original image 7, thereby obtaining the pupil enlarged image 9. However, the processing related to pupil enlargement may be performed directly on the original image 7 without acquiring the eye region image 8.
[0077]
In addition, in the process of S4a, extraction and masking may be performed on the red region in consideration of the white eye being full of blood. Such masking processing may be performed by any method such as RGB space method or Lab space method.
[0078]
Further, the order of execution of the process of S5b and the process of S5c may be reversed. In other words, the blur processing may be executed after the area that protrudes from the eyelid in the pupil area is deleted.
[0079]
Further, the process of S5b, that is, the blurring process is not necessarily executed, and it may be configured to select whether or not to be executed by the user or the designer. Similarly, it may be configured not to execute the blurring process.
[0080]
[[Second Embodiment]]
〔System configuration〕
Next, differences between the image correction apparatus 1b, which is the second embodiment of the image correction apparatus, and the image correction apparatus 1a will be described with reference to the drawings.
[0081]
The image correction apparatus 1b differs from the image correction apparatus 1a in that a correction processing unit 5b is provided instead of the correction processing unit 5a. The correction processing unit 5b differs from the correction processing unit 5a in the content of the process of enlarging the pupil region (see S5a in FIG. 6).
[0082]
FIG. 10 is a flowchart showing the correction processing by the correction processing unit 5b of the image correction apparatus 1b. FIG. 11 is a functional block diagram illustrating a configuration example of the correction processing unit 5b. The correction processing unit 5 b includes a blur processing unit 12, a region correction unit 13, a movement axis acquisition unit 14, a boundary line extraction unit 15, a boundary line processing unit 16, and a color determination unit 17. Hereinafter, the flow of the correction process in the correction processing unit 5b of the image correction apparatus 1b will be described with reference to FIGS.
[0083]
First, the movement axis 18 is acquired (S5a-1a). This process is executed by the movement axis acquisition unit 14. FIG. 12 is a diagram illustrating an example of the moving shaft 18. A method of using the moving shaft 18 will be described later. The moving axis 18 is obtained as a straight line including the center positions of the left and right pupils (corresponding to the cross mark (x mark) in FIG. 12) detected in the area detection process.
[0084]
Next, a boundary line between the pupil region and the white eye is extracted (S5a-2a). This process is executed by the boundary line extraction unit 15. FIG. 13 is a diagram illustrating a boundary line between a pupil region and a white eye. In FIG. 13, the boundary line between the pupil region and the white eye is indicated by a bold line. Such a boundary line is extracted by, for example, a general edge extraction process.
[0085]
Next, the extracted boundary line is translated according to the acquired movement axis 18 (S5a-3a). This process is executed by the boundary line processing unit 16. FIG. 14 is an image showing how the extracted boundary line is translated along the movement axis 18. As shown in FIG. 14, the extracted boundary line is translated in a direction away from the center position of the pupil. The moving distance may be set as appropriate. The moving distance is determined based on values such as the width of the face detected in the area detection process and the width of the detected pupil area, for example.
[0086]
Next, a color is given to each pixel in the pupil region (S5a-4a). This process is executed by the color determination unit 17. In the process by the boundary line processing unit 16 of the correction processing unit 5b (S5a-3a in FIG. 10), the boundary line is only moved as the pupil enlargement process. No pixel value is given. For this reason, a pixel value is given to such a difference area by the process of S5a-4a. Further, not only the difference area but also a new pixel value may be given to all pixels in the enlarged pupil area.
[0087]
Any pixel value may be given to the difference area. A pixel value may be given to the above-described difference region using a pixel value sampled from the pupil region before enlargement. Specifically, for example, the pixel value of the pixel located on the outermost side of the pupil region before enlargement may be given to all the difference regions located on the movement axis 18 from this pixel. Further, for example, a plurality of pixel values may be sampled from the pupil region before enlargement, and pixel values may be selected at random for each difference region from the sampled pixel values.
[0088]
In the correction processing by the correction processing unit 5b, the processing after S5b (see FIG. 6) is performed in the same manner as the correction processing unit 5a after the processing of S5a-4a.
[0089]
[Action / Effect]
According to the image correction apparatus 1b, the pupil region can be enlarged without enlarging the image of the pupil region itself. For this reason, it is possible to obtain the enlarged pupil image 9 without deforming the image reflected in the pupil by newly providing a pixel value only to the difference region. Therefore, it is possible to obtain an image in which the pupil is enlarged more naturally.
[0090]
[Modification]
In the image correction apparatus 1b, when the pupil region before enlargement does not cross the lower eyelid or the upper eyelid, when the boundary line is translated according to the movement axis 18, the pupil region is not a closed region. That is, a part of the boundary line forming the pupil region is opened, and the pupil region cannot be specified. In such a case, the method shown in the first embodiment or the third embodiment (magnification (scaling) processing with a predetermined reference as the origin) is adopted so that the enlarged pupil image 9 is obtained. It may be configured.
[0091]
Further, even if the pupil region before enlargement intersects the lower eyelid or the upper eyelid, they may not intersect if translated, and the pupil region may not be a closed region. In such a case, the enlarged pupil image 9 may be obtained by adopting the method shown in the first embodiment or the third embodiment. Also, the extracted boundary line and the upper or lower boundary are approximated by a curve (eg, Bezier curve, circle arc), the intersection of the boundary lines is calculated mathematically, and a closed space is generated. May be.
[0092]
The second embodiment may be configured not to execute the process of S5c (see FIG. 6), that is, the process of deleting the pupil region that has protruded due to the enlargement process. In the second embodiment, since the boundary line is moved in parallel with the movement axis, the enlarged pupil region hardly protrudes from the eyelid. For this reason, particularly when the amount of movement of the boundary line is small, it is possible to obtain the pupil enlarged image 9 in which the pupil is naturally enlarged without performing such deletion processing.
[0093]
[[Third embodiment]]
〔System configuration〕
Next, the difference between the image correction apparatus 1c, which is the third embodiment of the image correction apparatus, and the image correction apparatus 1b will be described with reference to the drawings.
[0094]
The image correction apparatus 1c is different from the image correction apparatus 1b in that a correction processing unit 5c is provided instead of the correction processing unit 5b.
[0095]
FIG. 15 is a flowchart showing the correction processing by the correction processing unit 5c of the image correction apparatus 1c. FIG. 16 is a functional block diagram illustrating a configuration example of the correction processing unit 5c. The correction processing unit 5c differs from the correction processing unit 5b in the content of the process of enlarging the pupil region (S5a in FIG. 6). As a configuration, the correction processing unit 5 c is different from the correction processing unit 5 b in that it includes a center detection unit 19 and a circle enlargement unit 20. The correction processing unit 5 c includes a blur processing unit 12, a region correction unit 13, a movement axis acquisition unit 14, a color determination unit 17, a center detection unit 19, and a circle enlargement unit 20. Hereinafter, the flow of correction processing in the correction processing unit 5c of the image correction apparatus 1c will be described with reference to FIGS.
[0096]
First, the center point of the pupil is detected. This process is executed by the movement axis acquisition unit 14 and the center detection unit 19. In the area detection process, the center of the pupil is detected, but the center of the pupil is detected again.
[0097]
Next, a straight line (corresponding to the movement axis 18 in the second embodiment) including the center positions of the left and right pupils detected in the region detection process is acquired as the x-axis (S5a-1b in FIG. 15). Next, a portion in the pupil region where the width in the x-axis direction is maximum is detected. Then, the midpoint of the pupil region at this time in the x-axis direction is detected as the center of the pupil. In addition, a half length of the width at this time is acquired as a radius (S5a-2b).
[0098]
Next, the range of the pupil area is expanded based on the center position of the pupil. This process is executed by the circle enlargement unit 20. Specifically, a region such as a circle is defined using the center point and the radius obtained by the center detection unit 19, and this region is set as a new pupil region (S5a-3b).
[0099]
For example, the circle may be defined using the center position of the pupil and the radius increased by a predetermined magnification. Further, for example, a region similar to a circle may be defined by using the center position and radius of the pupil and multiplying the radius by a weight corresponding to the angle from the x-axis. FIG. 17 is a diagram illustrating the latter method among the methods for defining a new pupil region. The angle from the x axis with respect to the radius is represented by (θ + π / 2) in FIG. A curved line with a dotted line in FIG. 17 indicates a newly defined pupil region.
[0100]
Next, a color is given to each pixel in the pupil region (corresponding to the difference region in the processing S5a-4a by the correction processing unit 5b) (S5a-4b). In this process, a pixel value is given to each pixel in the difference area by the same process as the process S5a-4a by the correction processing unit 5b.
[0101]
In the correction processing by the correction processing unit 5c, after S5a-4b, the processing after S5b (see FIG. 6) is performed in the same manner as the correction processing unit 5b.
[0102]
[Action / Effect]
According to the image correction apparatus 1c, the corrected pupil region, that is, the pupil region in the enlarged pupil image 9, is corrected into a regular shape such as a circle. For this reason, it is possible to provide the pupil enlarged image 9 having a clean pupil region.
[0103]
[Modification]
The third embodiment may be configured not to execute the process of S5c (see FIG. 6), that is, the process of deleting the pupil region that has protruded due to the enlargement process. In the third embodiment, the pupil region is newly defined according to the approximated radius and center. For this reason, the enlarged pupil region hardly protrudes from the eyelids. Therefore, particularly when the amount of enlargement of the radius is small, it is possible to obtain the enlarged pupil image 9 in which the pupil is naturally enlarged without performing such deletion processing.
[0104]
【The invention's effect】
According to the present invention, it is possible to easily obtain an image in which the pupil is naturally enlarged without the user operating an image editing device (including an information processing device in which image editing software is installed). For this reason, when acquiring an image in which the pupil is naturally enlarged, the user is not required to have a skilled technique.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of an image correction apparatus.
FIG. 2 is a diagram illustrating main processing of each functional block of the image correction apparatus.
FIG. 3 is a diagram illustrating an eye area image acquisition process in an area detection process;
FIG. 4 is a flowchart showing a flow of pupil area detection processing;
FIG. 5 is a diagram illustrating processing for detecting a pupil region in an eye region image.
FIG. 6 is a flowchart showing a flow of correction processing.
FIG. 7 is a diagram showing functional blocks of a correction processing unit in the first embodiment.
FIG. 8 is a diagram illustrating an example of an image in which a pupil region is enlarged.
FIG. 9 is a diagram illustrating an example of an image before the pupil region is enlarged and an image after the pupil region is enlarged.
FIG. 10 is a flowchart showing pupil area enlargement processing in the second embodiment.
FIG. 11 is a diagram illustrating functional blocks of a correction processing unit in the second embodiment.
FIG. 12 is a diagram illustrating processing for acquiring a movement axis.
FIG. 13 is a diagram illustrating boundary line extraction processing;
FIG. 14 is a diagram illustrating boundary line movement processing;
FIG. 15 is a flowchart showing pupil area enlargement processing in the third embodiment.
FIG. 16 is a diagram showing functional blocks of a correction processing unit in the third embodiment.
FIG. 17 is a diagram illustrating pupil area enlargement processing in the third embodiment;
[Explanation of symbols]
1a, 1b, 1c Image correction device
2 Input section
3 storage unit
4 area detector
5a, 5b, 5c Correction processing unit
6 Output section
7 Original image
8 eye area image
9 Eye magnified image
10 Area specification part
11 Enlargement processing section
12 Blur processing part
13 Area correction unit
14 Moving axis acquisition unit
15 Boundary line extraction unit
16 Boundary line processing part
17 color decision part
18 Movement axis
19 Center detector
20 yen expansion

Claims (4)

A pupil identifying means for identifying a pupil region of a person who is a subject in an image;
Contour extracting means for extracting the contour of the pupil region;
Contour operating means for expanding the pupil region by operating the contour of the pupil region;
A pixel value giving means for giving a pixel value indicating a pupil to each pixel of the pupil region expanded by the contour operating means;
In the image correction apparatus comprising,
The contour operating means includes
Eye center acquisition means for acquiring two points located at approximately the center of each of the left and right eyes of a person who is a subject in the image, and movement axis specifying means for specifying a straight line including the two points as a movement axis,
When a plurality of contours located at the boundary between the pupil region and the white eye region are obtained by the contour extraction means, the contour of the pupil region is expanded by moving each contour in a direction away from each other according to the movement axis.
Image correction device . The image correction apparatus according to claim 1 , wherein the pixel value providing unit samples a pixel value from the pupil region, and provides the sampled pixel value as a pixel value indicating a pupil. The image correction apparatus according to claim 1 or 2 further comprising a blurring unit that performs blurring processing on substantially boundaries expanded pupil area. Identifying a human pupil region that is a subject in the image;
Extracting a contour of the pupil region;
Expanding the pupil region by manipulating the contour of the pupil region;
Providing a pixel value indicating a pupil for each pixel in the expanded pupil region;
Is a program that causes an information processing apparatus to execute
In the step of expanding the pupil region,
Obtain two points located at the approximate center of each of the left and right eyes of the person in the image,
Specify a straight line including the two points as a movement axis,
When a plurality of contours located at the boundary between the pupil region and the white eye region are obtained by the step of extracting the contour, the contour of the pupil region is expanded by moving each contour in a direction away from each other according to the movement axis.
Program .

Images