JP2010141399A - Imaging device, image processing method in the imaging device and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display an image after correction processing without making a user be bored until the completion of correction processing for a face in image data after image pickup. <P>SOLUTION: An organ detecting section 20 detects a face in a picked-up image. A corrected main image processing section 1811 executes predetermined correction processing for a face region in the picked-up image based on a result of face detection by the organ detecting section 20. An effect image processing section 1833 applies reduction processing to the picked-up image to generate a small image before processing, executes correction processing for the face region in the small image before processing based on a result of face detection by the organ detecting section 20 to generate a processed small image. An effect processing section 211 executes effect processing using the processed small image in parallel to the corrected main image processing executed by the corrected main image processing section 1811. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus, an image processing method and an image processing program in the imaging apparatus.

  Conventionally, a face detection technique for detecting a face of a person or the like shown in an image is known. For example, in an imaging apparatus such as a digital camera, the face of the subject is detected using this face detection technology, and the result of face detection is used for controlling exposure, focus, and the like.

  On the other hand, various image processing techniques for applying various correction processes to the face of a subject based on the result of face detection using a face detection technique have been proposed. For example, Patent Document 1 discloses an image processing method in which an eye image is registered in advance, and an eye region in a captured image is corrected (corrected) using the eye image.

JP 2007-157176 A

  By applying the image processing technology as described above to an imaging device and correcting a face in an image when shooting a person as a subject, the facial expression of the person can be made brighter or beautiful. It becomes possible. However, since these correction processes take a certain amount of time, the user has to wait until the correction process is completed after shooting, which is tedious. In addition, the image (finished) after the correction process could not be confirmed in advance.

  The present invention has been made in view of the above, and can present an image after correction processing without boring the user until the correction processing for the face in the image data is completed after shooting. An object is to provide an imaging device, an image processing method in the imaging device, and an image processing program.

  In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention reduces an image data generated by an imaging unit that photoelectrically converts a subject image and image data generated by the imaging unit. A small image generation unit that generates small image data, a face detection unit that detects a face in the image data and / or small image data generated by the small image generation unit, and a face detection result by the face detection unit Based on the results of face detection by the main image processing unit with correction for generating corrected image data obtained by performing predetermined correction processing on a face area in the image data, and the face detection unit In addition to the processing by the small image processing unit that generates the processed small image data obtained by performing the correction processing on the face area in the small image data, and the processing by the main image processing unit with correction, the processing small image Use data A presentation processing unit which performs effect processing was, characterized in that it comprises a.

  In the imaging device according to the present invention, in the above invention, the small image processing unit performs a plurality of step-processed small image data by performing the correction process stepwise on a face area in the small image data. And the effect processing unit performs a process of displaying the plurality of step-processed small image data as a moving image as the effect process.

  Further, in the imaging apparatus according to the present invention, in the above invention, the main image processing unit with correction performs a plurality of types of correction processing on a face area in the image data, and the small image processing unit includes: The plurality of types of correction processing are sequentially performed on a face area in the small image data to generate the plurality of step-processed small image data.

  Moreover, in the imaging device according to the present invention, in the above invention, the effect processing unit is configured to perform, as the effect processing, a predetermined interval between at least one of the plurality of step-processed small image data. The present invention is characterized in that processing for displaying a moving image by inserting background image data is performed.

  In the imaging device according to the present invention, in the above invention, the small image processing unit includes a process of changing a size of the processed small image data, a process of cutting out a part of the processed small image data, and the processed small image. The effect processing unit changes the form of the processed small image data by performing at least one of a process of rotating data with a different rotation amount and a process of changing the outer shape of the processed small image data. As a process, a process for displaying the processed small image data whose form has been changed is performed.

  In the imaging device according to the present invention, in the above invention, the small image processing unit synthesizes a predetermined template at a position near a face area in the processed small image data, and the effect processing unit As the effect process, a process of displaying the small image data synthesized with the predetermined template is performed.

  Moreover, the imaging apparatus according to the present invention is characterized in that, in the above-described invention, the image capturing apparatus includes a recording unit that records the corrected image data and the processed small image data in association with each other.

  In the imaging device according to the present invention, in the above invention, the face detection unit detects a face in the small image data, and the main image processing unit with correction includes the face detection unit in the small image data. A corresponding position in the image data is specified as a face area based on the face detection result performed on the image data, and the correction process is performed on the specified face area.

  An image processing method in the imaging apparatus according to the present invention is an image processing method in an imaging apparatus including an imaging unit that photoelectrically converts a subject image to generate image data, and the image data generated by the imaging unit Image generating step for generating small image data obtained by reducing the image, a face detecting step for detecting a face in the image data and / or the small image data generated in the small image generating step, and the face detecting step The corrected image processing step for generating corrected image data obtained by performing a predetermined correction process on the face area in the image data based on the face detection result in the step, and the face in the face detection step In parallel with the processing of the small image processing step of generating the processed small image data obtained by performing the correction processing on the face area in the small image data based on the detection result, and the main image processing step with correction. , Characterized in that it comprises a and a presentation processing step of performing an effect process using the processing small image data.

  Further, the image processing program according to the present invention is a small image data obtained by reducing the image data generated by the imaging unit in a computer of an imaging apparatus including an imaging unit that photoelectrically converts a subject image to generate image data. A small image generation step for generating a face, a face detection step for detecting a face in the image data and / or the small image data generated in the small image generation step, and a face detection result in the face detection step And a main image processing step with correction for generating image data with correction obtained by performing a predetermined correction process on a face area in the image data, and a face detection result in the face detection step, A small image processing step for generating processed small image data obtained by performing the correction processing on the face area in the small image data, and a processing of the corrected main image processing step. In parallel with, characterized in that to execute the effect processing step of performing an effect process using the processing small image data.

  According to the present invention, the image data generated by the imaging unit is reduced to generate small image data, and correction processing is performed on the face area in the small image data to generate processed small image data. can do. In parallel with the process of performing the correction process on the image data generated by the imaging unit and generating the corrected image data, the effect process using the processed small image data can be performed. Therefore, the image after the correction processing can be presented without boring the user until the correction processing for the face in the captured image is completed after the shooting.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, a case where the imaging apparatus of the present invention is applied to a digital camera will be described as an example. Note that the present invention is not limited to the embodiments. Moreover, in description of each drawing, the same code | symbol is attached | subjected and shown to the same part.

(Embodiment)
FIG. 1 is a rear view of the digital camera 1. As shown in FIG. 1, the digital camera 1 includes a release switch (shutter switch) 3 for instructing photographing timing provided on the upper surface of the camera body 2, and a power switch 4 provided on the back surface of the camera body 2. And a menu switch 5, a cross key 6 having up / down / left / right direction switches (upper switch, lower switch, left switch and right switch), an OK switch 7 for confirming operation contents, a shooting mode and a playback mode. A playback switch 8 for switching, a display unit 23 for displaying various screens, and the like are provided. The power switch 4 is used for switching and inputting power on / off of the digital camera 1 and is realized by using, for example, a slide switch. The release switch 3 is, for example, a two-stage press button. When the release switch 3 is half-pressed, the first release switch is turned ON, and when the release switch 3 is fully pressed, the second release switch is turned ON. Although not shown, a flash, an imaging lens, and the like are disposed on the front surface of the camera body 2.

  When the power switch 4 is slid in the digital camera 1 to give an instruction to turn on the power, the digital camera 1 enters a state where it can shoot (shooting mode). In the shooting mode, a subject image incident through the imaging lens is output every frame (for example, 1/30 second) and displayed in real time on the display unit 23 as a live view image. While viewing the image, the release switch 3 is pressed to shoot a still image or a moving image. On the other hand, when the power is turned on, the shooting mode and the playback mode are switched according to the pressing operation of the playback switch 8. In this playback mode, the user enjoys displaying (playing back) still images and moving images captured by the digital camera 1 on the display unit 23.

  FIG. 2 is a block diagram showing a system configuration of the digital camera 1. As shown in FIG. 2, the digital camera 1 includes an imaging device 11, a lens system unit 12, a lens driving circuit 13, an imaging circuit 14, an SDRAM (Synchronous Dynamic Random Access Memory) 15, an AE unit 16, AF unit 17, image processing unit 18, organ detection unit 20 as a face detection unit, CPU 21, built-in memory 22, display unit 23, display drive circuit 24, communication I / F 25, and operation unit 26 A detachable memory 27, a power supply circuit 28, and a battery 29. The imaging circuit 14, SDRAM 15, AE unit 16, AF unit 17, image processing unit 18, organ detection unit 20, CPU 21, display drive circuit 24, and removable memory 27 are connected via a bus 30. Further, the image sensor 11, the lens system unit 12, the lens driving circuit 13, and the imaging circuit 14 constitute an imaging unit 10 that photoelectrically converts a subject image to generate image data of a captured image.

  The image pickup device 11 is an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and photoelectrically converts a subject image incident through the lens system unit 12 in units of frames and outputs an analog electric signal. To do. The lens system unit 12 includes an imaging lens including an AF (Auto-Focus) lens, a zoom lens, and the like, a diaphragm, a shutter, and the like. The lens driving circuit 13 drives the lens system unit 12.

  The imaging circuit 14 performs analog signal processing such as CDS (Correlated Double Sampling) and AGC (Automatic Gain Control) on the analog electrical signal output from the imaging device 11 and converts the analog electrical signal into a digital electrical signal. Digital signal processing such as pixel interpolation processing and color correction processing is performed on the digital electrical signal and output as image data. This image data is temporarily recorded in the SDRAM 15.

  The SDRAM 15 is used for temporary recording of image data output from the imaging circuit 14 and image data being processed by the image processing unit 18. For example, image data of an image (live view image) output from the imaging circuit 14 for each frame, image data of an image output from the imaging circuit 14 at the shooting timing (hereinafter referred to as “captured image” as appropriate), and the like. Is temporarily recorded. In the shooting mode, which is one of the modes of the digital camera 1, the subject image formed on the image sensor 11 is displayed on the display unit 23 in a moving image in real time. What is a live view image? This is the image.

  The AE unit 16 performs automatic exposure based on the image data output from the imaging circuit 14. The AF unit 17 performs automatic focus adjustment based on the image data output from the imaging circuit 14.

  The image processing unit 18 performs various types of image processing on the image data output from the imaging circuit 14 and performs processing for converting the image data into image data suitable for recording and display. For example, when recording image data of a photographed image or displaying recorded image data, image data compression processing or decompression processing based on a JPEG (Joint Photographic Experts Group) method or the like is performed. In addition, resizing processing to increase / decrease image data by increasing / decreasing the number of pixels, trimming processing to cut out a part from the image data, brightness adjustment processing to adjust the luminance component and color component of each pixel, color adjustment processing, transparency processing, etc. Perform various image processing on data as needed. For example, as a small image generation unit, the image processing unit 18 reduces the generated captured image to generate small image (thumbnail) image data (small image data).

  The image processing unit 18 includes a main image processing unit 181 with correction and an effect image processing unit 183 as a small image processing unit. The main image processing unit 181 with correction is performed when the correction mode for correcting the face in the captured image is selected as the setting relating to the imaging mode, and when the organ detection unit 20 detects the face in the captured image. Then, image processing including correction processing for processing the face area (main image processing with correction) is performed on the captured image. The effect image processing unit 183 generates a small effect image to be displayed on the display unit 23 until the corrected main image processing unit 181 finishes the corrected main image processing.

  The organ detection unit 20 detects, for example, a face area (face area) in the image data by applying pattern matching which is a well-known technique based on the image data of the small image, and the detection result of the face area Based on the above, it detects each face part such as left and right eyes, nose and lips. Data such as the position coordinates of the face area in the detected human image and the position coordinates of each part are recorded in the SDRAM 15.

  The CPU 21 reads out and executes a camera program from the built-in memory 22 in response to an operation signal from the operation unit 26, etc., and gives instructions to each unit constituting the digital camera 1, data transfer, and the like to control the operation of the digital camera 1. Control. The CPU 21 includes an effect processing unit 211. The effect processing unit 211 is generated by the effect image processing unit 183 in parallel with the main image processing with correction until the main image processing unit with correction 181 finishes the main image processing with correction after shooting. An effect process of displaying a plurality of effect small images on the display unit 23 is performed.

  The built-in memory 22 is an electrically rewritable nonvolatile memory such as a flash memory. For example, the built-in memory 22 operates the digital camera 1 and realizes various functions provided in the digital camera 1. The various camera programs and data used during the execution of the camera program are recorded in advance. The recording medium for recording the camera program and data used during the execution of the camera program is not limited to the flash memory, but an optical recording medium such as a CD-ROM or DVD-ROM, or a magnetic recording such as an MD. A semiconductor memory such as a medium, a tape medium, or an IC card can be used.

  The display unit 23 is for displaying various setting information of the digital camera 1 in addition to the captured image and the live view image, and is realized by a display device such as an LCD (Liquid Crystal Display) or an EL display (Electroluminescence Display). Is done. The display drive circuit 24 drives the display unit 23. On the display unit 23, the live view image is redrawn at a predetermined display frame rate during the shooting mode and continuously displayed as a moving image, and the shot image is displayed during the playback mode.

  The operation unit 26 receives various operations by the user, such as an instruction for photographing timing, a setting operation for various photographing modes including a correction mode and a playback mode, and a photographing condition setting operation, and notifies the CPU 21 of an operation signal. It is realized by button switches, slide switches, dials, etc., to which various functions are assigned. The operation unit 26 includes the release switch 3, the power switch 4, the menu switch 5, the cross key 6, the OK switch 7 and the reproduction switch 8 of FIG. 1.

  The communication I / F 25 is an interface for connecting the digital camera 1 to an external device such as a personal computer according to a communication standard such as USB (Universal Serial Bus).

  The removable memory 27 is a memory card that is detachable from the digital camera 1 such as an xD-Picture Card (registered trademark) or a Compact Flash (registered trademark) card. In the detachable memory 27, image data of a captured image is written by a read / write device (not shown) corresponding to the type, or image data recorded in the detachable memory 27 is read by the read / write device.

  The power supply circuit 28 converts the power supplied by the battery 29 loaded therein into predetermined power and supplies it to each part of the digital camera 1.

  Here, an outline of the correction mode, which is one of the functions of the digital camera 1 configured as described above, will be described. As described above, the digital camera 1 according to the present embodiment performs the main image processing with correction including the correction processing of the face area when the correction mode is selected as the setting related to the shooting mode.

  Here, the correction process is a process for expressing the face of a subject such as a person beautifully, and the processing content is not particularly limited, but examples include a smoothing process for smoothing the skin, Whitening processing for processing white, catchlight processing for synthesizing catchlight on the pupil, glossing processing for adjusting gloss and color of the lips by adjusting the brightness and color of the lips, and the like. In the smoothing process, for example, correction is performed by detecting a wrinkle or a spot portion from the skin region and replacing it with the surrounding skin state. In the whitening process, for example, correction for detecting a reddish portion of the skin and replacing it with the color of the surrounding skin, correction for increasing the whiteness of the skin, or processing for correcting the skin to a tan color is performed. In addition to the exemplified correction processing, for example, correction that makes the eyes look larger by adjusting the degree of opening of the eyes may be performed. These correction processes can be realized by using known techniques disclosed in, for example, Japanese Unexamined Patent Application Publication Nos. 2004-180114, 2004-222118, and 2004-318205. However, the present invention is not limited to the known techniques disclosed in these documents, and can be realized by appropriately using a correction process performed on the face. Hereinafter, in the present embodiment, three types of correction processing are performed in combination, for example, skin smoothing processing is performed as the first correction processing, skin whitening processing is performed as the second correction processing, and third correction processing is performed. As a lip gloss treatment.

  Incidentally, it takes a certain amount of time to perform the above-described correction processes on the captured image. For this reason, when the user selects the correction mode and performs shooting, the user has to wait until each correction process is completed, which is tedious. Therefore, the digital camera 1 according to the present embodiment performs an effect process for displaying the effect small image on the display unit 23 during the execution of the corrected main image process including each correction process. More specifically, at this time, an effect process for presenting the process of the main image process with correction to the user step by step is performed. Note that the order of each correction process performed on the actual captured image is not particularly limited. However, in order to present the process of the correction process, for example, the first correction process is referred to as a first-stage correction process, a first correction process, and the like. The second correction process is a second-stage correction process, and the third correction process is a final-stage correction process.

  First, the generated captured image is reduced to generate image data of a small image. Then, this small image is processed as a pre-processed small image, and a plurality of effect small images are generated. For example, a first stage processed small image obtained by performing smoothing processing on a face area in a pre-processed small image, a second stage processed small image obtained by performing whitening processing on a face area in the first stage processed small image, and a second stage processed Each image data (step-processed small image data) of the final processed small image obtained by applying the gloss process to the face area in the small image is generated. Also, a partial image obtained by cutting out a part of each of these processed small images and pre-processed small images, a partial enlarged image obtained by enlarging these partial images to a predetermined size, these partial enlarged images, each processed small image, and pre-processing A rotated small image is generated by rotating the small image by a predetermined amount. Then, as necessary, a predetermined template prepared in advance is combined with each processing small image, partial image, and rotation processing small image to obtain a plurality of effect small images. Here, the image processing for the pre-process small image for obtaining the effect small image does not take time as in the case of performing each correction process on the captured image, and can be realized in a short time. In addition to this, a moving image (animation image) or background image data of a still image used as an effect image is prepared in advance.

  If the effect small image is generated as described above, the process moves to the effect processing of the effect small image. For example, first, the display order of each effect small image is set. Then, according to the set display order, each small effect image is sequentially switched to the display unit 23 for each predetermined display time and displayed, and a moving image is displayed as a slide show. 3 and 4 are diagrams illustrating the principle of the effect process.

  For example, in the example of FIG. 3, based on the photographed image, the pre-processing small image, the partially enlarged image of the pre-processing small image, and the first stage processing small image and the second stage processing in which the templates are respectively combined at predetermined positions. A small image and a partially enlarged image of the final processed small image are generated. And it uses as a small image for production with the animation image prepared beforehand.

  As a specific effect process, first, as shown in FIGS. 3A to 3C, the pre-process small image and the pre-process small image are displayed in order, and the subject is zoomed in stages. indicate. Then, as shown in FIG. 3C, focus marks F11 and F12 are displayed on the face of the subject to be corrected. Next, as shown in FIG. 3D, an animation image prepared in advance is displayed. Then, as shown in FIG. 3E, a partial enlarged image of the first stage processed small image is displayed, and the first stage correction processing image for the face is presented to the user as a process of the correction processing. FIG. 3E shows a partially enlarged image of the first stage processed small image obtained by combining the templates T11 and T12 in the vicinity of the face of the subject. As described above, by appropriately combining the focus marks F11 and F12 (FIG. 3C) and the templates T11 and T12 (FIG. 3E) with the pre-processing image and each processing small image, it is possible to enhance the effect. it can.

  Next, after the animation image is displayed again as shown in FIG. 3 (f), the second stage processed small image obtained by combining the templates T13 and T14 in the vicinity of the face of the subject as shown in FIG. 3 (g). The partially enlarged image is displayed, and the second-stage correction processing image for the face is presented to the user as the next process of the correction processing. Next, the animation image is displayed again as shown in FIG. Then, as shown in FIG. 3 (i), a partial enlarged image of the final processed small image obtained by combining the templates T15 and T16 is displayed in the vicinity of the face of the subject, and the final correction processing image for the face (correction for the captured image). The final image of the supplementary image processing) is presented to the user.

  On the other hand, in the example of FIG. 4, a predetermined range of the pre-processing small image, the pre-processing small image, the first stage processing small image, the second stage processing small image, and the final processing small image is cut out based on the captured image. A partial image, a partial enlarged image obtained by enlarging these partial images with different sizes, and a rotation processed small image obtained by rotating the partial enlarged image of the second stage processed small image with different rotation amounts are generated as a small image for production. Use.

  As specific effect processing, first, as shown in FIGS. 4A to 4C, a partial image of a pre-processing small image or a partial enlarged image obtained by enlarging this partial image with different sizes is enlarged stepwise. Are displayed in order, and the subject is enlarged. Next, as shown in FIGS. 4D to 4I, the partial image of the first stage processed small image and the partial image obtained by enlarging the partial image with different sizes gradually become smaller or larger. In this way, the first-stage correction processing image for the face is presented to the user as a correction processing process while the subject is enlarged and reduced by displaying in order.

  Next, as shown in FIG. 4 (j) to FIG. 4 (l), while displaying the rotation processed small images obtained by rotating the partially enlarged images of the second stage processed small images with different rotation amounts in order and displaying the motion. Then, the second-stage correction processing image for the face is presented to the user as a correction processing process. Then, as shown in FIG. 4 (m), a partially enlarged image of the final processed small image is displayed, and a final corrected image (finished image) for the face is presented to the user.

  As described above, in this embodiment, until the main image processing with correction is completed, that is, until the image processing for the captured image is completed by performing each of the three types of correction processing, the small effect image is displayed. The used effect processing can be performed. Therefore, the user can wait while checking the stepwise process of the correction process, and is not bored.

  3 (a) to (c), (e), (g), (i) and FIGS. 4 (a) to (m) do not show images in a state where correction processing is reflected. Actually, an image of the state after the first stage correction process, the state after the second stage correction process, and the state after the final stage correction process is displayed on the display unit 23.

  Next, the operation of the digital camera 1 will be described. FIG. 5 is a flowchart showing the basic operation of the digital camera 1. As shown in FIG. 5, the digital camera 1 is in a standby state until the power switch 4 is slid and an instruction to turn on the power (power switch ON) is given, and sleep while the power switch is OFF (step a1: No). The state is continued (step a3), and the process returns to step a1.

  When the power switch is turned on (step a1: Yes), the power supply circuit 28 starts supplying power to each unit (step a5), and first, the shooting mode is set (step a7). For example, various settings relating to the shooting mode such as settings relating to photometry and exposure calculation, and settings relating to AE / AF operation are performed. In the present embodiment, at this time, a correction mode for correcting the face area when a face is detected in the captured image is set. Thereafter, photometry / calculation processing is performed (step a9).

  Subsequently, the live view image is captured and displayed on the display unit 23 (step a11). By the processing here, the image data of the subject image formed on the image sensor 11 is temporarily recorded in the SDRAM 18 and displayed on the display unit 23 as a live view image. The live view image is updated and displayed at a predetermined frame rate and is continuously displayed as a moving image.

  Thereafter, the process branches according to the operation input. That is, when the release switch 3 is pressed down one step and the first release switch is turned on (step a13: Yes), a photographing process is performed (step a15). If the first release switch is not turned on (step a13: No), the process proceeds to step a17. When the regeneration switch 8 is turned on (step a17: Yes), the regeneration mode is set and the regeneration process is performed (step a19). If the regeneration switch 8 is not turned on (step a17: No), the process proceeds to step a21. When the menu switch 5 is turned on (step a21: Yes), a setting change process is performed (step a23). For example, the CPU 21 starts setting change processing, displays a setting change screen on the display unit 23, and presents various setting menus. Then, the CPU 21 performs a process according to the setting item selected by the user operation, and executes a process for changing the setting of various modes according to the operation signal from the operation unit 26. Various settings relating to the shooting mode and the playback mode can be changed by the setting change processing here. For example, the setting as to whether or not the correction mode is selected as the shooting mode can be changed (switched). If the menu switch 5 is not turned on (step a21: No), the process proceeds to step a25.

  Then, until the power switch 4 is slid and the power cut-off (power switch OFF) is instructed (step a25: No), the process returns to step a9 to perform the above-described processing. When the power switch 4 is turned off (step a25: Yes), the power supply of each part by the power supply circuit 28 is stopped (step a27), and the basic operation is finished.

  Next, the photographing process (step a15) performed when the first release switch is turned on in the basic operation shown in FIG. 5 will be described. FIG. 6 is a flowchart showing a detailed processing procedure of the photographing process. As shown in FIG. 6, in the photographing process, first, as photographing preparation operation, the AF unit 17 performs AF (automatic focus) operation (step b1), and the AE unit 16 performs AE (automatic exposure) operation (step b3). .

  Thereafter, the second release switch is not turned on (step b5: No), and the first release switch remains on (step b7: Yes), and the standby state is entered. If the first release switch is turned off (step b7: No), the process returns to step a15 in FIG. 5, and then proceeds to step a17.

  When the release switch 3 is pressed in two steps and the second release switch is turned on (step b5: Yes), the CPU 21 drives the imaging unit 10 to control exposure during shooting (step b9). Image data reading processing is performed (step b11). With this processing, an image in the shooting range at the timing when the second release switch is turned on is generated as a shot image, and the image data is temporarily recorded in the SDRAM 15.

  Subsequently, the image processing unit 18 generates image data of a small image by reducing the image data of the captured image (Step b12). Then, the CPU 21 determines whether or not the shooting mode in which the correction mode is selected is set. If the correction mode is not selected (step b13: No), the process proceeds to step b21, and the image processing unit 18 performs normal image processing that does not correct the face on the captured image. Then, the process proceeds to step b23. On the other hand, when the correction mode is selected (step b13: Yes), the organ detection unit 20 performs face detection (step b15). The organ detection unit 20 detects a face area from the small image, and detects each face part in the face area based on the detection result of the face area.

  Thereafter, the CPU 21 determines whether or not a face has been detected by the face detection in step b15. If not (step b17: No), the CPU 21 proceeds to step b21. That is, as described above, the image processing unit 18 performs normal image processing that does not correct the face on the captured image. Then, the process proceeds to step b23. On the other hand, if a face is detected (step b17: Yes), the process proceeds to an image correction process with correction (step b19), and image processing for correcting the face is performed on the captured image. FIG. 7 is a flowchart showing a detailed processing procedure of the image processing with correction.

  As shown in FIG. 7, in the image processing with correction, first, the effect image processing unit 183 of the image processing unit 18 generates an effect small image (step c1). For example, the effect image processing unit 183 sets the small image generated in step b12 in FIG. 6 as the pre-processing small image, the first stage processed small image, the second stage processed small image, the final processed small image, the partial image, and the partial image. An enlarged image, a rotation processed small image, and the like are generated, and a predetermined template (focus marks F11 and F12 and templates T11 to T16 in FIG. 3) is combined with each image as necessary to produce a small effect image.

  Subsequently, the corrected main image processing unit 181 of the image processing unit 18 starts the corrected main image processing (step c3). FIG. 8 is a flowchart showing a detailed processing procedure of the main image processing with correction started here. As shown in FIG. 8, in the main image processing with correction according to the present embodiment, the main image processing unit 181 with correction first starts based on the face detection result performed on the small image in step b15 in FIG. Then, the position in the captured image corresponding to these detection positions in the small image is specified as the face area or the area of each face part in the captured image (step d1). Then, the corrected main image processing unit 181 performs, for example, a skin smoothing process as the first correction process on the captured image (step d2), and performs a skin whitening process as the second correction process (step d3). The lip gloss processing is performed as the third correction processing (step d4), and other necessary image processing is performed to generate a photographic image with correction.

  Then, if the corrected main image processing unit 181 starts the corrected main image processing, the CPU 21 produces the effect in parallel with the corrected main image processing until the correction main image processing ends. The effect processing using the effect small image generated by the image processing unit 183 and the effect image (such as the animation image in FIG. 3) prepared in advance as a template is performed.

  In other words, the CPU 21 sets the display order of each effect small image used for effect processing, and sequentially selects the effect small images according to the display order as shown in FIG. 7 (step c4). Subsequently, the CPU 21 starts a timer and starts measuring a predetermined time (display time) (step c5), and then performs an effect process in which the effect small image selected in step c4 is selected and displayed on the display unit 23. Perform (step c7).

  Then, the CPU 21 monitors the time count of the timer and enters a standby state until a predetermined time is exceeded. When the time measured by the timer exceeds the predetermined time (step c9: Yes), the CPU 21 determines whether or not the main image processing with correction is finished, and until it is finished (step c11: No), the step Returning to c4, the above processing is repeated. That is, the next small effect image is selected and the timer is reset to start counting a predetermined time again, and an effect process is performed in which the selected small effect image is displayed on the display unit 23 for the display time. On the other hand, when the main image processing with correction is completed (step c11: Yes), the process returns to step b19 in FIG. 6 and then proceeds to step b23.

  In step b23, the CPU 21 performs a process (rec view) for displaying the confirmation image on the display unit 23. When the correction mode is not selected or when the correction mode is selected but no face is detected in the captured image, the captured image after normal image processing is displayed on the display unit 23 as a confirmation image. On the other hand, when the correction mode is selected and a face is detected in the captured image, the captured image with the corrected face after image processing with correction is displayed on the display unit 23 as a confirmation image.

  Then, the CPU 21 records the image data (corrected image data) of the captured image after the normal image processing in step b21 or the captured image (corrected captured image) after the main image processing with correction in step b19 in the removable memory 27. (Step b25). And it returns to step a15 of FIG. 5, and transfers to step a17 after that.

  Here, the image data of the captured image is recorded in the removable memory 27 in association with the thumbnail data that is the image data of the small image. Further, the photographic image with correction is further recorded as an image file with correction associated with slide data for reproducing the effect process performed by repeating Step c4 to Step c9 of FIG. FIG. 9 is a diagram illustrating a data configuration example of an image file with correction. As shown in FIG. 9, the image file with correction includes a header 41, main image data 42, thumbnail data 43, and slide data 44. The main image data 42 is image data of a captured image with correction. The slide data 44 is, for example, for displaying a small image for production used in the production process as a slide show. The slide images 44 connect the small images for production in the display order and set the display time of each production image. Generated as data.

  In addition, it is good also as a structure which does not record the image for production (slide data) in association with the captured image with correction. Further, the image data of the original photographed image (original image) before the correction process may be recorded in association with the photographed image with correction.

  Next, the reproduction process (step a19) performed when the reproduction switch 8 is turned on in the basic operation shown in FIG. FIG. 10 is a flowchart showing a detailed processing procedure of the reproduction process. As shown in FIG. 10, in the reproduction process, first, the CPU 21 performs a process of reading a photographed image from the detachable memory 27 (step e1), and performs a process of displaying the read photographed image on the display unit 23 to reproduce (step e1). e3). For example, image data (main image data) of captured images that have been captured in the past and recorded in the removable memory 27 may be sequentially read and displayed one by one in the order of capture, or thumbnail data of a plurality of captured images. It is good also as a structure which performs the process which reads and displays a list, and displays the image data (main image data) of the picked-up image selected from the list according to user operation.

  Subsequently, for example, when the cross key 6 is turned on and a switching operation (playback switching operation) of a captured image being reproduced is input (step e5: Yes), the CPU 21 removes the designated captured image from the removable memory 27. (Step e7), and the process returns to step e3 to display (read) the captured image read on the display unit 23.

  When the captured image being reproduced is a captured image with a corrected face, for example, a slide show reproduction menu is presented. For example, the menu switch 5 or the cross key 6 is turned on to display this menu. When the slide show playback menu is selected and a slide playback operation is input (step e9: Yes), the CPU 21 reads the slide data associated with the corrected captured image being played back from the removable memory 27 and reads each slide data. Processing for sequentially displaying the small images for performance on the display unit 23 is performed (step e11). According to this, since it is possible to reproduce the effect process that was performed when the captured image with correction being generated is generated, the user can confirm the original captured image (original image) before performing the correction process by a slide show. can do.

  Then, until the playback switch 8 is turned on again (step e13: No), the process returns to step e5 to accept a playback switching operation and a slide playback operation. If the regeneration switch 8 is turned on (step e13: Yes), the regeneration process is terminated, the process returns to step a19 in FIG. 5, and then proceeds to step a21. As a result, the digital camera 1 changes from the playback mode to the shooting mode.

  As described above, according to the present embodiment, the correction mode for correcting the face in the photographed image is selected as the setting relating to the photographing mode, and the face in the photographed image is detected by the organ detection unit 20. When detected, the main image processing with correction is performed on the photographed image, and until the main image processing with correction is finished, the effect processing can be performed in parallel with the main image processing with correction. . Specifically, the captured image is reduced to generate a pre-processed small image, and the first-stage processed small image obtained by performing the smoothing process on the pre-processed small image or the second-stage obtained by performing the whitening process on the first-stage processed small image. It is possible to generate a final processed small image obtained by applying a gloss process to the step processed small image and the second step processed small image, and to generate a small effect image using the processed small image and the pre-processed small image. And the effect process which displays each image for an effect on the display part 23 sequentially can be performed. Therefore, until the main image processing with correction including each correction processing for the face in the photographed image is completed, the effect display can be performed while showing the process of the correction processing step by step, and the correction is performed without boring the user. An image after processing can be presented.

  In the above-described embodiment, the first correction process is a skin smoothing process, the second correction process is a skin whitening process, the third correction process is a lip gloss process, and the captured image is reduced. By performing these correction processes on the pre-processed small image, an effect display that presents the process of the correction process step by step to the user is performed. On the other hand, the combination and number of correction processes to be applied can be set as appropriate. Then, it is possible to perform a correction process on the pre-process small image according to the combination and number of correction processes to be applied, and to perform an effect display that presents the processing process to the user step by step.

  In the embodiment described above, three types of correction processing are performed in combination, and the first-stage processed small image and the second-stage processed small image obtained by sequentially performing these correction processes on the pre-processed small image. The final stage processing small image is used to perform an effect display in which the correction process is presented to the user step by step. On the other hand, it is good also as producing | generating a several step process small image by performing one correction process in steps. For example, in the case of performing whitening processing, the skin color may be gradually whitened to generate a plurality of small-level processed images. And it is good also as performing the display which displays these on the display part 23 in order, and performing the effect display which shows a process of a correction process to a user in steps.

  Further, the effect process illustrated in FIGS. 3 and 4 is an example, and the type, combination, and display order of effect small images used for the effect process can be set as appropriate. In the above-described embodiment, the display time of each small effect image is described as being the same. However, for example, it may be set individually according to the type of the small effect image. Alternatively, the processing time required for the main image processing with correction may be estimated in advance according to the type of correction processing performed in the main image processing with correction, and the display time may be assigned to each small effect image according to the estimated processing time. Good. If predetermined correction processing is to be performed, an estimated time may be set in advance. If the correction process can be selected according to the user operation, an estimated time is set for each correction process in advance, and the process time may be estimated in accordance with the actually performed correction process.

  In the above-described embodiment, as an example of the motion display, an example in which the rotation-processed small images obtained by rotating the partially enlarged images of the second-stage process small image with different rotation amounts are sequentially displayed (FIG. 4 ( j) to FIG. 4 (l)), the motion display is not limited to this. For example, it is good also as a structure which moves each image for production | presentation along a predetermined | prescribed path | route, and displays a motion.

  In the above-described embodiment, the first-stage processed small image, the second-stage processed small image, and the final-stage processed small image obtained by performing the correction process on the pre-processed small image in stages are generated. Although the effect process is performed using the processed small image, the effect process may be performed using any one of them.

  In the embodiment described above, the organ detection unit 20 performs face detection on the image data of the small image, detects the face area, and detects each face part. And based on this face detection result, it decided to specify the position in the picked-up image corresponding to these detection positions in a small image as a face area | region and area | region of each face part in a picked-up image. According to this, since face detection is performed on a small image having a small size, the processing load can be reduced as compared with the case where face detection is performed on a captured image. However, face detection may be performed on the image data of the photographed image to detect a face area or each face part area in the photographed image. Alternatively, based on the face detection result performed on the image data of the photographed image, the position in the small image corresponding to these detection positions in the photographed image can be changed to the face area or face part area in the small image. It may be specified as Then, a small effect image may be generated based on the specified face area or face part area.

  In the above-described embodiment, a thumbnail obtained by reducing a captured image is used as a small image. However, the small image is not limited to a thumbnail. For example, the captured image may be reduced to a VGA size (640 × 480 pixels) to generate a small image. In addition, the size of the small image can be set as appropriate, and the small image may be generated by performing reduction processing to a predetermined size smaller than the captured image.

  Further, part or all of the processing described as being performed by the CPU 21 in the above-described embodiment can be realized by hardware. On the other hand, the configuration in which the image processing unit 18, the organ detection unit 20, and the like are realized by hardware has been described, but may be realized as software by executing a predetermined program.

  In the above embodiment, a digital camera has been described as a specific example of the imaging apparatus. However, the present invention is not limited to this, and the present invention may be applied to a camera unit of a mobile phone or a camera unit attached to a PC. The case where the correction processing is performed by processing still image data has been described, but the target of the correction processing is not limited to still image data, and can be similarly applied to moving image data.

It is a rear view of a digital camera. It is a block diagram which shows the system configuration | structure of a digital camera. It is a figure explaining the principle of effect processing. It is another figure explaining the principle of effect processing. 3 is a flowchart showing basic operations of the digital camera. It is a flowchart which shows the detailed process sequence of an imaging | photography process. It is a flowchart which shows the detailed process sequence of an image process with a correction | amendment. It is a flowchart which shows the detailed process sequence of this image process with a correction | amendment. It is a figure which shows the data structural example of the image file with a correction | amendment. It is a flowchart which shows the detailed process sequence of a reproduction | regeneration process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 3 Release switch 4 Power switch 5 Menu switch 6 Four-way controller 7 OK switch 8 Playback switch 10 Image pick-up part 11 Image pick-up element 12 Lens system unit 13 Lens drive circuit 14 Image pick-up circuit 15 SDRAM
16 AE section 17 AF section 18 Image processing section 181 Main image processing section with correction 183 Production image processing section 20 Organ detection section 21 CPU
211 Production Processing Unit 22 Built-in Memory 23 Display Unit 24 Display Drive Circuit 25 Communication I / F
26 Operation Unit 27 Detachable Memory 28 Power Supply Circuit 29 Battery 30 Bus

Claims (10)

An imaging unit that photoelectrically converts a subject image to generate image data;
A small image generation unit that generates small image data obtained by reducing the image data generated by the imaging unit;
A face detection unit for detecting a face in the image data and / or in the small image data generated by the small image generation unit;
A corrected main image processing unit that generates corrected image data obtained by performing a predetermined correction process on a face region in the image data based on a face detection result by the face detection unit;
A small image processing unit that generates processing small image data obtained by performing the correction processing on a face region in the small image data based on a face detection result by the face detection unit;
In parallel with the processing by the corrected main image processing unit, an effect processing unit that performs effect processing using the processed small image data;
An imaging apparatus comprising: The small image processing unit performs the correction process stepwise on the face area in the small image data to generate a plurality of step processed small image data,
The imaging apparatus according to claim 1, wherein the effect processing unit performs a process of displaying the plurality of step-processed small image data as moving images as the effect process. The main image processing unit with correction performs a plurality of types of correction processing on the face area in the image data,
The small image processing unit sequentially performs the plurality of types of correction processing on a face region in the small image data to generate the plurality of step-processed small image data. Imaging device.   The effect processing unit performs a process of inserting predetermined background image data between at least one of the plurality of step-processed small image data and displaying a moving image as the effect process. The imaging apparatus according to claim 2, characterized in that: The small image processing unit includes a process for changing the size of the processed small image data, a process for cutting out a part of the processed small image data, a process for rotating the processed small image data with different rotation amounts, and the processed small image data. Change the form of the processed small image data by performing at least one of the processes of changing the outer shape of
The imaging apparatus according to claim 1, wherein the effect processing unit performs a process of displaying the processed small image data in which the form has been changed as the effect process. The small image processing unit synthesizes a predetermined template at a position near the face area in the processed small image data,
The imaging apparatus according to claim 1, wherein the effect processing unit performs a process of displaying the small image data in which the predetermined template is combined as the effect process.   The imaging apparatus according to claim 1, further comprising a recording unit that records the corrected image data and the processed small image data in association with each other. The face detection unit detects a face in the small image data,
The main image processing unit with correction specifies a corresponding position in the image data as a face region based on a face detection result performed on the small image data by the face detection unit, and sets the specified face region. The imaging apparatus according to claim 1, wherein the correction process is performed on the imaging apparatus. An image processing method in an imaging device including an imaging unit that photoelectrically converts a subject image to generate image data,
A small image generation step of generating small image data obtained by reducing the image data generated by the imaging unit;
A face detection step of detecting a face in the image data and / or in the small image data generated in the small image generation step;
Based on the face detection result in the face detection step, a corrected main image processing step for generating corrected image data obtained by performing a predetermined correction process on the face area in the image data;
Based on the face detection result in the face detection step, a small image processing step of generating processed small image data obtained by performing the correction processing on the face region in the small image data;
In parallel with the processing of the corrected main image processing step, the effect processing step of performing the effect processing using the processed small image data,
An image processing method in an imaging apparatus, comprising: In a computer of an imaging apparatus having an imaging unit that photoelectrically converts a subject image to generate image data,
A small image generation step of generating small image data obtained by reducing the image data generated by the imaging unit;
A face detection step of detecting a face in the image data and / or in the small image data generated in the small image generation step;
Based on the face detection result in the face detection step, a corrected main image processing step for generating corrected image data obtained by performing a predetermined correction process on the face area in the image data;
A small image processing step for generating processed small image data obtained by performing the correction processing on a face region in the small image data based on the face detection result in the face detecting step;
In parallel with the main image processing step with correction, an effect processing step for performing an effect process using the processed small image data;
An image processing program for executing

Images