JP2010191784A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for shortening a time until processing a face. <P>SOLUTION: An imaging device 1 includes: an imaging part 20 for picking up a subject image to generate an image; a face area detecting part 40 for detecting an face area of a person from the image picked up by the imaging part; a storage part 10 for storing area information indicating the face area detected by the face area detecting part 40 from the through image picked up by the imaging part 20; an unsupported face estimation area selecting part 50 for selecting an unsupported face estimation area not corresponding to the face area detected when picking up the image, out of a face estimation area estimated with the presence of the face area in the image, based on the area information stored in the storage part 10, when the image is picked up by the imaging part 20; and an image processing part 90 for performing image processing to the unsupported face estimation area selected by the unsupported face estimation area selecting part 50. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus.

  Currently, a technique for detecting a human face area in a moving image (also referred to as a through image) is known (see, for example, Patent Document 1). There is also known a technique for performing various processes (for example, red-eye correction, skin color correction, backlight correction, hereinafter referred to as “face processing”) on a face area.

  The apparatus described in Patent Document 1 detects a face area from an area narrowed down based on a through image captured before capturing a main image after capturing a still image (also referred to as a main image).

JP 2007-42072 A

  However, the apparatus described in Patent Document 1 has a problem in that it takes time to perform face processing because the face area is detected after the main image is captured. The present invention has been made in view of such circumstances, and an object of the present invention is to provide a technique for shortening the time until face processing is performed.

  In order to solve the above problems, an imaging device according to one embodiment of the present invention includes an imaging unit that captures a subject image and generates an image, and a face region that detects a human face region from the image captured by the imaging unit A detection unit; a storage unit that stores area information indicating a face area detected by the face area detection unit from a through image captured by the imaging unit; and a main image captured by the imaging unit when the main image is captured by the imaging unit. When an image is captured, it does not correspond to the face area detected at the time of capturing the main image from the face estimation areas estimated to have a face area in the main image based on the area information stored in the storage unit An unsupported face estimation region selection unit that selects an unsupported face estimation region, and an image processing unit that performs image processing on the unsupported face estimation region selected by the unsupported face estimation region selection unit.

  The imaging apparatus is not supported by a detection failure probability calculation unit that calculates a detection failure probability that is a probability that the face region detection unit has failed to detect a face region in an unsupported face estimation region, and a scaling rate according to the detection failure probability. You may make it further provide the uncorresponding face estimation area expansion / contraction part which expands / contracts a face estimation area.

  The imaging apparatus includes a detection failure probability calculation unit that calculates a detection failure probability that is a probability that the face region detection unit has failed to detect a face region in an unsupported face estimation region, and performs image processing on the unsupported face estimation region A parameter setting unit that sets a parameter indicating the strength of the time, the parameter setting unit sets a parameter indicating a weaker strength as the detection failure probability is higher, and the image processing unit responds to the strength of the parameter. Then, image processing may be performed on the unsupported face estimation area.

  In the imaging apparatus, the detection failure probability calculation unit refers to the region information stored in the storage unit, and corresponds to an unsupported face estimation region among a plurality of through images captured within a certain time before imaging of the main image. The ratio of the through image in which the face area to be detected was not detected may be calculated as the detection failure probability.

  In the imaging apparatus, the unsupported face estimation region selection unit, when a change in the position of the face region in the plurality of through images is smaller than a predetermined threshold, uses the face region detected from the latest through image as the face estimation region. Also good. In addition, when the position of the face area in the plurality of through images is moving in a certain direction, the unsupported face estimation area selecting unit expands the face area detected from the latest through image in the certain direction. It may be a face estimation area. Further, when the position of the face area in the plurality of through images is moving in a certain direction, the unsupported face estimation area selection unit may set an area including all the face areas in the plurality of through images as the face estimation area. .

  According to the present invention, the image processing is performed on the non-corresponding face estimation area without detecting the face area after the main image is captured, so that the time until the face processing is performed can be shortened.

1 is a block diagram illustrating a configuration example of an imaging apparatus 1 according to a first embodiment of the present invention. 6 is an explanatory diagram for explaining an operation of a face area detection unit 40. FIG. FIG. 4 is a diagram illustrating an example of face area information stored in a face area information storage unit 10. 6 is an explanatory diagram for explaining an operation of an unsupported face estimation region selection unit 50. FIG. 10 is an explanatory diagram for explaining an operation of an unsupported face estimation region selection unit 50. FIG. 10 is an explanatory diagram for explaining an operation of an unsupported face estimation region selection unit 50. FIG. 10 is an explanatory diagram for explaining an operation of an unsupported face estimation region selection unit 50. 3 is a flowchart illustrating an example of the operation of the imaging apparatus 1. It is a block diagram which shows the structural example of the imaging device 2 which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the imaging device 2 which concerns on the 2nd Embodiment of this invention. 3 is a flowchart illustrating an example of the operation of the imaging apparatus 3.

[First embodiment]
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of an imaging apparatus 1 according to the first embodiment of the present invention. FIG. 2 is an explanatory diagram for explaining the operation of the face area detection unit 40. FIG. 3 is a diagram illustrating an example of face area information stored in the face area information storage unit 10. 4-7 is explanatory drawing for demonstrating operation | movement of the uncorresponding face estimation area | region selection part 50. FIG. As illustrated in FIG. 1, the imaging device 1 includes a face area information storage unit 10, an image storage unit 12, an imaging unit 20, a display control unit 30, a face region detection unit 40, an unsupported face estimation region selection unit 50, and image processing. Part 90 is provided.

  The imaging unit 20 includes a lens (not shown) that collects light, an imaging element (not shown) that performs photoelectric conversion, an image generation unit (not shown) that generates an image (through image and main image) from an electrical signal, and the like. However, details are omitted because it is a known technique. The imaging unit 20 generates a through image at a predetermined timing (for example, 30 frames / second) when a release signal is not acquired during imaging. The imaging unit 20 supplies the generated through image to the display control unit 30 and the face area detection unit 40 every time a through image is generated. The imaging unit 20 generates a main image when a release signal is acquired during imaging. When the main image is generated, the imaging unit 20 supplies the generated main image to the display control unit 30, the face area detection unit 40, and the image processing unit 90.

  The face area detection unit 40 acquires a through image and a main image from the imaging unit 20. When acquiring the through image, the face area detection unit 40 detects a human face area from the through image. For example, when acquiring the through image Gn shown in FIG. 2A, the face area detection unit 40 detects the face area An and the face area Bn. The same applies when the main image is acquired.

  When the face area detection unit 40 detects a face area from the through image, the face area detection unit 40 supplies area information indicating the face area detected from the through image (hereinafter referred to as “face area information”) to the display control unit 30 and the face area information. Is stored in the face area information storage unit 10. For example, when the face area An and the face area Bn are detected from the through image Gn shown in FIG. 2A captured at a certain time Tn, the face area detection unit 40 identifies the through image identification information “ Gn ”(for example, time“ Tn ”, imaging order“ n ”), face area information indicating the face area An (PAn, LAnx, LAny), and face area information indicating the face area Bn (PBn, LBnx, LBny) And the face area information is stored in the face area information storage unit 10. Similarly, when the face area detection unit 40 detects the face area Bn + 1 from the through image Gn + 1 shown in FIG. 2B captured at time Tn + 1, which is one through image capturing time after time Tn, the through image identification is performed. Information “Gn + 1” and face area information (PBn + 1, LBn + 1x, LBn + 1y) indicating the face area Bn + 1 are supplied to the display control unit 30 and the face area information is stored in the face area information storage unit 10. “PAn” is the upper left coordinate of the face area An in the through image Gn, “LAnx” is the length of the face area An in the through image Gn in the horizontal direction (X-axis direction), and “LAny” is the face in the through image Gn. This is the length in the vertical direction (Y-axis direction) of the region An. “PBn” is the upper left coordinate of the face area Bn in the through image Gn, “LBnx” is the length in the horizontal direction (X-axis direction) of the face area Bn in the through image Gn, and “LBny” is the face area Bn in the through image Gn. In the vertical direction (Y-axis direction). “PBn + 1” is the upper left coordinate of the face area Bn + 1 in the through image Gn + 1, “LBn + 1x” is the length in the horizontal direction (X-axis direction) of the face area Bn + 1 in the through image Gn + 1, and “LBn + 1y” is the face area Bn + 1 in the through image Gn + 1 In the vertical direction (Y-axis direction). If the face area cannot be detected from the through image, the face area detection unit 40 supplies the display control unit 30 with face area information (for example, null) that does not specify the face area.

  FIG. 3 shows the face area information stored in the face area information storage unit 10 when the face area An and the face area Bn are detected from the through image Gn and the face area Bn + 1 is detected from the through image Gn + 1. In the above case, the face area information storage unit 10 stores face area information (PAn, LAnx, LAny), face area information (PBn, LBnx, LBny), and through image identification information in association with the through image identification information “Gn”. The face area information (PBn + 1, LBn + 1x, LBn + 1y) is stored in association with “Gn + 1”. Note that the face area detection unit 40 is a face composed of the reference coordinates (upper left) of the face area, the length of the face area in the horizontal direction (X-axis direction), and the length of the face area in the vertical direction (Y-axis direction). Instead of supplying the area information to the display control unit 30 and storing the face area information in the face area information storage unit 10, the face area information configured in another format is supplied to the display control unit 30 and the face area is stored. Information may be stored in the face area information storage unit 10. For example, the face area detection unit 40 supplies face area information composed of two reference coordinates (for example, upper left and lower right) of the face area to the display control unit 30 and transmits the face area information to the face area information storage unit 10. May be stored.

  When the face area detection unit 40 detects a face area from the main image, the face area detection unit 40 supplies the face area information detected from the main image to the unsupported face estimation area selection unit 50. The face area detection unit 40 detects the face area information in the same format as the face area information stored in the face area information storage unit 10 when a face area is detected from the through image. It is preferable to supply to. When the face area cannot be detected from the main image, the face area detection unit 40 supplies face area information (for example, null) that does not specify the face area to the uncorresponding face estimation area selection unit 50.

  The display control unit 30 acquires a through image or a main image from the imaging unit 20. Further, the display control unit 30 acquires face area information related to the through image from the face area detection unit 40. When the display control unit 30 acquires the through image and the face area information, the display unit 30 displays information (for example, a frame) representing the area indicated by the face area information together with the through image (superimposed on the through image). (Not shown). When acquiring the main image, the display control unit 30 displays the main image on the display unit.

  The uncorresponding face estimation area selection unit 50 estimates that a face area exists in the main image based on the face area information stored in the face area information storage unit 10 when the main image is captured by the imaging unit 20. From the areas (hereinafter referred to as “face estimation areas”), areas not corresponding to the face areas detected when the main image is captured (hereinafter referred to as “unsupported face estimation areas”) are selected. Details will be described below.

  When it is determined that the main image has been captured, the uncorresponding face estimation area selection unit 50 first extracts a face estimation area based on the face area information stored in the face area information storage unit 10. For example, when the uncorresponding face estimation area selection unit 50 acquires the face area information Ar detected from the main image Gr shown in FIG. Face area information Ar-1 detected from the through image Gr-1 shown in FIG. 4E captured at time Tr-1 one time before the through image imaging time in the area information storage unit 10 and time Tr 4 is captured at the time Tr-2 one time before the through-image capturing time from -1, and the face area information Ar-2 detected from the through-image Gr-2 shown in FIG. Face area information Ar-3 detected from the through image Gr-3 shown in FIG. 4C, which was imaged at time Tr-3 before time, time Tr-4 one time before the through image imaging time from time Tr-3 The through image Gr− shown in FIG. Detected from the face region information Ar-4 and the face detected from the through image Gr-5 shown in FIG. 4A captured at time Tr-5 one imaging time before the time Tr-4. When the area information Ar-5 is stored, the positions of the face area information Ar-5, the face area information Ar-4, the face area information Ar-3, the face area information Ar-2, and the face area information Ar-1 are stored. Based on the change, as shown in FIG. 4G, an area Ax where a face area is estimated to exist at time Tr is extracted as a face estimation area.

  Further, the uncorresponding face estimation area selection unit 50 acquires the face area information As detected from the main image Gs shown in FIG. 5F captured at a certain time Ts from the face area detection unit 40. Each through image Gs-n (n) captured at time Ts-n (n = 5, 4, 3, 2, 1) shown in FIGS. 5A to 5E in the area information storage unit 10. When each face area information As-n (n = 5, 4, 3, 2, 1) detected from = 5, 4, 3, 2, 1) is stored, each face area information As-n Based on the change in position and size of (n = 5, 4, 3, 2, 1), as shown in FIG. 5G, face estimation is performed on an area Ax that is estimated to have a face area at time Ts. Extract as a region.

  In other words, the uncorresponding face estimation area selection unit 50 estimates the face area of the main image from the position and size of the face area of the previous through image (before capturing the main image), and extracts it as the face estimation area. Note that FIG. 4 illustrates, for example, imaging in a situation where the position of the imaging apparatus 1 moves up with respect to a stationary person or an angle (angle) moves up, or a person relative to a fixed imaging apparatus 1 It is an imaging in a situation in which moves downward. FIG. 5 illustrates, for example, a situation in which the position of the imaging device 1 moves forward with respect to a stationary person (approach to the subject), and a situation in which a person moves forward with respect to the fixed imaging device 1 ( Imaging in the approach of the subject) or imaging in which the subject size is enlarged (zoomed in) in the stationary imaging device 1 that is stationary. 4 and 5, the face estimation area is extracted with reference to five through images, but the number of through images to be referred to is not limited to five.

  Note that the unsupported face estimation area selection unit 50 enlarges the face area detected from the latest through image in a certain direction when the position of the face area in the plurality of through images moves in a certain direction. An area may be extracted as a face estimation area. For example, as shown in FIGS. 4A to 4E, the unsupported face estimation area selection unit 50 determines the position of each face area Ar-n (n = 5, 4, 3, 2, 1). When moving in a certain direction, an area Ax obtained by enlarging the face area Ar-1 in a certain direction may be extracted as a face estimation area as shown in FIG. Note that the unsupported face estimation region selection unit 50 may calculate an enlargement amount that is enlarged in a certain direction in accordance with a movement amount that moves in a certain direction. Further, in the above case, the uncorresponding face estimation area selection unit 50 may extract an area including all the face areas in the plurality of through images as the face estimation area. For example, the unsupported face estimation area selection unit 50 extracts the area Ax shown in FIG. 4 (i) including all the face areas Ar-n (n = 5, 4, 3, 2, 1) as the face estimation area. May be.

  Furthermore, when the change in the position of the face area in the plurality of through images is smaller than a predetermined threshold, the unsupported face estimation area selection unit 50 extracts the face area detected from the latest through image as the face estimation area. May be. For example, when the uncorresponding face estimation area selection unit 50 acquires the face area information At detected from the main image Gt shown in FIG. 6F captured at a certain time Tt from the face area detection unit 40, Each through image Gt-n (n) captured at the time Tt-n (n = 5, 4, 3, 2, 1) shown in FIGS. 6A to 6E in the region information storage unit 10. = 5, 4, 3, 2, 1), each face area information At-n (n = 5, 4, 3, 2, 1) is stored, and the center of gravity of the face area At-1 is stored A rectangular region K that is the same as Q and has a length in the X-axis direction and the Y-axis direction that is twice the length of the face region At-1 in the X-axis direction and the Y-axis direction (predetermined threshold value) When all of the face area At-n (n = 2, 3, 4, 5) is included, as shown in FIG. 6G, the face area At-1 is set as the face estimation area Ax. It may put.

  Note that the unsupported face estimation area selection unit 50 estimates a face area (a face area related to the same face from the position and size of the face area) from all through images captured within a certain time before the main image is captured. Even if the face area) cannot be detected, if the face area can be detected from through images of a predetermined ratio or more, the face estimation area may be extracted based on the face area. For example, when the predetermined ratio is 80%, the non-corresponding face estimation region selecting unit 50 determines the face region from 24 or more through images out of 30 through images captured within 1 second before capturing the main image. Can be detected, the face estimation area may be extracted based on the face area.

  The unsupported face estimation region selection unit 50 that has extracted the face estimation region selects an unsupported face estimation region that does not correspond to the face region detected from the main image from the face estimation regions. For example, the unsupported face estimation area selection unit 50 acquires the face area information Bu and Cu detected from the main image Gu shown in FIG. 7A captured at a certain time Tu from the face area detection unit 40, and 7 (d) to 7 (h), each through image Gu-n (n = 5, 4, 3, 2) captured at time Tu-n (n = 5, 4, 3, 2, 1). , 1) to each face area information Au-n (n = 5, 4, 3, 2, 1), Bu-n (n = 5, 4, 3, 2, 1) and Cu-n (n = 5, When the face estimation regions Ax, Bx, and Cx shown in FIG. 7B are extracted based on the change in position and size of 4, 3, 2, 1), face estimation is performed as shown in FIG. Of the areas Ax, Bx, and Cx, a face estimation area Ax that does not have a corresponding face area on the main image Gu is selected as an unsupported face estimation area X. The uncorresponding face estimation area selection unit 50 determines that the position and size of the face estimation area and the face area of the main image are substantially the same (for example, the difference between the positions of both areas is within a predetermined range). When the difference in size between the two areas is within a predetermined range), it is determined that the two areas correspond to each other.

  Note that the unsupported face estimation region selection unit 50 does not select the unsupported face estimation region when all the face estimation regions correspond to the face regions detected from the main image. For example, the unsupported face estimation region selection unit 50 does not select the unsupported face estimation region when acquiring the face region information Bu and Cu and extracting only the face estimation regions Bx and Cx. In addition, the unsupported face estimation area selection unit 50 does not detect a face area from the main image, but when the face estimation area is extracted, selects all face estimation areas as unsupported face estimation areas. For example, when the unsupported face estimation area selection unit 50 acquires face area information (for example, null) that does not specify a face area and extracts the face estimation areas Ax, Bx, and Cx, the face estimation areas Ax, Bx, Cx is selected as the unsupported face estimation region X.

  When the unsupported face estimation area selection unit 50 selects the unsupported face estimation area, the face area information acquired from the face area detection unit 40 (face area information detected from the main image) and the unsupported face estimation area Is supplied to the image processing unit 90. When the uncorresponding face estimation area selection unit 50 does not extract the uncorresponding face estimation area, the face area information acquired from the face area detection unit 40 (face area information detected from the main image) is sent to the image processing unit 90. Supply.

  The image processing unit 90 acquires the main image from the imaging unit 20. Further, the image processing unit 90 acquires face area information (face area information detected from the main image) and face area information indicating the unsupported face estimation area from the unsupported face estimation area selection unit 50. When only the face area information (face area information detected from the main image) is acquired from the unsupported face estimation area selection unit 50, the image processing unit 90 applies the face to the main image acquired from the imaging unit 20. Image processing (face processing) is performed on the face area indicated by the area information (face area information detected from the main image) and stored in the image storage unit 12. However, when face area information (for example, null) that does not specify a face area is acquired, face processing is not performed. When the image processing unit 90 acquires the face region information (face region information detected from the main image) and the face region information indicating the unsupported face estimation region from the unsupported face estimation region selection unit 50, the image processing unit 90 The main image acquired from 20 is subjected to face processing on the face area indicated by the face area information (face area information detected from the main image) and the uncorresponding face estimation area, and stored in the image storage unit 12. However, when face area information (for example, null) that does not specify a face area is acquired, face processing is performed only on the unsupported face estimation area.

  FIG. 8 is a flowchart illustrating an example of the operation of the imaging apparatus 1. Specifically, the flowchart of FIG. 8 is a flowchart for explaining an operation of performing face processing on the main image captured while capturing a through image.

  The face area detection unit 40 acquires a through image from the imaging unit 20 (step S100). The imaging unit 20 supplies the generated through image to the face area detection unit 40. The face area detection unit 40 that has acquired the through image from the imaging unit 20 detects the face area from the through image (step S110). The face area detection unit 40 that has detected the face area from the through image stores the face area information indicating the face area in the face area information storage unit 10 (step S120).

  The face area detection unit 40 determines whether the main image has been acquired from the imaging unit 20 (step S130). When it is determined that the main image is not acquired from the imaging unit 20 (step S130: No), the face area detection unit 40 determines that the main image is acquired from the imaging unit 20 (step S130: Yes). The area detection unit 40 repeatedly executes steps S100 to S120.

  On the other hand, when it is determined that the main image has been acquired from the imaging unit 20 (step S130: Yes), the face area detection unit 40 detects the face area from the main image (step S140). The face area detection unit 40 that has detected the face area from the main image supplies the face area information indicating the face area to the unsupported face estimation area selection unit 50.

  Subsequent to step S140, the unsupported face estimation area selection unit 50 extracts a face estimation area based on the face area information stored in the face area information storage unit 10 (step S150). The unsupported face estimation region selection unit 50 that has extracted the face estimation region selects an unsupported face estimation region that does not correspond to the face region detected from the main image from the face estimation regions (step S155). The unsupported face estimation region selection unit 50 that has selected the unsupported face estimation region indicates the face region information (face region information detected from the main image) acquired from the face region detection unit 40 and the unsupported face estimation region. The face area information is supplied to the image processing unit 90.

  The image processing unit 90 performs face processing on the face area indicated by the face area information (face area information detected from the main image) and the uncorresponding face estimation area with respect to the main image acquired from the imaging unit 20. It memorize | stores in the memory | storage part 12 (step S190). And this flowchart is complete | finished.

[Second Embodiment]
Hereinafter, a second embodiment of the present invention will be described in detail with reference to the drawings. FIG. 9 is a block diagram illustrating a configuration example of the imaging apparatus 2 according to the second embodiment of the present invention. As shown in FIG. 1 and FIG. 9, the imaging device 2 is the same except for the imaging device 1, the unsupported face estimation region selection unit 52, the detection failure probability calculation unit 60, and the unsupported face estimation region expansion / contraction unit 70. Therefore, the unsupported face estimation region selection unit 52, the detection failure probability calculation unit 60, and the unsupported face estimation region expansion / contraction unit 70 will be described.

  The unsupported face estimation region selection unit 52 extracts an unsupported face estimation region, similarly to the unsupported face estimation region selection unit 50 of the imaging device 1. When the unsupported face estimation region selection unit 52 extracts the unsupported face estimation region, the unsupported face estimation region selection unit 52 supplies the face processing unit 90 with the face region information acquired from the face region detection unit 40 (face region information detected from the main image). Then, the face area information indicating the unsupported face estimation area is supplied to the detection failure probability calculation unit 60. If the uncorresponding face estimation area selection unit 52 does not extract the uncorresponding face estimation area, the face area information acquired from the face area detection unit 40 (face area information detected from the main image) is sent to the image processing unit 90. Supply.

  The detection failure probability calculation unit 60 calculates a probability that face area detection has failed in the unsupported face estimation area (hereinafter referred to as “detection failure probability”). Specifically, when the detection failure probability calculation unit 60 acquires face region information indicating an unsupported face estimation region from the unsupported face estimation region selection unit 52, the face region stored in the face region information storage unit 10 With reference to the information, the ratio of the through image in which the face region corresponding to the unsupported face estimation region among the plurality of through images captured within the predetermined time before the capturing of the main image can be detected is calculated as the detection failure probability. For example, the detection failure probability calculation unit 60 relates to one uncorresponding face estimation area (the face corresponding to the same face from the position and size of the face area) among 30 through images captured within 1 second before the main image is captured. When there are three through images in which the face area corresponding to the face area estimated as the face area) cannot be detected, the detection failure probability is calculated as 10%. When the detection failure probability calculation unit 60 calculates the detection failure probability, the face failure information indicating the unsupported face estimation region acquired from the unsupported face estimation region selection unit 52 and the calculated detection failure probability are used as the unsupported face estimation. It supplies to the area expansion / contraction part 70.

  The uncorresponding face estimation region enlargement / reduction unit 70 acquires face region information indicating a non-corresponding face estimation region and detection failure probability from the detection failure probability calculation unit 60. When the uncorresponding face estimation region enlargement / reduction unit 70 acquires the face region information indicating the unsupported face estimation region and the detection failure probability from the unsupported face estimation region enlargement / reduction unit 70, the uncorresponding face is determined according to the enlargement / reduction rate according to the detection failure probability. Scale the estimated area. For example, the unsupported face estimation region enlargement / reduction unit 70 enlarges the unsupported face estimation region with a larger enlargement ratio as the detection failure probability is higher. When the unsupported face estimation area is enlarged or reduced, the unsupported face estimation area enlargement / reduction unit 70 supplies face area information indicating the unsupported face estimation area after enlargement / reduction to the image processing unit 90. If the unsupported face estimation area enlargement / reduction unit 70 does not scale the unsupported face estimation area, the unsupported face estimation area enlargement / reduction unit 70 supplies face area information indicating the unsupported face estimation area acquired from the detection failure probability calculation unit 60 to the image processing unit 90. To do.

  Note that the image processing unit 90 acquires face region information (face region information detected from the main image) from the unsupported face estimation region selection unit 52 and receives an unsupported face estimation region ( When face area information indicating unextended or expanded unsupported face estimation area) is acquired, face area information (face area information detected from the main image) is acquired for the main image acquired from the imaging unit 20. Is subjected to face processing and stored in the image storage unit 12. However, when face area information (for example, null) that does not specify a face area is acquired, face processing is performed only on the unsupported face estimation area.

[Third embodiment]
Hereinafter, a third embodiment of the present invention will be described in detail with reference to the drawings. FIG. 10 is a block diagram illustrating a configuration example of the imaging apparatus 3 according to the third embodiment of the present invention. As shown in FIGS. 1, 9, and 10, the imaging device 3 is the same except for the imaging device 1 or the imaging device 2, the detection failure probability calculation unit 62, the parameter setting unit 80, and the image processing unit 92. Therefore, the detection failure probability calculation unit 62, the parameter setting unit 80, and the image processing unit 92 will be described.

  The detection failure probability calculation unit 62 calculates the detection failure probability similarly to the detection failure probability calculation unit 60 of the imaging device 2. When the detection failure probability calculation unit 62 calculates the detection failure probability, the face failure information indicating the unsupported face estimation region acquired from the unsupported face estimation region selection unit 52 and the calculated detection failure probability are unsupported face estimation. While supplying to the area expansion / contraction part 70, the calculated detection failure probability is supplied to the parameter setting part 80.

  The parameter setting unit 80 acquires the detection failure probability from the detection failure probability calculation unit 62. When the detection failure probability is acquired from the detection failure probability calculation unit 62, the parameter setting unit 80 sets a parameter indicating the strength when image processing is performed on the unsupported face estimation region. Specifically, the parameter setting unit 80 sets (determines) a parameter indicating a weaker strength as the detection failure probability is higher. When the parameter setting unit 80 sets a parameter, the parameter setting unit 80 supplies the set parameter to the image processing unit 92.

  The image processing unit 92 acquires the main image from the imaging unit 20. The image processing unit 92 acquires face area information (face area information detected from the main image) from the unsupported face estimation area selection unit 52. Further, the image processing unit 92 acquires face area information indicating an unsupported face estimation area from the unsupported face estimation area enlargement / reduction unit 70. Further, the image processing unit 92 acquires parameters from the parameter setting unit 80.

  When only the face area information (face area information detected from the main image) is acquired, the image processing unit 92 performs face area information (detected from the main image) on the main image acquired from the imaging unit 20. Image processing (face processing) is performed on the face area indicated by (face area information) and stored in the image storage unit 12. However, when face area information (for example, null) that does not specify a face area is acquired, face processing is not performed. The image processing unit 92 has acquired face area information (face area information detected from the main image), face area information indicating an unsupported face estimation area (unexpanded or expanded unsupported face estimation area), and parameters. In this case, the face area indicated by the face area information (face area information detected from the main image) and the unsupported face estimation area (unexpanded or unsupported after expansion) with respect to the main image acquired from the imaging unit 20 (Face estimation area) is subjected to face processing (the unsupported face estimation area is subjected to face processing according to the intensity of the parameter) and stored in the image storage unit 12. However, when face area information (for example, null) that does not specify a face area is acquired, face processing is performed only on the unsupported face estimation area. Note that when the image processing unit 92 performs face processing with high intensity, the change due to face processing becomes larger or the area (pixel) to which face processing is applied than when face processing with low intensity is performed. There will be more. In other words, the higher the detection failure probability, the smaller the change due to face processing, and the smaller the area (pixel) to which the face processing is applied.

  FIG. 11 is a flowchart illustrating an example of the operation of the imaging apparatus 3. Specifically, the flowchart of FIG. 11 is a flowchart for explaining an operation of performing face processing on the main image captured while capturing a through image. Since steps S200 to S240 shown in FIG. 11 are the same as steps S100 to S140 shown in FIG. 8, only steps S250 to S290 will be described.

  The uncorresponding face estimation area selection unit 52 extracts a face estimation area based on the face area information stored in the face area information storage unit 10 (step S250). The unsupported face estimation region selection unit 52 that has extracted the face estimation region selects an unsupported face estimation region that does not correspond to the face region detected from the main image from the face estimation regions (step S255). The unsupported face estimation region selection unit 52 that has selected the unsupported face estimation region supplies the face region information (face region information detected from the main image) acquired from the face region detection unit 40 to the image processing unit 90, and Face area information indicating the corresponding face estimation area is supplied to the detection failure probability calculation unit 62.

  The detection failure probability calculation unit 62 that has acquired the face area information indicating the unsupported face estimation area calculates the detection failure probability (step S260). The detection failure probability calculation unit 62 that has calculated the detection failure probability includes the face region information indicating the unsupported face estimation region acquired from the unsupported face estimation region selection unit 52 and the detection failure probability as the unsupported face estimation region enlargement / reduction unit 70. And the detection failure probability is supplied to the parameter setting unit 80.

  The unsupported face estimation region enlargement / reduction unit 70 that has acquired the face region information indicating the unsupported face estimation region and the detection failure probability expands / contracts the unsupported face estimation region by the enlargement / reduction ratio according to the detection failure probability (step S270). The unsupported face estimation region enlargement / reduction unit 70 that has expanded and reduced the unsupported face estimation region supplies the image processing unit 92 with face region information indicating the unsupported face estimation region after the enlargement / reduction.

  The parameter setting unit 80 that has acquired the detection failure probability sets a parameter indicating the strength when image processing is performed on the unsupported face estimation region (step S280). The parameter setting unit 80 that has set the parameters supplies the set parameters to the image processing unit 92.

  The image processing unit 92 that has acquired the face region information indicating the uncorresponding face estimation region has the face region indicated by the face region information (face region information detected from the main image) with respect to the main image acquired from the imaging unit 20. Then, face processing is performed on the unsupported face estimation area (the unsupported face estimation area is subjected to face processing according to the intensity of the parameter) and stored in the image storage unit 12 (step S290). And this flowchart is complete | finished.

  As described above, according to the present embodiment, the image processing is performed on the unsupported face estimation area without detecting the face area after the main image is captured, so that the time until the face processing is performed can be shortened.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  1, 2 and 3 Imaging device 10 Face region information storage unit 12 Image storage unit 20 Imaging unit 30 Display control unit 40 Face region detection unit 50, 52 Unsupported face estimation region selection unit 60, 62 Detection failure probability calculation unit 70 Unsupported Face estimation area enlargement / reduction unit 80 Parameter setting unit 90, 92 Image processing unit

Claims (7)

An imaging unit that captures a subject image and generates an image;
A face area detection unit that detects a human face area from an image captured by the imaging unit;
A storage unit that stores area information indicating a face area detected by the face area detection unit from a through image captured by the imaging unit;
When the main image is picked up by the image pickup unit, it is detected at the time of picking up the main image from the face estimation regions that are estimated to exist in the main image based on the region information stored in the storage unit. A non-corresponding face estimation region selection unit that selects an uncorresponding face estimation region that does not correspond to the face region that has been made,
An image processing apparatus comprising: an image processing unit that performs image processing on the unsupported face estimation region selected by the unsupported face estimation region selection unit. The imaging device according to claim 1,
A detection failure probability calculation unit that calculates a detection failure probability that is a probability that the face region detection unit failed to detect a face region in the unsupported face estimation region;
An imaging apparatus, further comprising: an unsupported face estimation area enlargement / reduction unit that enlarges / reduces the unsupported face estimation area according to an enlargement / reduction ratio corresponding to the detection failure probability. The imaging device according to claim 1,
A detection failure probability calculation unit that calculates a detection failure probability that is a probability that the face region detection unit failed to detect a face region in the unsupported face estimation region;
A parameter setting unit that sets a parameter indicating an intensity when image processing is performed on the unsupported face estimation region;
The parameter setting unit
The higher the detection failure probability is, the more the parameter indicating the weaker intensity is set,
The image processing unit
An image pickup apparatus that performs image processing on an unsupported face estimation area according to the intensity of the parameter. In the imaging device according to any one of claims 2 and 3,
The detection failure probability calculation unit
A through area in which a face area corresponding to the unsupported face estimation area was not detected among a plurality of through images captured within a certain period of time before imaging of the main image with reference to the area information stored in the storage unit An imaging apparatus, wherein a ratio of images is calculated as the detection failure probability. The imaging device according to claim 1,
The unsupported face estimation region selection unit
An imaging apparatus, wherein when a change in position of a face area in a plurality of through images is smaller than a predetermined threshold, a face area detected from the latest through image is set as the face estimation area. The imaging device according to claim 1,
The unsupported face estimation region selection unit
When the position of the face area in a plurality of through images is moving in a certain direction, an area obtained by enlarging the face area detected from the latest through image in a certain direction is set as the face estimation area. Imaging device. The imaging device according to claim 1,
The unsupported face estimation region selection unit
An imaging apparatus characterized in that when the positions of face areas in a plurality of through images are moving in a certain direction, an area including all the face areas in the plurality of through images is set as the face estimation area.

Images