JP4894278B2 - Camera apparatus and camera control program - Google Patents

Description

  The present invention relates to a camera device having a subject counting function, and a camera control program for giving a subject counting function to the camera device.

Conventionally, a camera apparatus having a function of recognizing a subject has been proposed. The camera device having this function determines whether or not the main subject is a person, and performs different exposure settings when it is determined that the main subject is a person and when it is determined that the main subject is not a person (Patent Literature). 1).
JP 05-041830 A

  By the way, when taking a group photo, it is a common scene that a photographer or the like visually confirms the number of gatherers in order to confirm whether or not everyone is gathering. At this time, if a function for counting the number of persons based on the subject determined to be a person is added to the camera device, a complicated pre-shooting operation in which a photographer or the like visually confirms the number of persons becomes unnecessary, and convenience is obtained. However, a group photo is generally taken at a sightseeing spot, and there are unavoidable people around the gathering. Therefore, the number of desired subjects such as the number of persons to be gathered cannot be properly counted even if a function of counting the number of people based on the formed subject image is added.

  The present invention has been made in view of such conventional problems, and an object of the present invention is to provide a camera device and a camera control program capable of appropriately counting the number of desired subjects.

  The invention described in claim 1
  Imaging means for imaging a subject;
  Display means for displaying an image formed by the imaging means;
  A selection means for arbitrarily selecting a plurality of subjects to be counted from the image displayed on the display means;
  Counting means for counting, for each subject, the number of image portions whose similarity with feature data extracted from each subject is greater than or equal to a predetermined value in the image displayed on the display means;
  Display control means for simultaneously displaying the number counted for each subject by the counting means on the display means while distinguishing for each subject;
  It is characterized by providing.
  The invention according to claim 2 further includes
  The display control means displays a sample image of each subject and the number of each subject in association with each other on the display means for each of the plurality of subjects selected by the selection means.
  The invention according to claim 3 further includes
  The display control means associates the sample image of each subject with the number of each subject in a state where the image portion corresponding to each subject counted by the counting means is identified and displayed in the image displayed on the display means. It is characterized by being displayed at the same time.
  The invention according to claim 4 further includes
  The display control means performs painting with different shapes corresponding to each of a plurality of subjects to identify and display an image portion corresponding to each subject.
  The invention according to claim 5 further includes:
  A cursor whose position is arbitrarily displaced in accordance with the operation is displayed on the display means, and further includes an area designating means for designating an arbitrary area whose diagonals are two points designated by the cursor,
  The counting means counts the number of image portions whose similarity with the feature data extracted from each subject is greater than or equal to a predetermined value in the area specified by the area specifying means.
  The invention described in claim 6 further includes:
  A cursor whose position is arbitrarily displaced according to the operation is displayed on the display means, and further includes a position designation means for designating an arbitrary position indicated by the cursor,
  The selection means selects a subject at a position designated by the position designation means.
  The invention according to claim 7 further includes
  The position specifying means displays a cursor whose position and size are arbitrarily displaced according to an operation on the display means, and specifies an arbitrary position and size indicated by the cursor,
  The selection unit selects a subject from an area corresponding to the position and size designated by the position designation unit.
  The invention described in claim 8 further includes:
  The image pickup apparatus further includes shooting control means for controlling a shooting operation of the camera device based on the number of the image portions counted by the counting means.
  The invention according to claim 9 further includes
  Storage means for storing shooting conditions corresponding to each of a plurality of shooting scenes;
  Selection means for selecting one of the plurality of shooting scenes based on the number of the image portions counted by the counting means;
  The image processing apparatus further includes shooting control means for controlling the shooting operation of the camera device based on the shooting conditions stored in the storage means corresponding to the shooting scene selected by the selection means.
  The invention described in claim 10 further includes:
  An instruction selecting means for selecting any of the plurality of shooting scenes based on a photographer's instruction operation;
  When the shooting scene is selected by the instruction selection unit, the shooting control unit controls the shooting operation of the camera device based on the shooting condition stored in the storage unit corresponding to the shooting scene. It is characterized by doing.
  The invention described in claim 11 further includes:
  The imaging control unit controls at least one of a focusing operation, a depth of field, an exposure condition, and a filtering process of the camera device.
  The invention according to claim 12
  A computer having a camera device including an image forming means for forming an image of a subject and a display means for displaying an image formed by the image forming means;
  A selection means for arbitrarily selecting a plurality of subjects to be counted from the image displayed on the display means;
  Counting means for counting, for each subject, the number of image portions whose similarity with feature data extracted from each subject is greater than or equal to a predetermined value in the image displayed on the display means;
  Display control means for causing the display means to display the number counted for each subject by the counting means for each subject;
  It is characterized by functioning.

As described above, according to the present invention, a plurality of subjects to be counted are arbitrarily selected from within an image captured and displayed, and the number counted for each selected subject is determined as each subject. By distinguishing them and displaying them simultaneously on the display means, it is possible to simultaneously display the number of each of a plurality of arbitrarily selected subjects for comparison and confirmation .

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

(First embodiment)
1A is a front view of a digital camera 1 common to each embodiment, FIG. 1B is a rear view, and FIG. 1C is a side perspective view. The main body 2 of the digital camera 1 is provided with a release button (shutter switch) 3 and a power switch 4 having a half-press function on the upper surface portion, and a grip portion 5, a strobe 6 and an image pickup device on the front portion. A light receiving window 7 of the lens unit is arranged. Further, on the back side, a mode changeover switch 8, a zoom operation key 9, a cursor key 10 having a left / right / up / down operation unit, an enter / OK key 11, a DISP key 12, a menu key 13 and an LCD functioning also as an electronic viewfinder are provided. A display unit 14 is disposed, and a memory medium / battery storage unit 15 for storing a memory medium and a battery is provided. In addition, an image pickup device 20 including a rotary mirror 18, a lens group 19, a CCD, and the like are disposed.

  FIG. 2 is a block diagram illustrating a schematic circuit configuration of the digital camera 1. The digital camera 1 includes a main control unit 100, an imaging control unit 101, and an image processing unit 102. The main control unit 100 includes an extraction region selection unit 103, a template setting unit 104, a feature data template memory 105, and a shooting scene setting unit 106. The extraction region selection unit 103 includes a cursor operation from the cursor operation unit 107. The information is input, and the mode selection operation information from the mode selection operation unit 108 is input to the shooting scene setting unit 106. The shooting control unit 101 receives setting information from the shooting scene selection unit 103, recognition information from a recognition processing block described later, photometry information from the photometry unit 109, and distance measurement information from the range finding unit 110. The imaging control unit 101 controls the illumination unit 111, the optical system driving unit 112, the aperture / shutter driving unit 113, the imaging driving unit 114, and the signal processing unit 115 based on the input information.

  On the other hand, an aperture 117, a shutter 118, and an imaging unit 119 are disposed on the optical axis of the photographing optical system 116. The photographing optical system 116 is driven by the optical system driving unit 112, the diaphragm 117 and the shutter 118 are driven by the diaphragm / shutter driving unit 113, and the imaging unit 119 is driven by the imaging driving unit 114, respectively. The signal processing unit 115 processes the analog signal from the imaging unit 119 and converts it into a digital signal, and this digital image signal is input to the image signal processing unit 102 via the image buffer memory 120.

  The image processing unit 102 includes a preprocessing block 121, a feature extraction block 122, and a recognition processing block 123. In this image processing means 102, image data from the image buffer memory 120 that does not pass through the blocks 121 to 123 is input to the image compression / encoding means 124, while image data that passes through the blocks 121 to 123 is The data is input to the image compression / encoding unit 124 and the photographing control unit 101. The image compression / encoding unit 124 compresses and encodes these image data, and the processed image data is recorded in the image recording unit 125. The information from the feature extraction block 122 is configured to be input and stored in the template memory 105 of the feature data.

  FIG. 3 is a block diagram showing a specific circuit configuration of the digital camera 1. In the figure, the operation unit 23 is composed of the switches and key groups shown in FIG. 1 such as the release button 3 and the power switch 4, and the operation information of the switches and key groups is controlled via the input circuit 24. Input to the unit 25. The control unit 25 is a microcomputer including a CPU and its peripheral circuits, a RAM that is a working memory of the CPU, and the like, and controls each unit.

  The control unit 25 includes a display memory 26, a display drive block 27, an image buffer memory 28, an image signal processing unit 29, a compression encoding / decompression decoding unit 30, a still image / moving image memory 31, a program memory 32, data A memory 33, a memory IF 34, an external I / O interface 35, a communication control block 36, a power supply control block 37, and a photographing control unit 38 are connected. The display memory 26 temporarily stores various display data displayed on the display unit 14. The display drive block 27 drives the display unit 14, and the image buffer memory 28 temporarily stores the image data when processing it.

  The image signal processing unit 29 is a DSP that executes various processes on an image signal taken in by the control unit 25 from an image sensor described later. The compression encoding / decompression decoding unit 30 decompresses the image data processed by the image signal processing unit at the time of recording, and decompresses and decodes the recorded image data at the time of reproduction. The still image / moving image memory 31 records and saves image data (still image data) captured by operating the release button 3. The program memory 32 includes a control program of the control unit 25 shown in a flowchart to be described later, a shooting control program for each shooting scene such as “person shooting mode” and “night scene shooting mode”, target subject type information for each shooting scene, the target Feature amount data corresponding to the type of subject is stored.

  The data memory 33 stores various data in advance and stores data other than the image data, and also functions as a template memory shown in a flowchart described later. The memory IF 34 is connected to a removable external memory medium 39. The external I / O interface 35 is connected to the USB connector 40, the communication control block 36 is connected to the antenna 42 via the wireless LAN transmission / reception unit 41, and the battery 43 is connected to the power control block 37. The electric power from the battery 43 is supplied to each unit via the power control block 37 and the control unit 25.

  The photographing control unit 38 is connected to each irradiation drive unit 44 for driving the irradiation angle of the strobe 6 and a strobe illumination driving unit 45 for driving the irradiation. The light receiving angle driving unit 47 to be driven, the color temperature detecting unit 48 for detecting and outputting the color temperature from the photometric / ranging sensor 46, the photometric unit 49 for detecting and outputting the photometric data, and the distance measuring data are detected and output. A distance measuring unit 50 is connected. Furthermore, the first and second angular velocity sensors 17 and 17 are connected to the photographing control unit 38 via angular velocity detectors 51 and 52 and integrators 53 and 54, respectively.

  On the other hand, the zoom lens unit 55 is provided with the rotary mirror 18, the lens group 19, and the imaging device 20, and a drive mechanism 56 that rotationally drives the rotary mirror 18 and the lens group 19. An inserted diaphragm 57 is provided, and a shutter 58 is disposed in front of the image sensor 20.

  Further, the photographing control unit 38 includes an electric mirror Y direction driving unit 59, an electric mirror X direction driving unit 60, a focus lens driving unit 61, a zoom lens driving unit 62, an aperture driving unit 63, a shutter driving unit 64, and a video signal. A processing unit 65 and a timing control & driver 66 are connected. The electric mirror Y direction drive unit 59 drives the drive mechanism 56 to operate the rotary mirror 18 in the vertical direction, and the electric mirror X direction drive unit 60 operates in the left and right direction. The focus lens driving unit 61 drives the focus lens in the lens group 19, and the zoom lens driving unit 62 zooms in the lens group 19 to enlarge or reduce the subject image in accordance with the operation of the zoom operation key 9. The lens is driven. The aperture driving unit 63 drives the aperture 57, and the shutter driving unit 64 drives the shutter 58. The video signal processor 65 is supplied with an A / D circuit that converts an analog signal from the image sensor 20 into a digital signal, a CDS that holds the digital image signal from the A / D circuit, and an image signal from the CDS. It consists of a gain adjustment amplifier (AGC), which is an analog amplifier.

  FIG. 4 is a flowchart showing the processing procedure of the present embodiment, and FIG. 5 is an explanatory diagram corresponding to this flowchart. The control unit 25 executes processing according to the flowchart shown in FIG. 4 based on the program stored in the program memory 32 in the state where the image recognition photographing mode is set. First, it is determined whether or not a scene-specific shooting mode is set by an operation of the operation unit 23 by the user (step S101). If the scene-specific shooting mode is not set, other mode processing is executed (step S102). If the scene-by-scene shooting mode is set, preset target subject type information is read from the memory (program memory 32) in accordance with the selected shooting scene (step S102). Further, feature data corresponding to the type of the subject is read from the memory (program memory 32) and stored in the template memory (data memory 33) (step S103).

  On the other hand, if the shooting mode for each scene is not set, it is determined whether or not the shooting mode for the custom setting scene is set (step S104). If it is not set, the process proceeds to other processing. If the shooting mode of the custom setting scene is set, a sample extraction region is selected with a cursor or the like from within the shooting through image field of view (step S105). That is, in the custom setting scene shooting mode state, the subject image formed on the image sensor 20 by the lens group 19 is displayed on the display unit 14 as a through image P1, as shown in FIG. ing. In the shooting mode of the custom setting scene, the cursor-shaped cursor C1 whose position and size can be changed is displayed on the through image P1 by operating the cursor key 10. When the position and size of the cursor C1 are adjusted by operating the cursor key 10 and determined by operating the enter / OK key 11, the area surrounded by the cursor C1 is selected as the sample extraction area. Therefore, in the example of FIG. 5A, the fruit (mikan) region surrounded by the cursor C1 is selected as the sample extraction region.

  Next, features are extracted from the block image in the selected region by feature extraction processing (step S106), and the extracted feature data is stored in the template memory (step S107). Therefore, as a result of the processing in step S107, as shown in FIG. 5B, the template memory includes the feature amount data D of the sample image P2, as hue = HHH, saturation = SSS, brightness = VVV, contour shape = fff, size = LLL will be stored.

  Then, in step S108 following step S103 or step S107, the through image currently displayed on the display unit 14 is read. Next, a region to be identified by two diagonal points is selected from the captured subject image by a cursor operation or the like (step S109). That is, when executing the process of selecting the region to be identified, as shown in FIG. 5C, the cross-shaped first cursor C2 different from the cursor C1 is displayed on the through image P1 on the display unit 14. Display. Then, when the position of the first cursor C2 is adjusted by operating the cursor key 10 and determined by operating the enter / OK key 11, the position of the first cursor C2 is determined. Next, when the cursor key 10 is operated, as shown in FIG. 5D, the second cursor C3 separated from the first cursor C2 is displayed, and the position of the second cursor C3 is moved by operating the cursor key 10. When the adjustment is performed and the determination / OK key 11 is operated, the position of the second cursor C3 is determined. Then, a rectangular area whose diagonal is the first cursor C2 and the second cursor C3 is determined, and this rectangular area is set as an identification area F1 that is an area to be identified. At this time, the sample image P2 is displayed at the corner of the display unit 14.

  Next, an extraction area is detected from the subject image in the identification area F1 (step S110). As will be described later, this extraction area is detected by dividing the area into brightness or color difference signals from the brightness signal and color difference signal of the image data of the through image, for example, by hue of similar colors, and the boundary lines of the areas. A contour line is extracted, and a portion surrounded by the contour line is detected as one extraction region. Therefore, as shown in FIG. 5 (D), if there are four mandarin oranges m and one apple a in the identification area F1, the four shaded yellow areas surrounded by the outlines. A total of five extraction regions, i.e., the mandarin orange m region and one apple a region that is a yellow-red hue region and surrounded by a contour line, are detected. Subsequently, the detected extraction areas are sequentially selected (step S111), and the feature extraction process of the through image in the selected extraction area is executed (step S112). That is, the feature amount of the feature type included in the feature amount data D is extracted in the selected extraction region. Therefore, in the present example, the feature amount data D is hue, saturation, brightness, contour shape, and size. Therefore, the feature amount of these hue, saturation, brightness, contour shape, and size is included in the extraction region. To extract.

  Then, the feature amount extracted in step S112 is compared with the hue = HHH, saturation = SSS, brightness = VVV, contour shape = fff, size = LLL of the feature amount data D stored in the template memory. The similarity is calculated (step S113). That is, the ratio between each value of the feature value data D and each value of the extracted feature value is calculated. Next, it is determined whether or not the degree of similarity, which is the calculated ratio, is greater than or equal to a predetermined value (step S114). If it is less than the predetermined value, the process proceeds to step S117 without performing steps S115 and S116. move on. If the similarity is greater than or equal to a predetermined value, the current selected area is stored in the data memory 33 as an existing area of a similar subject (step S115). In addition, the value of the counter that counts the identification number of similar subjects is counted up (step S116). At this time, if the selected extraction region is mandarin orange m, the similarity is greater than or equal to a predetermined value, so the process of step S116 is executed and the counter value is counted up. However, if the selected extraction region is apple a, the degree of similarity is less than a predetermined value, so that the process of step S116 is not executed and the value of the counter is not counted up.

  Then, it is determined whether or not the processing in steps S111 to S116 has been executed up to the last extraction region (step S117), and the processing from step S111 is repeated until the last extraction region is reached. Therefore, in this example, the process from step S111 is repeated five times, and if the last extracted area is obtained by repeating the process five times, the identified similar subject area and the count number are displayed. (Step S117). At this time, in the present example, as described above, there are four mandarin oranges m and one apple a in the identification area F1, so that the processing in this step S117 is shown in FIG. As described above, the display unit 14 displays the similar subject area F2 and the count number N (Count = 4).

  Next, a photographing process is executed according to the similar subject area or the number of counts (step S118). That is, focus control such as automatic focusing (AF) processing is performed so that the identified subject is in focus, or the zoom magnification and the lens focal length are adjusted so that the identified subject has a full angle of view. Angle of view control is performed, or automatic scene selection processing such as automatic selection of a shooting scene-specific program according to the identified subject is performed. Therefore, after the process of step S118 is executed, the user operates the release button 3 to capture an image according to the process of step S118.

  In the present embodiment, the count number N is displayed on the display unit 14, but the count number N may be notified by voice without being displayed.

FIG. 6 is a diagram illustrating an operation example when a human face is extracted and photographed as a specific example. In this example,
1) Human face feature data is previously stored as a template in a memory.
1 ′) Alternatively, as shown in the figure, a subject image that is a sample of a human face that is a subject to be identified is selected by selecting a region from the through image, and a subject image in the selected region is selected. And register as a sample of the subject to be identified.

When the “person mode” is selected, the stored human face feature data is recalled, and the template feature data matches the feature amount of the subject in the designated image area. A template matching operation for searching whether or not there is a subject to be performed is performed. That is,
2) First, a region to be identified from the through image of the subject image is selected by a cursor operation or the like.
3) Next, from the luminance signal and color difference signal of the captured image data, the region is divided into similar luminance values or color difference signals, for example, according to similar hues, and a contour line that is a boundary line of the region is extracted. .
4) For example, only a region having a hue close to the hue of the template image is extracted from each divided region (extraction region).
5) Furthermore, binarize the image of the area, and if necessary, perform expansion and reduction processing to reduce fine irregularities, etc., and compare the shape and size of the obtained contour with the feature amount of the template If the similarity is higher than a predetermined value, the area is recognized as a human face to be recognized and the number is counted.
6) The count value of the identified subject is displayed, and the contour line of the identified area or the subject image filled in the superimposed area is displayed on the through image so that the identified area can be distinguished and displayed.

  FIG. 7 to FIG. 9 are diagrams showing examples of other subject recognition and counting processing operations. FIG. 7 shows an example of counting wild birds. As shown in FIG. 7A, the cursor C1 is displayed on the through image P1, and the position / size of the cursor C1 is adjusted by the operation of the cursor key 10. When determined by operating the OK key 11, the area surrounded by the cursor C1 is selected as the sample extraction area. Features are extracted from the block image in the selected region by feature extraction processing, and the extracted feature data is stored in the template memory.

  Further, as shown in (B), a cursor C2 different from the cursor C1 is displayed on the through image P2 on the display unit 14. Then, the position of the cursor C2 is determined by adjusting the position of the cursor C2 by operating the cursor key 10 and determining by the operation of the OK / OK key 11. The rectangular area surrounded by the cursor C2 is defined as an identification area F1 that is an area to be identified. At this time, the sample image P2 is displayed at the corner of the display unit 14. Then, as described above, the number of identifications of similar subjects similar to the sample image P2 is counted by detecting an extraction region from the subject image in the identification region F1, and the count number N (Count = 10) which is the number of wild birds. ) Is displayed.

  FIG. 8 shows an example in which the number of customers is counted. As shown in FIG. 8A, a sample image P2 stored in advance in the template memory is displayed on the display unit 14, and as shown in FIG. 8B, the cursor C2 is moved to the through image P1. Display above. Then, the position of the cursor C2 is determined by adjusting the position of the cursor C2 by operating the cursor key 10 and determining by the operation of the OK / OK key 11. The rectangular area surrounded by the cursor C2 is defined as an identification area F1 that is an area to be identified. Then, as described above, the number of identifications of similar subjects similar to the sample image P2 is counted by detecting an extraction region from the subject image in the identification region F1, and the count number N (Count = 2570) which is the number of spectators. ) Is displayed.

  FIG. 9 is an example of counting displayed products for each type. As shown in FIG. 9A, after the through image P1 is displayed on the display unit 14, a sample stored in the template memory in advance as shown in FIG. 9B. The images A and B are displayed, and the cursor C2 is displayed on the through image P1. Then, the position of the cursor C2 is determined by adjusting the position of the cursor C2 by operating the cursor key 10 and determining by the operation of the OK / OK key 11. The rectangular area surrounded by the cursor C2 is defined as an identification area F1 that is an area to be identified. Then, the number of identifications of similar subjects similar to the sample images A and B is counted by detecting an extraction region from the subject image in the identification region F1, and the count number N (Count A = 03) for each type of display product. (Count b = 04) is displayed. Of course, it is possible to display the number of counts for each type not only for the display product shown in (A) but also for the display product shown in (C).

  FIG. 10 is a diagram showing a display example of the recognition processing and recognition subject setting menu in the present embodiment. In response to the operation of the menu key 13 (MENU), the setting menu for the image recognition process shown in FIG. In this state, when “image recognition” is selected by operating the up / down operation unit of the cursor key 10 and the enter / OK key 11 is operated, the image recognition photographing mode is set. Further, when the right operation portion of the cursor key 10 is operated in the display state of (a), the state shifts to the display state of (a ′), “OFF (not recognized)”, “Auto (by shooting scene)”.・ ・ Displays the selection menu in the image recognition mode. In addition, when “recognized subject setting” is selected on the menu screen shown in FIG. 5B (the same screen as FIG. 1A), the screen shifts to the recognized subject setting menu screen shown in FIG.

FIG. 11 is a flowchart showing details of the feature extraction process executed in step S106 in the flowchart of FIG. First, in step S105, target image data in the sample extraction area selected by the cursor or the like is captured (step S201). Next, pre-processing (1) is executed, image enhancement processing, sharpening processing, noise removal processing, etc. are performed (step S202), pre-processing (2) is executed, binarization processing, or Normalization processing, rotation processing, coordinate conversion processing, and the like are performed (step S203). Further, the feature extraction process (1) is executed to perform filter processing, contour extraction, region extraction, thinning, expansion / contraction processing, etc. (step S204), and the feature extraction process (2) is executed to increase the size. Then, calculation of circumference, area, etc., or circularity, calculation of Fourier descriptor, calculation of contour shape evaluation value, etc. are performed.
(Sharpening, contour extraction)
FIGS. 12 and 13 show an image obtained by a primary differential filter process or a secondary differential filter process as an example of the sharpening process in the pre-process (1) in step S202 and the contour extraction process in the feature extraction process (1) in step S204. It is a figure which shows the example of this sharpening process and edge (contour) extraction process. As shown in FIG. 12, an image f (i, j) in which the edge of the portion where the gradation changes is blurred
Δx _f = f (i + j, j) −j (i−1, j),
Δy _f = f (i + j, j) −j (i−1, j),
g (i, j) = √ {(Δx _f ) ² + (Δyf) ² },
Or g (i, j) = | Δxf | + | Δyf |
When the primary part and the gradient (Gradient) are obtained by the calculation such as the above, it is possible to extract the gradient part and the contour whose gradation changes.

Or
∇ ² f (i, j) = ∂ ² f / ∂x ² + ∂ ² f / ∂y ² , or
∇ ² f (i, j) = f (i + 1, j) + f (i−1, j) + f (i, j−1) + f (i, j−1) −4f (i, j)
By performing a second-order differential (Laplacian) process that is further differentiated by operations such as the above, and subtracting this result from the original image data, an image that emphasizes the high-frequency component of the edge portion can be synthesized, and blurred edges and contours can be emphasized Can do.

  In order to perform edge and contour enhancement processing by software processing, first-order differential spatial filter operators (operators) such as “Prewitt filter”, “Sobei filter”, “Kirsch filter”, and “Roberts filter” shown in FIG. Alternatively, a second-order differential spatial filter operator such as “Laplacant filter” or the like can be used. As illustrated, the input image 220 is converted into an output image 222 by an operation 221. The input image 223 on the A-A ′ line is converted into an output image 224 on the A-A ′ line by the Prewitt filter (first-order differential filter) 224, and the A- It is converted into an output image 227 on the A ′ line, and is converted into an output image 229 on the AA ′ line by a Laplacent filter (secondary differential filter) 228.

  Such filter processing can be used for image sharpening processing in the above-described preprocessing, contour extraction or edge extraction in feature extraction processing, and the like. To use the contour shape, outline pattern, area, etc. of the subject image as feature data, perform brightness enhancement of the image data that you want to extract the features for each pixel by performing contour enhancement, edge detection filter operation, etc. Then, after converting the image into an extracted image, feature data and the like are extracted.

(Binarization, averaging, luminance conversion)
14 and 15 show the luminance as an example of the image enhancement processing in the preprocessing (1) in step S202, the binarization processing in the preprocessing (2) in step S203, and the feature extraction processing (1) in step S204. It is a figure which shows the example of this extraction process. As shown in the linear luminance conversion (improvement of intermediate gradation) in FIG. 14A, the input image 230 is converted into an output image 232 by an operation (conversion formula) 231. Further, the luminance histogram distribution P (x) 233 of the input image is converted into the luminance histogram distribution P (x) 235 of the output image by the luminance conversion equation 234. Further, as shown in the binarization process of (b), the luminance distribution P (x) 236 of the input image is converted into the luminance distribution P (x) 238 of the output image by the conversion equation 237. Further, as shown in the predetermined luminance extraction process of FIG. 15, the luminance distribution P (x) 239 of the input image is converted into the luminance distribution P (x) 241 of the output image by the conversion equation 240.

  As described above, when the distribution pattern of the image data (luminance value and color difference value) of the subject image is used as feature data, the luminance distribution (histogram) of the image data is obtained, and the luminance conversion process is performed to obtain the intermediate gradation. Can be emphasized and compressed. Further, binarization can be performed based on a predetermined threshold value, or only a region having a predetermined luminance can be extracted. Alternatively, luminance values and color difference values are averaged (mosaicized) for each number of pixels, and the patterns can be simplified and the information amount can be compressed.

(Conversion of image data)
Note that RGB signals, luminance signals Y, color difference signals Cb and Cr, or HSV (or HSB) data representing hue / saturation / lightness, etc. of image data can be easily converted to each other by the following conversion formula.
For example, to convert RGB data to YCbCr data,
Y = 0.299 * R + 0.587 * G + 0.114 * B,
Cb = 0.172 * R−0.339 * G + 0.511 * B + CENTER
Cr = 0.511 * R−0.428 * G−0.083 * B + CENTER
To convert YCbCr data to RGB data,
R = Y + 0.000 * (Cb-CENTER) + 1.371 * (Cr-CENTER)
G = Y−0.336 * (Cb−CENTER) −0.698 * (Cr−CENTER)
B = Y + 1.732 * (Cb-CENTER) + 0.000 * (Cr-CENTER)

To convert RGB data (each 0 to 1) to HSV (or HSB) data,
If cmax = maximum (R, GB) and cmin = minimum (R, GB),
Lightness V = cmax, saturation S = (cmax−cmin) / cmax (where c = 0, S = 0),
When R = cmax, hue H = 60 ° * {(GB) / (cmax-cmin)},
When G = cmax, hue H = 60 ° * {2+ (BR) / (cmax-cmin)},
When B = cmax, hue H = 60 ° * {4+ (RG) / (cmax−cmin)},
When H <0, 360 ° is added to H. When S = 0, H = 0.

(Hue extraction, skin color extraction)
FIG. 16 is an example of region extraction processing in the feature extraction processing (1) of step S204, and shows an example of human skin color region extraction as an example of extracting a predetermined color region. (A) is an example of spectral reflectance characteristics of human skin, and (b) is an example of RGB values and HSV values of the skin color region in the photographed image sample (hue: Hue of 0 to 360 °, chromatic Degree: Saturation is 0 to 255, Brightness: Value of Brightness is 0 to 255). (C) is an example of skin color hue: Hue: 0 to 360 °, saturation: Saturation: 0 to 1 distribution, and most of the skin color image data is in the range of 6 ° to 38 ° in the hue circle. Many distributions are known (Skin Color Analysis (by J. Sherrah and S. Gong) http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPLES/GONGI/cvOnline-skinColourAnalysis.html ). By using these, it is possible to appropriately extract a subject area of a specific color, such as a skin color of a human face having a hue in a range of about 6 ° to 38 ° from the HSV value.

(Expansion, contraction, thinning, line drawing)
FIG. 17 is a diagram showing an example of the expansion / contraction process in the feature extraction process (1) in step S204. In order to determine the shape and characteristics of the subject from the contour shape or the like, it is preferable to first determine after simplifying the shape and compressing the information amount into a line figure. For example, as shown in FIG. 1A, the 1-pixel is thickened to the outside by one layer (of 8 neighboring pixels) in the binarized image outline or the boundary area between 1-pixel and 0-pixel. When the so-called “expansion” process is performed, small holes and grooves in the boundary such as the contour are removed, and conversely, as shown in FIG. The contraction process removes protrusions and isolated points at the boundary, so the shape can be simplified by combining the expansion process and the contraction process.

  Further, a rough shape of a skeleton can be obtained by so-called “thinning” (Thinning) processing in which a center having a line width of 1 is extracted from a binarized image. Further, only the boundary line of the contour is obtained by “boundary line tracking” processing in which the periphery of the binarized contour pattern is traced along the boundary from the outside and the inside, the boundary pixel is set to 1, and the remaining pixels are set to 0. Can be converted to. Alternatively, a “polyline approximation” process or the like in which only pixels selected for every several pixels of the contour shape are connected by a polyline may be used.

(Shape identification 1)
Shape identification is performed by storing binarized or line figured template images and their feature values, and the degree of correlation and similarity between the binarized and line figured image and feature data of the input image. Can be calculated to identify the shape and determine the similarity. Alternatively, it is possible to search for similar images and graphics by a method such as template matching for sequentially searching the positions of images similar to the template image from the images in the region. Also, for simplicity, similarities are easily obtained from the binarized image of the subject image, the vertical and horizontal sizes of the contour figure, boundary line figure, etc., radius, perimeter length, number of pixels, area, ratio of geometric dimensions, etc. Degree determination and subject identification may be performed.

For example, as shown in FIG. 18, since the circumference of a circle is L = 2πr and the area S = πr ² , (circumference length L) ² / (area A) = 4π, so the circularity of the contour of the figure is reduced. ,
Circularity = (peripheral length L) ² / (area S), or
Calculate with circularity e = 4π (circularness) = 4πS / L ² ,
Whether the object has a round shape or a sharp object with many irregularities is calculated according to whether the circularity is close to 4π (= 12.57) or the circularity is close to 1.0, and the shape is identified. Available. Similarly, as shown in FIG. 19, an evaluation value such as “strip length” of a shape may be obtained from the aspect ratio of the contour = length h / width w.

(Shape identification 2. Declination function, Position coordinate function)
Further, as shown in FIG. 20, along the contour line of the figure, the deflection angle θ (s) is obtained sequentially from the start point, converted into a one-dimensional function (deflection angle function), and used as the feature amount of the contour shape. Available. Alternatively, as shown in FIG. 21, similarly, position coordinates x (s), y (s) or z (s) = x (s) + j · y (s) are sequentially obtained along the contour line. Thus, it can be converted into a one-dimensional position coordinate function and used as a feature amount of the contour shape.

(Shape identification 3. Fourier descriptor)
(Z descriptor)
Further, for example, as shown in FIG. 22, the above-mentioned declination function θ (s) is normalized, and the normalized declination function: θ _N (s) = θ (s) −θ (0) −2πs / L ,
And Fourier transform of the discretized data φ [i] of (i = 0, 1, 2,..., N−1) to obtain the following Z (Zahn) type Fourier descriptor, It can be used to identify contour shapes.
Discretized data of θ _N (s): φ [i] = θ [i] −θ [0] −2πi / N
(I = 0,1,2 ..., N-1)
Discrete Fourier transform of θ _N (s) (= Z-type descriptor): Cz [k] = (1 / N) Σφ [i] EXP (−j2πki / N).

(G type descriptor)
Similarly, as shown in FIG. 23, the above-described complex plane coordinate z (s) of the position coordinate may be subjected to discrete Fourier transform to obtain a G (Grundlund) type Fourier descriptor, which may be used for identifying the contour shape. .
Complex plane coordinates z (s) = x (s) + j · y [s] of position coordinates,
discretized data of z (s) z (i) = x [i] + j · y [0] (i = 0, 1, 2..., N−1),
Discrete Fourier transform of z (s) (= G descriptor): Cg [k] = (1 / N) Σz [i] EXP (−j2πki / N).

(P type descriptor)
Also, as shown in FIG. 24, an exponential function w [i] of the deflection angle θ [i] approximated by a polygonal line is obtained, a P (Phase) shape descriptor obtained by Fourier transforming w [i] is obtained, and the contour shape is obtained. It may be used for identification.
w [i] = exp (jθ [i]) = cosθ [i] + sinθ [i]
= (Z [i + 1] -z [i]) / δ,
However, the line segment δ = | z [i + 1] −z [i] |
Discrete Fourier transform of w [i] (= p-type descriptor): Cp [k] = (1 / N) Σw [i] exp (−j2πki / N).

(Comparison of features, search for similar figures)
In addition, a pattern matching method such as template matching or a block matching method in motion vector detection can be used to determine and search the similarity between the stored template image and the subject image. By template matching, for example, (m × n) recorded reference images (or feature data) t [k, l] are matched from the feature data of the input image f [i, j] in the feature extraction region. Detect the position of the image. Place the center (or end point) of the reference image so that it overlaps a certain point (i, j) of the input image, and perform raster scanning of the point (i, j) in order vertically and horizontally, and the similarity of the image data of the overlapping part By calculating sequentially, the position point (i, j) having the highest degree of similarity can be obtained as a position where there is a similar subject.

(Correlation degree (correlation coefficient))
As shown in FIG. 25, the correlation between the input image f [k, l] and the template image (reference image) t [k, l] is Pearson's correlation coefficient (product moment correlation coefficient) R The position where the correlation coefficient R is largest can be obtained as the position where there is a similar subject to be searched.
R = (covariance between image f and image t) / f (standard deviation of image f) · (standard deviation of image t)
= [ΣL _{= 0} ⁿ⁻¹ ΣK _{= 0} ^m−1 {f [k, l] −f _AV } {t [k, l] −t _AV }]
/ √ [Σ _{L = 0} ⁿ⁻¹ Σ _{K = 0} ^m−1 {f [k, l] −f _AV } ² ] · √ [Σ _{L = 0} ⁿ⁻¹ Σ _{K = 0} ^m−1 {t [ k, l] −t _AV } ² ]
However, f _AV : the average value of the image data (luminance value, color difference value, feature amount, etc.) of the reference image f [k, l],
t _AV : average value of image data (luminance value, color difference value, feature amount, etc.) of the reference image t [k, l].

(Similarity of template matching)
When searching by scanning an image t [k, l] having an image size (m × n) from the image f [i, j], such as template matching, the similarity can be calculated by the following equation. A point (i, j) at the scanning position where r (i, j) is the largest is obtained as a similar subject position.
R (i, j) = Σ _{L = 0} ⁿ⁻¹ ΣK _{= 0} ^m−1 f [i− (m / 2) + k, j− (n / 2) +1] · t [k, l]
However, (i, j): position coordinates of points, f [i, j]: input image data, t [k, l]: template image data, (m × n): size of template image.
Similar to the correlation coefficient R described above, this may be normalized by subtracting the average value.

(Absolute difference sum, distance)
Since the calculation for multiplication increases with the similarity, R (i, j), when the average value can be subtracted or normalization can be omitted, such as a binary image, instead of the similarity, D (i, j) representing the degree of difference (“distance”) is obtained from the difference sum between images as in the following equation, and this can be used as an evaluation function. In this case, the calculation can be speeded up because it can be calculated only by addition and subtraction. In this case, the point (i, j) having the smallest distance D (i, j) represents the matching position.
D (i, j) = Σ _{L = 0} ⁿ⁻¹ Σ _{K = 0} ^m−1 | f [i− (m / 2) + k, j− (n / 2) +1] −t [k, l] |
However, (i, j): position coordinates of points, f [i, j]: input image data, t [k, l]: template image data, (m × n): size of template image.

(Specific subject recognition processing, face recognition processing)
In the above recognition processing, there are cases where dedicated identification data or recognition processing is required for each subject to be noticed in a specific subject or scene-specific shooting mode. For example, an example of recognizing a human face in a “person” shooting scene, a “person and landscape” shooting scene, or the like will be described.

FIG. 26A shows an example of a mask pattern for extracting an eye area of a face. Using this as a reference pattern, search is performed using template matching or the like, and an image area having an eye or a face is searched from an input image. Searchable. (B) is an example of data for recognizing the eye. For example, when the eye length = b / a, the condition of α ₁ ≦ b / a ≦ α ₂ is satisfied, or (eye area) S ₁ ≈π × a × b, (area of black eye (pupil)) S ₂ = π × r ² , and ratio of black eyes (pupil) S ₂ / S ₁ = r ² / ab, β ₁ ≦ r ² / ab ≦ beta ₂ the condition region of a subject image matching the like can be identified as a "region of the eye." (C) is a setting of condition data for recognizing a human face. For example, (length from below the eyebrows to below the nose) h1≈ (length from below the nose to the chin) h2, or An area of the subject image that satisfies the conditions such as (right eye width) W ₁ ≈ (between eyes) W ₂ ≈ (left eye width) W ₃ can be identified as a “face area”.

FIG. 27 is a flowchart illustrating a processing procedure of human recognition processing 1 (medium distance, head recognition). First, an image is captured (step S301), and a contour is extracted based on luminance or color difference data (step S302). Next, the image is divided into regions having the extracted contour as a boundary (step S303), and any target region is selected from the divided regions (step S304). Then, the peripheral length L and area S of the contour line are calculated, and the circularity (circularness) is calculated by e = 4πS / L ² (step S305). It is determined whether the calculated e is “e1 ≦ e ≦ e2” (step S306). If this determination is NO, it is recognized that the region is not a human head region (step S307), and the process proceeds to step S312 described later.

  If the determination is YES, a face template is set as a template to be compared, and the similarity between the set face template and the selected image (target region) is calculated (step S308). Next, it is determined whether or not the calculated similarity is greater than or equal to a predetermined value (step S309). If it is less than the predetermined value, it is recognized that it is not a human head region (step S309). S307). If it is equal to or greater than the predetermined value, it is recognized as a human face region (step S310), and the position coordinates recognized as the head are stored (step S311). Next, it is determined whether or not the region selected this time is the last region in the image (step S312). If it is not the last region, the process returns to step S304 to select the next region and Repeat the process. When the above processing is executed up to the last area, the determination in step S312 is YES, and the stored position coordinates are output (step S313).

  FIG. 28 is a flowchart showing a processing procedure of human identification processing 2 (short distance, face recognition). First, an image is captured (step S401), and a contour is extracted based on luminance or color difference data (step S402). Next, the image is divided into regions having the extracted contour as a boundary (step S403), and any target region is selected from the divided regions (step S404). Then, based on the RGB or color difference data, the average RGB or average color difference data of the selected region is calculated (step S405), and the calculated RGB or color difference value is converted into HSV (step S406).

Subsequently, it is determined whether or not the converted HSV is a skin color region, that is, whether or not the hue (Hue) is 6 to 38 ° (step S407). If this determination is NO, the region is not a face region. Is determined (step S415). If the determination in step S407 is yes, a face mask pattern (see FIG. 26) is set, and eye and pupil regions are searched in the target region (step S408). Next, it is determined whether or not the eye region has been detected (step S409). If it cannot be detected, it is determined that the eye region is not a face region (step S415). If detected, the aspect ratio (b / a) of the eye and the area ratio (r ² / ab) of the eyes and pupils (black eyes) are calculated (step S410), and the ratio of the eyes and pupils is within a predetermined range. Whether or not the condition of α ₁ ≦ b / a ≦ α ₂ is satisfied (step S411). If this determination is NO, it is determined that the region is not a face region (step S415).

If the determination in step S411 is YES, it is determined whether or not the ratio between the eyes and the pupil is within a predetermined range, that is, whether or not the condition of β ₁ ≦ r ² / ab ≦ β ₂ is satisfied. (Step S412). If this determination is NO, it is determined that the region is not a face region (step S415). If the determination in step S412 is YES, the right eye width W ₁ , the right eye-left eye interval W ₂ , and the left eye width W ₃ are calculated (step S413), and W ₁ and W ₂ , W _It is determined whether ₃ is equal, that is, whether W ₁ −δ ≦ W ₂ ≦ W ₁ + δ and W ₁ −δ ≦ W ₃ ≦ W ₁ + δ (step S414). If this determination is NO, it is determined that the region is not a face region (step S415). If this determination is YES, that is, if all the determinations in steps S407, S409, S411, S412, and S414 are YES, the region selected in step S404 is recognized as a face region (step S416). Further, the position coordinates of the recognized face area are stored (step S417). Next, it is determined whether or not the region selected this time is the last region in the image (step S418). If it is not the last region, the process returns to step S404 to select the next region and Repeat the process. When the above processing is executed up to the last area, the determination in step S418 is YES, and the recognition result is output (step S419).

(Second Embodiment)
29 and 30 show a second embodiment of the present invention. In the present embodiment, reference image data and feature amount data of a specific subject are set corresponding to each shooting scene, and these scene-specific reference images and feature data are sequentially set in the template memory. The through image of the focused subject image is sequentially compared with the image and feature data of the set specific image, and if it is recognized that there is a matching subject, the subject of the scene and the number of recognized subjects The corresponding scene-specific shooting program is automatically selected according to the above.

  That is, in this embodiment, as shown in FIG. 29, in the program memory 32, when a person is photographed (1), when a plurality of persons are photographed as a landscape (2), when a flower is photographed (3)・ For each shooting scene, etc., a scene-specific shooting control program suitable for shooting the scene is stored, and the sample image 300 corresponding to the shooting scene and “color enhancement is set to skin color. ”And the like related to the scene-specific shooting control program. Further, a reference image (an image for comparison different from the sample image 300) or feature data of the reference image is stored corresponding to each shooting scene.

  That is, for example, an image of one person's face is stored as a reference image or feature amount data of the reference image is stored in correspondence with a shooting scene in which a plurality of persons in the shooting scene (2) are captured. Corresponding to a shooting scene in which a flower composed of a plurality of petals in the shooting scene (3) is captured, an image of one petal is stored as a reference image, or feature amount data of this reference image is stored. In addition, “No.” is attached to each feature amount data.

  FIG. 30 is a series of flowcharts showing the processing procedure of the present embodiment, and the control unit 25 executes processing according to the flowchart shown in FIG. First, it is determined whether or not the automatic selection function of the shooting scene is turned on by the operation of the operation input unit 35 by the user (Step S501). The process proceeds (step S502). If it is ON, a focus frame is selected (step S503). That is, in this embodiment, a plurality of focus frames are displayed on the display unit 14 together with the subject through image, and one of the plurality of focus frames is selected by an operation on the operation unit 23 by the user. Therefore, in this embodiment, the region to be identified in the through image is designated by selecting the focus frame.

  Next, the subject image (subject through image) displayed on the display unit 14 is read (step S504), and AF processing is executed on the subject around the focus frame to focus (step S505). Further, contour extraction is performed on the subject image in the focused frame peripheral region based on the luminance or color difference data (step S506), and the image is divided into a plurality of regions with the extracted contour as a boundary (step S507).

  Further, the reference image or feature data stored corresponding to the first shooting scene (shooting scene (1)) is read and set in the template memory (data memory 38) (step S508). Subsequently, the largest or central contour region is selected from within the focus region (within the focus frame) (step S509), and the above-described feature extraction processing is executed on the through image of the selected region (step S509). S510). Further, the feature amount extracted in step S510 is compared with the feature amount of the template (reference image or feature amount data stored in the template memory) to calculate the similarity (step S511). Then, it is determined whether or not the calculated similarity is greater than or equal to a predetermined value (step S512). If not, the process proceeds to step S515 without performing steps S513 and S514. However, if the similarity is equal to or greater than a predetermined value, the subject is recognized as a subject corresponding to the reference feature amount set in the template, and the feature amount No. And the position coordinates of the subject are stored (step S513), and the number of recognized subjects is counted (step S514).

  Further, the next divided area in the area divided in step S507 is selected (step S515), and it is determined whether or not all areas have been compared (step S516), and all areas have been compared. Until the process from step S510 is repeated. If all the areas have been compared, it is determined whether there is a recognized object or area, or whether the count value of the number of recognitions counted in step S514 is 1 or more (step S514). S517). If the result of this determination is that there is no recognized subject or area, or the count value of the recognized number of counts is 0, the reference image or feature quantity data stored corresponding to the next shooting scene Is set in the template memory (data memory 38) (step S518). Then, it is determined whether or not all the shooting scenes have been compared (step S519), and if not compared, the processing from step S509 described above is repeated. If all the shooting scenes have been compared, the fact that the automatic selection of the shooting scene has failed is displayed on the display unit 14 or the normal automatic shooting mode is selected (step S520), which will be described later. The process proceeds to step S523.

  On the other hand, as a result of the determination in step S517, if there is a recognized subject or area, or the count value of the counted number of recognition is 1 or more, the template recognized as matching and the recognized subject, Alternatively, a shooting scene is automatically selected according to the number of areas (step S521).

  That is, for example, as described above, the feature amount data of the reference image is stored as the face image of one person corresponding to the shooting scene in which a plurality of persons in the shooting scene (2) are shot, and in step S508, It is assumed that this feature amount data is set in the template memory. In this state, when the user holds the camera with three persons as subjects and is positioned within the focus frame, the number of recognitions “3” is counted in step S514 by repeating the processing of S510 to S516. . Also, since the photographic scene corresponding to the person “3” is the photographic scene (2), the case (2) where a plurality of persons are photographed is automatically selected. Then, the selected shooting scene is displayed on a part of the display unit 14, and shooting conditions and the like are automatically set according to the selected shooting scene. That is, as described above, the program memory 32 stores a shooting control program for each shooting scene that is suitable for shooting a plurality of persons (2). Set conditions automatically. Thereby, it is possible to perform shooting under appropriate shooting conditions according to the number of recognized subjects and the like.

(Third embodiment)
FIGS. 31 and 32 show a third embodiment of the present invention, in which shooting conditions are automatically set according to the number of subjects recognized as a person. FIG. 31 is a series of flowcharts showing the processing procedure of the present embodiment, and the control unit 25 executes processing according to the flowchart shown in FIG. 31 based on the program stored in the program memory 32. First, it is determined whether or not the person photographing mode is set by the operation of the operation unit 23 by the user (step S601). If not set, the process proceeds to other photographing mode processing (step S602). If the person shooting mode is set, the subject image displayed as a through image on the display unit 14 at this time is captured (step S603). Then, similarly to step S109 of FIG. 4 described above, a region to be identified by two diagonal points is selected from the captured subject image (step S604).

  Next, the extracted subject image is subjected to contour extraction processing and divided into a plurality of regions (step S605). That is, as described above, the region is divided into luminance or color difference signals from the luminance signal and color difference signal of the subject image according to similar luminance or color difference signals, for example, by hues of similar colors, and the contour line that becomes the boundary line of the region is further divided. The subject image is divided into a plurality of regions by extracting and defining a portion surrounded by the contour line as one contour region (extraction region). Subsequently, the reference image and feature data corresponding to the person stored in advance in the program memory 32 are read out and set in the template memory (data memory 33) (step S606).

  Of the plurality of contour regions divided in step S603, the largest region, the region closest to the subject, or the region near the center is first selected (step S607). Further, the subject in the selected area is measured, and AF processing is performed to focus on the subject (step S608), and the above-described feature extraction process is executed on the subject in the selected area (step S609). . The feature amount extracted in step S609 is compared with the feature amount of the template (reference image or feature amount data stored in the template memory) to calculate the similarity (step S610). Then, it is determined whether or not the calculated similarity is greater than or equal to a predetermined value (step S611). If not, the process proceeds to step S614 without performing the processes of steps S612 and S613. If the similarity is equal to or greater than a predetermined value, the person is recognized as a person, and the position coordinates of the subject in the through image and the distance information of the subject measured in step S608 are stored (step S612). Further, the value of the counter that counts the number of people is counted up, and the next area in the contour area divided in step S605 is selected (step S613). Then, it is determined whether or not all contour regions have been compared (step S614), and the processing from step S608 is repeated until all the regions have been compared.

  If all the areas have been compared, it is determined whether or not the recognition count (number of people) counted in step S613 is 1 or more (step S615), and if the recognition count is 0. Displays that the person cannot be recognized on the display unit 14 or selects another shooting mode (step S616).

On the other hand, if the result of determination in step S615 is that the recognition count number (number of people) is 1 or more, shooting conditions and the like are set according to the number of subjects recognized as this person (number of people) (step 615). S617). Specifically, the processes shown in the following steps S618 to S628 are executed. First, it is determined whether or not the recognition count number is 1 (step S618). If the recognition count number is 1, the range of the distance a to the front a [mm] and the rear b [mm] of the subject. The depth of field is set to (step S619). For example, the hyperfocal distance (HFD) is set to HFD = (L ² / a) −L or HFD = (L ² / b) + L.

Next, the following aperture value F corresponding to the focal length (f) and the set hyperfocal length (HFD) is set (step S620).
F = f ² / (δ · HFD)
(Δ: Allowable circle of confusion)
Further, the color enhancement is set to the skin color, and the sharpness is set to be slightly soft (step S621). Thereafter, shooting processing is executed, and in response to the user operating the release button 3, image data is captured from the image sensor 20, compressed and encoded by the compression encoding / decompression decoding unit 30, and a still image / moving image It records in the memory 31 (step S629).

  Therefore, when the recognition count number = 1, as shown in FIG. 23A, there is one person, and a still image shot under the shooting conditions set in steps S619 to S621 is taken. It is recorded in the still image / moving image memory 31. In other words, if the number (number of subjects) of the recognized subjects is one, it is determined that the person has taken a portrait, and only the subject is focused, and the background is slightly blurred so that the background is slightly blurred. Accordingly, the subject depth Z (or the hyperfocal distance HFD corresponding thereto) is set only in the range of about 10 cm before and after the person, and the aperture value (F value) is set according to the hyperfocal distance HFD and the lens focal length (f). And an exposure time (shutter speed) corresponding to a Tv value (= Ev−Av) that becomes an appropriate exposure value (Ev) based on the photometric value in conjunction with the Apex value (Av value) of the set aperture value. Set. For portrait photography of a person, color correction filter processing for enhancing skin color is added, and sharpness by the contour enhancement processing filter is set slightly soft (soft focus).

If the recognition count number ≠ 1 as a result of the determination in step S618, it is determined whether the recognition count number is 2 to 5 (step S622). When the recognition count number is 2 to 5, the depth of field is set so that the distance from the nearest person distance L1 to the farthest person distance L2 is in focus (step S623). For example, the hyperfocal distance (HFD) is set to HFD = (L·L1) / (L−L1) or HFD = (L·L2) / (L−L2).

Next, the following aperture value F corresponding to the set hyperfocal distance (HFD) is set (step S624).
F = f ² / (δ · HFD)
(Δ: Allowable circle of confusion)
Further, the color enhancement is set to skin color, and the sharpness is set to normal (step S625). Thereafter, the photographing process is executed (step S628). Therefore, when the recognition count number = 3, as shown in FIG. 32 (B), there are three persons and still images shot under the shooting conditions set in steps S623 to S625 are still images. Recorded in the image / moving image memory 31. In other words, when the number of recognized subjects (number of persons) is about 2 to 5, it is determined as a snapshot of a plurality of persons, and the focus is adjusted from the distance L1 of the nearest person to the distance L2 of the farthest person. As described above, the depth of field Z (or the corresponding hyperfocal distance HFD) and the aperture value (F value) are set, and the exposure time for proper exposure is set in conjunction with the above, and Then, color correction filter processing for emphasizing the skin color is added, and the sharpness by the contour enhancement processing filter is set to normal, and shooting is performed.

If the recognition count number is not 2 to 5 as a result of the determination in step S622, the recognition count number is 6 or more. If the recognition count is 6 or more, the depth of field is set so that the closest person is focused from the distance L1 to the point at infinity (step S626). For example, the hyperfocal distance (HFD) is set to HFD = L1.

Next, the following aperture value F corresponding to the set hyperfocal distance (HFD) is set (step S627).
F = f ² / (δ · HFD)
(Δ: Allowable circle of confusion)
Further, the color enhancement is set to the skin color, and the sharpness (outline enhancement) is set to be slightly hard (step S628). Thereafter, the photographing process is executed (step S629). Therefore, when the recognition count number is 6 or more, as shown in FIG. 32C, there are 16 people, for example, and still images shot under the shooting conditions set in steps S626 to S68. Is recorded in the still image / moving image memory 31. In other words, if the number of recognized subjects is 6 or more, it is determined that a commemorative photo is taken by a plurality of people, and the closest person is focused not only on the person but also on the background and scenery in the distance. In the same manner as described above, the depth of field Z (or the corresponding hyperfocal distance HFD) and the aperture value (F value) are set so that the distance from the distance L1 to infinity (∞) is in focus. Set the exposure time for proper exposure. Further, the sharpness by the edge enhancement processing filter is set to be slightly hard so that a person becomes smaller and a distant scenery can be clearly seen.

  In this way, depending on the distance and the number of recognized subjects, the shooting process such as the depth of field and aperture setting, and the selection of various filter processes is controlled, so that the situation of the subject can be obtained even in the same person shooting. Accordingly, automatic shooting closer to the user's intention can be performed. Similarly, in other shooting scenes, more detailed shooting conditions and image processing can be set in the shooting scene according to the shooting distance and the number of recognized subjects.

  In the present embodiment, the process of step S604 is executed, and the region to be identified at two diagonal points is selected from the captured subject image, as in step S109 of FIG. 4 described above. However, the entire region of the subject image may be set to be identified without performing step S604.

Here, the equation used in the flowchart of FIG. 31 will be described. The depth of field is expressed by the following equation from the focal length (f), aperture value (F), photographing distance (F), and the like of the photographing lens. Can be calculated.
Forward depth of field Tf = δFL ² / (f ² + δFL)
Back depth of field Tr = δFL ² / (f ² −δFL)
Depth of field Z = Tf + Tr = δFL ² f ² / (f ² −δFL)

Further, when converted into the distance from the imaging surface to the subject, the depth-of-field limit near point and far point are obtained.
Depth of field limit near point Lmin = L−Tf = f ² L / (f ² + δFL)
Depth of field limit far point Lmax = L + Tf = f ² L / (f ² −δFL)
Similarly, depth of field Z = Tf + Tr = Lmax−Lmin
Where f: focal length, F: aperture value (F value), L: photographing distance to the subject, δ: diameter of allowable circle of confusion.

Further, in the “hyperfocal distance” (HFD), which is an imaging distance (L) at which the depth of field far point (Lmax) becomes infinity (∞), the distance from ½ of HFD to infinity. It appears in focus by (∞). Similarly, from the relationship between the hyperfocal distance (HFD) and the shooting distance (L), the depth of field can be obtained as in the following equation.
Hyperfocal distance HFD = f ² + δF,
Depth of field limit near point Lmin = (hyperfocal distance HFD × shooting distance) ÷ (hyperfocal distance HFD + shooting distance L) = f ² L (f ² + δFL)
Depth of field limit far point Lmax = (hyperfocal distance HFD × shooting distance) ÷ (hyperfocal distance HFD−shooting distance L) = f ² L (f ² −δFL)
Depth of field Z = Lmax−Lmin = 2δFL ² f ² / (f ⁴ −δ ² F ² L ² )

Therefore, as in step S618, in order to set the shooting distance L to be in the range from the front a [mm] of the subject to the rear b [mm] of the subject,
Forward depth of field Tf = a, backward depth of field Tr = b, depth of field Z = Tf + Tr = a + b,
Alternatively, HFD = (L ² / a) −L or HFD = (L ² / b) + L may be set.

Further, as in step S622, in order to set the focus from the shooting distance L1 to the shooting distance L2,
Depth of field limit near point Lmin = L1, Depth of field limit far point Lmax = L2, Depth of field Z = Lmax−Lmin = L2−L1,
Alternatively, HFD = (L·L1) / (L−L1) or HFD = (L·L2) / (2−L) may be set.

Also, as in step S625, in order to set the focus from the shooting distance L1 to infinity (∞),
Hyperfocal distance HFD = f ² / δF = L1,
Should be set.

Therefore, the aperture value F = f ² / (δ · HFD) is set according to each set depth of field, or the corresponding hyperfocal length HFD and focal length (f),
Further, exposure value (Av) = subject luminance value (Ev) + sensitivity value (Sv) = aperture value (Av) + shutter speed value (Tv),
From the relationship of the aperture value (Av) = log ₂ (F) ² and the shutter speed value (Tv) log ₂ (1 / T), the shutter speed value (Tv) satisfying the appropriate exposure value Ev = Ev−Av = Ev−log. ₂ (F) ² , exposure time (T) = 1 / (2 to the power of Tv) may be set.

  In the embodiment, the present invention is applied to a digital camera. However, the present invention can be applied not only to a camera but also to various devices such as a mobile phone having an imaging function.

(A) is a front view of a digital camera common to each embodiment of the present invention, (B) is a rear view, and (C) is a side perspective view. FIG. 2 is a block diagram showing a schematic circuit configuration of the digital camera. 2 is a block diagram showing a specific circuit configuration of the digital camera. FIG. It is a flowchart which shows the process sequence in 1st Embodiment. It is a figure which shows the example of a display screen of the embodiment. It is a figure which shows the operation example in the case of extracting and photographing a person's face. It is a figure which shows the example which counts a wild bird. It is a figure which shows the example which counts the number of customers. It is a figure which shows the example which counts display goods for every classification. It is a figure which shows the example of a display of the recognition processing in this Embodiment, and the setting menu of a recognition subject. It is a flowchart which shows the detail of the feature extraction process performed by step S106 in the flowchart of FIG. It is a figure which shows the example of the sharpening process of the image by a primary differential filter or a secondary differential filter process, and an edge (contour) extraction process. It is a figure which shows the example of the sharpening process of the image by a primary differential filter or a secondary differential filter process, and an edge (contour) extraction process. It is a figure which shows the example of an expansion and contraction process. It is a figure which shows the extraction process of predetermined brightness | luminance. It is a figure which shows the example of extraction of a human skin color area | region as an example which extracts the area | region of a predetermined color. It is a figure which shows the example of the expansion / contraction process in the characteristic extraction process (1) of step S204. It is a figure which shows the example which extracts the feature of an outline shape. It is explanatory drawing which calculates | requires evaluation values, such as a shape "thin length", from the aspect ratio = length h / width w of an outline. It is a figure which shows the example of the declination function of a contour shape, and a position coordinate function. It is a figure which shows the example of the declination function of a contour shape, and a position coordinate function. It is a figure which shows the example of the Fourier descriptor of an outline shape. It is a figure which shows the example of a G-type descriptor. It is a figure which shows the example of a P-type descriptor. It is explanatory drawing which calculates the correlation degree and similarity degree of an input image and a template image. It is a figure which shows the example of the mask pattern for extracting the area | region of the eye of the face. It is a flowchart which shows the process sequence of the human recognition process 1 (medium distance, head recognition). It is a flowchart which shows the simple identification process procedure of a human face. It is a figure which shows the sample image etc. of the imaging scene in the 2nd Embodiment of this invention. It is a flowchart which shows the process sequence in the embodiment. It is a flowchart which shows the process sequence in the 2nd Embodiment of this invention. It is a figure which shows the example of the picked-up image in the same embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Digital camera 2 Main body 3 Release button 8 Mode change switch 9 Zoom operation key 10 Cursor key 13 Menu key 14 Display part 20 Image sensor 23 Operation part 25 Control part 30 Compression encoding / decompression decoding part 31 Still image / moving image memory 32 Program memory 33 Data memory 35 Operation input unit 38 Shooting control unit 39 External memory medium 46 Distance sensor 56 Drive mechanism 58 Shutter 64 Shutter drive unit

Claims (12)

Imaging means for imaging a subject;
Display means for displaying an image formed by the imaging means;
A selection means for arbitrarily selecting a plurality of subjects to be counted from the image displayed on the display means;
Counting means for counting, for each subject, the number of image portions whose similarity with feature data extracted from each subject is greater than or equal to a predetermined value in the image displayed on the display means ;
Display control means for simultaneously displaying the number counted for each subject by the counting means on the display means while distinguishing for each subject;
A camera apparatus comprising: 2. The display control means, for each of a plurality of subjects selected by the selection means, displays a sample image of each subject and the number of each subject in association with each other on the display means. The camera device described. The display control means associates the sample image of each subject with the number of each subject in a state where the image portion corresponding to each subject counted by the counting means is identified and displayed in the image displayed on the display means. The camera device according to claim 2, wherein the camera device displays the images simultaneously. 4. The camera apparatus according to claim 3, wherein the display control means performs painting with different shapes corresponding to each of a plurality of subjects to identify and display an image portion corresponding to each subject. A cursor whose position is arbitrarily displaced in accordance with the operation is displayed on the display means, and further includes an area designating means for designating an arbitrary area whose diagonals are two points designated by the cursor ,
2. The counting unit according to claim 1, wherein the counting unit counts the number of image portions whose similarity with the feature data extracted from each subject is greater than or equal to a predetermined value in the region specified by the region specifying unit. 5. The camera device according to any one of 4. A cursor whose position is arbitrarily displaced according to the operation is displayed on the display means, and further includes a position designation means for designating an arbitrary position indicated by the cursor,
  The camera apparatus according to claim 1, wherein the selection unit selects a subject at a position designated by the position designation unit. The position specifying means displays a cursor whose position and size are arbitrarily displaced according to an operation on the display means, and specifies an arbitrary position and size indicated by the cursor,
  The camera device according to claim 6, wherein the selection unit selects a subject from an area corresponding to the position and size designated by the position designation unit. Wherein based on the number of counted the image portion by the counting means, a camera apparatus according to any one of claims 1 to 7, characterized by further comprising a photographing control means for controlling the photographing operation of the camera device. Storage means for storing shooting conditions corresponding to each of a plurality of shooting scenes;
Selection means for selecting one of the plurality of shooting scenes based on the number of the image portions counted by the counting means;
Claim 8, characterized in that the selection means in response to the selected photographing scene by on the basis of the imaging conditions stored in said storage means, further comprising a photographing control means for controlling the photographing operation of the camera device The camera device described. An instruction selecting means for selecting any of the plurality of shooting scenes based on a photographer's instruction operation;
When the shooting scene is selected by the instruction selection unit, the shooting control unit controls the shooting operation of the camera device based on the shooting condition stored in the storage unit corresponding to the shooting scene. The camera device according to claim 9. The camera apparatus according to claim 8 , wherein the photographing control unit controls at least one of a focusing operation, a depth of field, an exposure condition, and a filtering process of the camera apparatus. A computer having a camera device including an image forming means for forming an image of a subject and a display means for displaying an image formed by the image forming means;
A selection means for arbitrarily selecting a plurality of subjects to be counted from the image displayed on the display means;
Counting means for counting, for each subject, the number of image portions whose similarity with feature data extracted from each subject is greater than or equal to a predetermined value in the image displayed on the display means ;
Display control means for causing the display means to display the number counted for each subject by the counting means for each subject;
A camera control program characterized by being made to function.

Images