JP2008227918A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device which is improved in convenience by reducing user's burden by more highly automating framing of a group photo, etc. <P>SOLUTION: N pieces of face recognition regions are acquired (S11) and are stored as an array of structures Frect(i) (i=0 to N-1) of the recognized N pieces of face recognition regions. Then, the array of structures is sorted based on the area (height Ydim by width Xdim) of the face recognition region of each structure (S12) to be rearranged into Frect(0)>Frect(1)>...>Frect(N-1) according to the sizes. The face recognition regions are evaluated based on the ratio of their areas to the face recognition region Frect(0) having the largest area multiplied by a predetermined threshold value ThS (0<ThS<1), that is, the area of the individual face recognition regions from Frect(1) to Frect(N-1) is compared with the reference area (face recognition region Frect(0)×ThS). Then, k pieces (1≤k≤N, at least one (Frect(0))) of face recognition regions having an area larger than the reference area are determined as important subjects (S14). Then, a normalized frame wherein a rectangular region containing the face recognition regions determined as the important subjects is fit, is determined and then is zoomed in. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a face recognition function in an imaging apparatus, and more particularly to an imaging apparatus having a function of automating framing when a plurality of faces are recognized in an imaging apparatus having a face recognition means.

  One of the main uses of an imaging apparatus such as an electronic still camera in recent years is portrait photography, that is, photographing a person's face as a main subject. In addition, group photographs are often taken where a plurality of people are lined up. In these photographing applications, the importance of the portion corresponding to the background is relatively low, while the importance of the face of the person serving as the subject is relatively high.

As an invention for obtaining an appropriate angle of view with respect to a subject, an invention for recognizing a face and giving a warning to urge framing again as in Patent Document 1, There has also been an invention in which the size of a recognized area is measured appropriately and automatically zoomed, and a camera shake correction lens is driven so that the face is centered, or the angle of view is adjusted using electronic zoom.
JP-A-11-008788 JP 2004-320286 A

  However, the configuration described in Patent Document 1 requires manual adjustment including a zoom operation in order to perform framing again manually, which is inconvenient and troublesome. The configuration described in Patent Document 2 provides a function for adjusting the angle of view for each person, particularly when there are a plurality of persons as subjects. When the recognition area is widely dispersed, there is a problem that an appropriate correspondence for accommodating a plurality of persons suitable for photographing in one image is not shown.

  The present invention is directed to solving the above-described problems of the prior art, and reduces the burden on the user by automating framing to a higher degree in photographing a human subject including a face, particularly in a group photo. An object of the present invention is to provide an imaging device with improved performance.

  In order to achieve the above object, an imaging apparatus according to claim 1 of the present invention includes a moving unit that moves an imaging surface within a predetermined range, a zoom unit that optically changes a focal length, and monitoring. In an imaging apparatus having an extraction unit that recognizes a plurality of faces from image data and extracts a face recognition area, when the plurality of face recognition areas are extracted from the image data, the area or circumference of each recognized face recognition area Based on the length, determination means for determining whether the face area should be included in the angle of view, and automatically zooming or imaging so that the angle of view includes a part or all of the face area to be included in the angle of view By providing the driving means for moving the surface, it is possible to automatically perform optimum zooming when there are a plurality of faces as in a group photo.

The image pickup apparatus according to claim 2 is the image pickup apparatus according to claim 1, wherein the determination unit sets a predetermined ratio with respect to the face recognition area having the maximum area among the plurality of extracted face recognition areas as a threshold value. By making a comparison by comparing the areas of the face recognition areas, it is possible to improve the precision of zoom operation in a subject environment that is a characteristic of the group photo and that is relatively close to the face recognition area. The described imaging apparatus is the imaging apparatus according to claim 1 or 2, wherein the zooming unit zooms to the telephoto side or the moving unit determines the moving position of the imaging surface before performing the face recognition process. By performing the zoom operation to the wide angle side, the face recognition area is extracted from the widest field angle, thereby improving the zoom operation precision.

  Further, the imaging device according to claims 4 and 5 is the imaging device according to claims 1 to 3, wherein when a plurality of face recognition regions are extracted from the image data, the number of faces included in the angle of view sequentially decreases. The zooming operation to the telephoto side and the zooming operation to the wide-angle side so that the number of faces included in the angle of view sequentially increases, so that the composition intended by the user according to the number of faces to be noticed. Candidates can be automatically set in a stepwise and easy manner.

  According to the present invention, it is possible to automatically perform optimal zooming on a subject having a plurality of faces such as a group photo.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a schematic configuration of an imaging apparatus according to an embodiment of the present invention. In FIG. 1, reference numerals 100A and 100B denote a photographing lens having a zoom / focus function and a motor for driving the photographing lens. Reference numerals 101A and 101B denote a diaphragm / shutter for controlling the amount of light to be exposed to the CCD 103 and a motor for driving the diaphragm / shutter. The motors 100 </ b> B and 101 </ b> B are controlled by a motor driver 102 based on a control signal from the CPU 200.

  As shown in (Table 1), there are nine zoom positions from Zp0 (wide angle side) to Zp8 (telephoto side). Here, with reference to the coordinate space of the image data (width 320 × height 240 pixels) being monitored at Zp0 (wide angle side), the area corresponding to the angle of view at each zoom position Zp is normalized frame F (0) to Define as F (8). As a conceptual example, FIG. 2 shows an example of regions of normalized frames F (0), F (4), and F (8).

撮像部はＣＣＤ１０３によって生成される画像信号に対し、増幅器１０４Ａで信号を増幅された後にＡ／Ｄ変換器１０４Ｂでデジタル画像信号に変換される。以上の各回路の同期はシグナルジェネレータ１０５により制御する。また、デジタル画像信号はＶＲＡＭコントローラ１０６を経由してＶＲＡＭ１０７にピクセルごとの輝度色差画像データとして格納される。

The imaging unit amplifies the signal generated by the CCD 103 with the amplifier 104A and then converts the image signal into a digital image signal with the A / D converter 104B. The synchronization of the above circuits is controlled by the signal generator 105. The digital image signal is stored as luminance color difference image data for each pixel in the VRAM 107 via the VRAM controller 106.

  The CCD 103 can be continuously moved within a certain range on a plane perpendicular to the optical axis by the CCD drive mechanism 103A. The CCD drive mechanism 103A is applied by using a known camera shake correction mechanism that servo-controls driving of the imaging surface by the motor driver 102 using, for example, a hall element as a sensor and a voice coil and a permanent magnet as an actuator.

  The CPU 200 performs overall control including the imaging system. The CPU 200 has a bus, in which there is a VRAM 107, a resizing module 200A for enlarging / reducing image data, a JPEG coder 200B for compressing and converting image data into JPEG code, a SDRAM 200C for general-purpose memory, a DMA controller 200D for performing DMA control, and detachable. Are connected to a card controller 200E that controls writing / reading to / from the memory card 201, a video controller 200F that generates a video signal to be output to the LCD monitor 202, and a flash memory 200G in which image data is stored for each control program and scene. Yes.

  Further, as an interrupt signal to the CPU 200, a release key 120 for directing shooting and recording, a scene button 121 for setting a shooting scene, and the like are provided in the operation unit 109, and there is a clock generator 108 for performing a timer interrupt.

  In the present embodiment, the face recognition of the imaging apparatus is determined at predetermined time intervals based on image data in the VRAM 107 that is updated every frame during monitoring. A region that seems to correspond to a face is detected by evaluation value calculation based on pixel values using a predetermined local window as a template. The method will be described below.

  A face recognition evaluation value f (x) of an area is obtained for an arbitrary rectangular area x with respect to a position and size corresponding to a part in image data to be evaluated. For deriving f (x), the local window i of the rectangular region x composed of the four patterns A, B, C, and D shown in FIG. 3 is used.

  Since there are a plurality of local windows i and relative positions in the region including the same pattern, and the detected face parts are different, the feature amounts are evaluated independently. Each local window i is composed of two regions, and includes a normal region Ap and a reverse region An (the forward / reverse notation is for convenience of identification, and it is synonymous to assign forward / reverse). . The feature value h (i) of each local window i is the difference between the sum s (i, Ap) of pixel values in the normal area Ap of the local window i and the sum s (i, An) of pixel values in the reverse area An, That is, (Equation 1)

で求める。

Ask for.

  If the feature value h (i) is obtained for all the local windows i, the face recognition evaluation value f of the area is calculated as a weighted sum related to the local window i using a predetermined weight coefficient a (i) for each local window i. (Equation 2)

にて求める。各局所窓ｉの大きさ、相対位置、重み係数ａ（ｉ）は予め顔画像のデータベースから学習によって得たデータを与える。例えば、本実施の形態では、（表２）のようなデータ群を与え、プログラム上では、それぞれ（表２）のＳ５０に示すような等価な構造体の形式で扱う。

Ask for. The size, relative position, and weighting factor a (i) of each local window i give data obtained by learning from a face image database in advance. For example, in the present embodiment, a data group as shown in (Table 2) is given and handled in the program in the form of an equivalent structure as shown in S50 of (Table 2).

そして、本実施の形態では、図２に示した画像データを３２０×２４０ピクセル、評価する顔認識領域の最小サイズを４０×４０ピクセル、同最大サイズ２４０×２４０ピクセルとして、４ピクセル単位で評価する（図４参照）。また、（表２）に示す局所窓フレームの適用概念を図５に示す。

In this embodiment, the image data shown in FIG. 2 is evaluated in units of 4 pixels, with 320 × 240 pixels, the minimum size of the face recognition area to be evaluated is 40 × 40 pixels, and the maximum size is 240 × 240 pixels. (See FIG. 4). Moreover, the application concept of the local window frame shown in (Table 2) is shown in FIG.

  6 and 7 are flowcharts showing the flow of processing for determining and registering the face recognition area. The flowchart of FIG. 6 is a process for determining the rectangular area x to be evaluated from the area where the face recognition area is small to the large area, and from the upper left to the lower right of the image. The flowchart of FIG. For the rectangular area x, the face recognition evaluation value f (x) is calculated by (Equation 2), and when it is larger than a predetermined threshold Th, a procedure for additionally registering it as an array of the data structure Fract as a face recognition area is shown. .

  In this process, since various partial areas are independently evaluated with respect to the position and size of the image space, a plurality of face recognition areas may be extracted. That is, in the present embodiment, it is possible to recognize a plurality of faces within the range of effects assumed by the recognition method.

  The N face recognition areas extracted as face recognition areas by the above processing are stored as a structure array Fract (i) (i = 0, 1, 2,..., N). Further, as shown in FIG. 8, the data format of each face recognition determination area Fract (i) is the upper left coordinates (Xb, Yb) of the starting image space, the width and height of the rectangle (Xdim, Ydim), It is comprised by the face recognition evaluation value V obtained by the process.

  Further, in order to determine the zoom position Zp, processing for determining a part of the recognized face recognition area as an important subject is performed. As a method for determining an important subject, in this embodiment, a method based on the area of the face recognition area is first presented, and a method based on the face recognition evaluation value is presented second. Any of these methods may be used.

Thereafter, for the sake of convenience, Fract (i) parameters (start point (Xb, Yb), size (Xdim, Ydim), and evaluation value V are expressed in the form of Fract (i) .Xb, respectively).
As a first method, as shown in the flowchart of FIG. 9, the evaluation is performed by the area ratio of a region using a predetermined threshold ThS (0 <ThS <1, typically ThS = 0.5). . First, N face recognition areas are acquired based on the flowchart shown in FIG. 7 (S11), and the structure Fect (i) of the recognized N face recognition areas: (i = 0 to N−1). Are sorted (S12). The sorting criterion is determined by the area (Xdim × Ydim) of the area obtained from the area vertical and horizontal sizes (Xdim, Ydim) of each structure. As a result, the structures are rearranged in the order of Fract (0)> Flect (1)>...> Flect (N-1) according to the area.

  At this time, the face recognition area having the maximum area is Fract (0), and therefore each area (Flect (i) .Xdim × Flect (i). Ydim) is compared with the determination area (Flect (0) .Xdim × Fect (0) .Ydim × ThS) (S13). The former is larger than the latter k (1 ≦ k ≦ N, and at least one (Fect (0)) satisfies the condition) face recognition regions Fect (0), Fract (1),..., Fect (k−1) Are determined to be important subjects (S14).

  In the process based on the second face recognition evaluation value described above, the same process as the flowchart of FIG. 9 is performed using the face recognition evaluation value V as in the flowchart shown in FIG.

  FIG. 11 is a flowchart showing a process for determining a zoom position so as to include an area recognized as an important subject. FIGS. 12A to 12D, 13A to 13D, and FIG. FIG.

  As shown in the flowchart of FIG. 11, in the zoom process of the group photo, first, the zoom is performed to the wide angle side (Zp0) (initial state shown in FIG. 12A) (S21). The purpose of this is to acquire a monitoring image for face recognition evaluation. Thereafter, image data to be evaluated (320 × 240 pixels) is acquired from the monitoring image (S22). Next, a flowchart (FIG. 9 or FIG. 10) for determining an important subject for acquiring an important subject area is entered, and the structures Fect (0), Fract (1),... k) is acquired (S23).

  Thereafter, the rectangular area R is calculated in order to determine the zoom position Zp on the widest angle side including all areas including Fract (0) to Fract (k−1) (see FIG. 12B). Further, using the above-described normalized frame F (Zp) corresponding to each zoom position Zp (9 levels from 0 to 8) shown in (Table 1), a normalized frame in which the rectangular region R is accommodated is determined (FIG. 12). (See (c)) (S24). Finally, the zoom is made to the determined zoom position Zp, and the process is completed (Zoom 1 up to Zp1 shown in FIG. 12D) (S25).

  Further, in the present embodiment, a method of driving only Fract (0) as an important subject and driving to the zoom position Zp on the widest angle side including, for example, the Fract (0) region by the same method as described above (FIG. 13A). )-(D)). In this case, in the face recognition area recognized by the monitoring image data, one of a plurality of face recognition areas is selected, and this is set as a rectangular area R, and the above-described processing is performed to perform zoom processing of the important subject intended by the user. Can do.

  This operation is performed by using the “WIDE key” and the “TELE key” to perform bidirectional state transition as shown in FIG. 14 to provide multi-stage zooming according to the number of characteristic faces. Can do.

  In the above-described processing, the normalized frame in which the rectangular area R is accommodated is determined and zoom processing is performed. The center position of the rectangular area R is obtained, and the angle of view is within the movable range of the imaging surface by the moving means. The zoom processing may be performed by moving to the center and determining the normalized frame in which the moved rectangular area R is accommodated, and the zoom can be performed with the important subject intended by the user placed at the center.

  The imaging apparatus according to the present invention can automatically perform optimum zooming on a subject having a plurality of faces such as a group photo, and when the imaging apparatus having face recognition means recognizes a plurality of faces. This is useful as a function for automating framing.

1 is a block diagram illustrating a schematic configuration of an imaging device according to an embodiment of the present invention. Diagram showing the concept of normalized frames The figure which shows the pattern of the face recognition judgment frame Diagram showing an example of the structure of internal processing data The figure which shows the application concept of a face recognition determination frame Flowchart showing an evaluation target area acquisition process in a process of determining and registering a face recognition area The flowchart which shows the acquisition process of the face recognition area in the process which determines and registers a face recognition area The figure which shows the example of a data structure of a face recognition rectangular area Flow chart showing important subject determination processing Flowchart showing another important subject determination process Flow chart showing automatic zoom processing The figure which shows the state transition (a)-(d) which carried out the automatic zoom process The figure which shows the state transition (a)-(d) which carried out the automatic zoom process further from FIG. Diagram illustrating bidirectional state transition using “WIDE key” and “TELE key”

Explanation of symbols

100A Zoom / focus lens 100B, 101B Motor 101A Shutter / aperture 102 Motor driver 103 CCD
103A CCD drive mechanism 104A Amplifier 104b A / D converter 105 Signal generator 106 VRAM controller 107 VRAM
108 Clock generator 109 Operation unit 200 CPU
200A Resize module 200B JPEG coder 200C SDRAM
200D DMA controller 200E Card controller 200F Video controller 201 Memory card 202 LCD monitor

Claims (5)

A moving unit that moves the imaging surface within a predetermined range; a zoom unit that optically changes a focal length; and an extracting unit that recognizes a plurality of faces from the image data being monitored and extracts a face recognition area. In the imaging device,
When extracting a plurality of face recognition areas from the image data, based on the area or circumference of each recognized face recognition area, determination means for determining whether the face area should be included in the angle of view;
Driving means for automatically zooming or moving the imaging surface so that the angle of view includes a part or all of the face area to be included in the angle of view;
An imaging apparatus comprising:   2. The imaging according to claim 1, wherein the determination unit makes a determination by comparing face recognition area areas with a predetermined ratio to a face recognition area having a maximum area among the plurality of extracted face recognition areas as a threshold value. apparatus.   The zoom operation is performed to the widest angle side before performing a face recognition process for performing a zoom operation on the telephoto side by the zoom unit or determining a moving position of the imaging surface by the moving unit. 2. The imaging device according to 2.   4. The zoom operation according to claim 1, wherein when a plurality of face recognition areas are extracted from the image data, the zoom operation is performed toward the telephoto side so that the number of faces included in the angle of view sequentially decreases. The imaging device described.   5. The image pickup apparatus according to claim 4, wherein when a plurality of face recognition areas are extracted from the image data, the zoom operation is performed to the wide angle side so that the number of faces included in the angle of view sequentially increases.

Images