JP2010027000A - Image detection device and image detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To track and detect a seamless face by using an inspection processing speed that follows the frame of a moving image in order to clearly photograph a moving image that changes with time. <P>SOLUTION: This image detection device includes a scaling part 201 for expanding or reducing image data on a screen, an image detecting part 203 for detecting a specific image in image data subjected to scaling processing by scanning a rectangular area of a prescribed size on the screen provided with a plurality of sections horizontally, or horizontally and vertically, wherein the image data obtained by the scaling part includes the plurality of sections having the prescribed number of pixels, a candidate area processing part 204 for setting a detection likelihood value corresponding to the degree of overlap of the sections of the screen of the specified image obtained by the detection, and a storing part 202 for storing the set detection likelihood value. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image detection apparatus and an image detection method for detecting a specific image from a screen, and more particularly to an image detection apparatus for detecting a human face image in a specific area from a screen and performing a desired process. The present invention relates to an image detection method.

  In recent years, digital still cameras mainly focusing on still images and digital video cameras mainly focusing on moving images detect the face of a person to be captured, and the detection results are AF (Auto Focus) and AE (AE). A function for controlling Auto Exposure) or AWB (Auto White Balance), or adjusting the flesh tone of a photographed image has been installed. In order to capture a momentary scene that changes from moment to moment clearly while taking advantage of the above control, it is necessary to detect the scene while tracking the face at a speed comparable to or faster than the frame rate of the moving image. .

With respect to the method of performing face detection, Patent Document 1 discloses whether or not an object is a face based on a detection accuracy value indicating the likelihood of a face, and performs scanning by reducing the image stepwise. A method for retrieving the position and size of a face from among the methods is disclosed.
In this patent document 1, a rectangular area having a predetermined size is scanned on the screen by moving Δx in the horizontal direction and Δy in the vertical direction from the upper left end to the lower right end of the captured image. Find the position of.
In addition, a reduced image obtained by reducing the captured image stepwise by a predetermined reduction ratio Δr is generated, and the generated reduced image is repeatedly scanned to obtain the face size.
Then, from all the pixels in the rectangular area of the predetermined size, all the differences between the luminance values of the two pixels determined in advance by learning are calculated, and the detection accuracy values indicating the facialness are obtained based on the calculation results. . The detection accuracy value of the image in the rectangular area has a threshold value for determining whether or not the object is a face in advance, and if the calculated detection accuracy value is equal to or greater than the threshold value, the object is the face. If it is lower than the threshold, it is determined that the object is not a face.

In the method of performing face detection in Patent Document 1, the number of times of arithmetic processing when processing one image is very large. The number of times of calculation processing for determining whether or not the object is a face is “number of times of image reduction × number of times of movement of the rectangular area”. The large number of calculation processes is a cause of reducing the face detection processing speed.
In order to solve this disadvantage, Patent Document 2 searches for candidate areas that are highly likely to include a face based on color characteristics, and only the candidate areas are searched for faces, thereby reducing the number of calculation processes. A technique is disclosed.
Here, “highly likely to include a face” is, for example, an image region having a skin color. However, this method has a disadvantage that if the background has a color close to skin color, the background is included in the candidate area. Further, the skin color is not limited to only the face, and for example, a portion where the skin is exposed, such as an arm or a foot, is a skin color, and there is a disadvantage that the portion is included in the candidate region.

In order to solve this disadvantage, in Patent Document 3, face detection results (face size, face position, etc.) obtained by past shooting are accumulated, and in this imaging apparatus, at which position Learn what size the face tends to be shot. Based on the learning data, a method is disclosed in which a candidate area where a face may exist is detected in captured image data, and the face detection process is performed only on the area, thereby reducing the number of calculation processes.
However, this method is very effective when the scene to be captured is fixed as in the case of a fixed point camera and the tendency of the image to be captured is fixed, but it is not portable as in a digital still camera or digital video camera. In the case of an imaging device, since the tendency of a captured image changes depending on time and place, the learning data is not useful and a candidate area cannot be specified.

  This face detection method obtains a detection accuracy value indicating whether or not a rectangular area is a face, and if the detection accuracy value is equal to or greater than a threshold value, determines that the target is a face, and if the detection accuracy value is lower than the threshold value. The method of judging that the object is not a face is taken. If the detection accuracy value of the rectangular area is lowered due to the condition of the captured image, the face is not determined. Conditions for lowering the detection accuracy value include face orientation, how light strikes, presence / absence of objects that hide the face such as glasses / hats, obstacles crossing the front of the face, and the like. The fact that there is a moment when a face cannot be detected means that the position and size of the face in the captured image is lost at that moment, and subsequent face detection cannot be tracked.

  In order to solve this disadvantage, Patent Document 4 records ambient information obtained from an area determined by a relative relationship with the face position in association with the detected face position, and if no face is detected. In this case, based on the position of the face detected immediately before, a plurality of face position candidates are obtained, the surrounding information corresponding to each position candidate is obtained, and associated with the position of the face detected immediately before. A method is disclosed in which a position candidate having the most similar surrounding information to the surrounding information is used as the face position.

Examples of the surrounding information described above include clothes positioned below the face. However, in this method, when a face is not detected, the surrounding information associated with the position of the face detected immediately before does not always remain, and therefore similar surrounding information may not be found.
For example, if clothes are considered to be surrounding information, if the face cannot be detected due to a change in face orientation, if the orientation of the whole body changes as the reason for the change of face orientation, It is possible to change. Also, if the face is no longer detected by the way the light hits, if the light hits the entire body changes, for example, when a person enters the shadow of a building, the light that hits the clothes may also change. Conceivable.
In addition, when a face is no longer detected by an obstacle that crosses the front of the face, there are small obstacles that cover only the face as obstacles that cross, and there are large obstacles that cover the entire body. It is conceivable that this will also hide clothes. In addition, when shooting a person in the same clothes, such as a uniform, there are a plurality of position candidates having similar surrounding information, and the position of the face cannot be narrowed down.

JP 2005-157679 A JP 2001-309225 A JP 2007-122484 A JP 2007-42072 A

  Accordingly, it is an object of the present invention to provide an image detection apparatus and method for speeding up processing detection and performing seamless face tracking detection.

  An image detection apparatus according to the present invention includes a scaling unit that enlarges or reduces image data on a screen, and a plurality of sections having a predetermined number of pixels with respect to the image data obtained by the scaling unit. An image detection unit that scans the screen provided in the section in a rectangular area of a predetermined size in the horizontal direction, or in the horizontal and vertical directions, and detects a specific image in the scaled image data; and A candidate area processing unit that sets a detection accuracy value corresponding to the degree of overlap of the sections of the screen of the specific image obtained by detection, and a storage unit that stores the detection accuracy value set by the candidate area processing unit. Have.

  The image detection method of the present invention includes a step of enlarging or reducing the image data on the screen by scaling processing, and a step of providing a plurality of sections having a predetermined number of pixels in the image data enlarged or reduced by the scaling processing. Scanning the plurality of sections provided on the screen with a rectangular area of a predetermined size in the horizontal direction on the screen, or in the horizontal and vertical directions, and the scaling process by scanning the rectangular area A step of detecting a specific image in the obtained image data, a step of setting a detection accuracy value according to the degree of overlap of the section corresponding to the specific image obtained by the detection, and the set detection accuracy value Storing.

  In the image detection apparatus or the image detection method of the present invention, a plurality of sections composed of a plurality of pixels are provided in a frame image, and a plurality of sections on the screen are scanned in a horizontal direction and a vertical direction in an inspection area of a specific size. If a specific image is included in the inspection area, an inspection accuracy value is set. Otherwise, the inspection accuracy value is set to zero (0), and a similar inspection is performed on each frame image to obtain a detection candidate area. .

  According to the image detection apparatus and the image detection method of the present invention, it is possible to increase the speed of face detection without interrupting the tracking of, for example, face detection of a specific image area on the screen.

  FIG. 1 shows a configuration example of an imaging apparatus 100 that is an embodiment of the present invention. Hereinafter, an image detection apparatus that performs face detection will be described as a specific example, but the present invention is not particularly limited to a face as image data, and may be another specified image on a screen.

  The imaging apparatus 100 includes an optical block 101, a signal conversion unit 102, a camera signal processing unit 103, a face detection unit 104, a display processing unit 105, an image signal processing unit 106, a storage unit 107, a display unit 108, an image RAM (Random Access Memory). ) 109, a central processing unit (CPU) 110, a read only memory (ROM) 111, a RAM 112, and an image bus 113.

  The optical block 101 includes a lens group for imaging a subject (not shown), an aperture adjustment mechanism, a focus adjustment mechanism, a zoom mechanism, a shutter mechanism, a flash mechanism, and the like, and is supplied from a control unit (CPU) 110 described later. Control such as zoom control, shutter control, and exposure control is performed according to the control signal.

  The signal conversion unit 102 is configured by an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) image sensor, and an input subject image is formed on an image forming plane in the optical block 101. The signal conversion unit 102 receives, for example, an image capture timing signal supplied from the control unit 110 in response to a shutter operation, and captures an optical signal of a subject image formed on the imaging plane as an imaging (video) signal. It is converted into an electric signal and supplied to the camera signal processing unit 103.

  The camera signal processing unit 103 performs various signal processing on the imaging signal output from the signal conversion unit 102 based on the control signal from the control unit 110. Specifically, for example, processing such as gamma correction and AGC (Auto Gain Control) is performed on the imaging signal from the signal conversion unit 102, and processing for converting the imaging signal into image data as a digital signal is performed. . The camera signal processing unit 103 further performs white balance control, exposure correction control, and the like on the image data based on a control signal from the control unit 110.

  The face detection unit 104 receives the image data output from the camera signal processing unit 103 via an image RAM (Random Access Memory) 109, detects a human face from the received image data, and detects the detection result. It supplies to the control part 110. The configuration of the face detection unit 104 and details of the face detection process will be described later.

  The display processing unit 105 receives image data output from the camera signal processing unit 103 and an image signal processing unit 106 described later via the image RAM 109, converts the image data into an image having a resolution suitable for display, for example, and displays the display unit 108. To supply.

  The display unit 108 displays the image data processed by the display processing unit 105 on the display screen. The display unit 108 is used as a view finder in the imaging apparatus, and is also used as a monitor for an image reproduced from the storage unit 107. As the display unit 108, for example, an LCD (Liquid Crystal Display) can be used.

The image signal processing unit 106 receives the image data output from the camera signal processing unit 103 via the image RAM 109, compresses and encodes this image data, and outputs it to the storage unit 107 as a moving image or still image data file. To do.
Further, the image signal processing unit 106 decodes an image data file read from the storage unit 107 described later, and supplies the decoded image data file to the display processing unit 105 via the image RAM 109. For example, a moving picture experts group (MPEG) system can be applied as a moving picture encoding system, and a JPEG (Joint Photographic Experts Group) system can be applied as a still image encoding system.

  The storage unit 107 stores the image file generated by being encoded by the image signal processing unit 106. As the storage unit 107, for example, a drive device of a recording medium such as a magnetic tape or an optical disk, a flash memory, or an HDD (Hard Disc Drive) can be used. In addition to reading out the stored image data file to the image signal processing unit 106, the storage unit 107 supplies information accompanying the image data file to the control unit 110. The information attached to the image data file is data such as imaging conditions and face detection results embedded in the image data file, for example.

  The image RAM 109 is connected to the camera signal processing unit 103, the face detection unit 104, the display processing unit 105, and the image signal processing unit 106 through an image bus 113. The image RAM 109 is shared by these processing blocks connected to the image bus 113, and image data is transferred between the blocks via the image RAM 109. In this example, these processing blocks are described as transmitting and receiving image data via the image RAM 109. However, the present invention is not limited to this example. For example, the face detection unit 104 and the display processing unit Reference numeral 105 may be configured to directly receive output image data from the camera signal processing unit 103 and the image signal processing unit 106 without using the image bus 113.

  The control unit (CPU) 110 includes a CPU and the like, and controls the entire imaging apparatus (100) using the RAM 112 as a work memory in accordance with a program stored in advance in the ROM 111. For example, the CPU of the control unit 110 exchanges commands and data with each unit of the imaging apparatus 100 and controls each of them. Further, the CPU of the control unit 110 generates a control signal for controlling the focus, aperture, zoom, and the like of the optical block 101 based on a control signal, an imaging signal, and the like according to an operation on an operation unit (not shown), and the optical block 101.

  FIG. 2 shows a configuration example of a circuit block of the face detection unit 104. The face detection unit 104 includes a scaling unit 201 as an image conversion unit, an image buffer 202, a face detection core unit 203 as a face detection processing unit, a candidate area processing unit 204, and a controller 205.

The scaling unit 201 reduces or enlarges the size of the image data read from the image RAM 109 via the image bus 113 (the number of pixels in the horizontal direction and the vertical direction) so as to be suitable for the face detection processing in the face detection core unit 203. Scaling processing is performed and supplied to the image buffer 202.
In addition, a face detection candidate area bitmap calculated by the candidate area processing unit 204 as a candidate area having a high possibility of including a face is received from the controller 205, and scaling processing is performed in the same manner as an image, which is supplied to the image buffer 202. To do. Details of the scaling processing of the face detection candidate area bitmap will be described later.

The image buffer 202 temporarily holds the image data whose size has been converted by the scaling unit 201. Based on the control of the controller 205, a scanning process for cutting out rectangular image data of a predetermined size at a designated position from the image data is performed and supplied to the face detection core unit 203.
Further, it is determined from the face detection candidate area bitmap size-converted by the scaling unit 201 whether or not the extracted rectangular image data falls within the face detection candidate area, and face detection processing is performed (On ) / Not performed (Off) flag data is created and supplied to the face detection core unit 203. The details of a method for creating flag data indicating whether face detection is performed (On) or not performed (Off) will be described later.

  The face detection core unit 203 performs face detection processing on the rectangular image data supplied from the image buffer 202. The face detection core unit 203 performs face determination processing and overlap determination processing as face detection processing, and outputs the face detection result to the control unit 110 and the candidate area processing unit 204. However, the above-described face detection processing is performed on the rectangular image data with On flag indicating that face detection supplied from the image buffer 202 is performed (On) / not performed (Off). Face detection processing is not performed for the rectangular image data.

  The candidate area processing unit 204 calculates an area where a face is highly likely to be detected in the next frame image based on the face detection result supplied from the face detection core unit 203, and uses the area as the next frame image. Then, the face detection candidate area is set. The region where the possibility of detecting a face is high is a peripheral image at the position where the face is detected in the current frame image. The set face detection candidate area data is supplied to the controller 205. A method for calculating face detection candidate area data will be described later.

  Based on the control of the CPU provided in the control unit 110, the controller 205 performs instructions such as an instruction for a reduction rate (or enlargement rate) of the image data to the scaling unit 201 and an instruction for a memory address for writing or reading to the image buffer 202. Each part of the detection part 104 is controlled. Further, the face detection candidate area data obtained by the candidate area processing unit 204 is converted into a face detection candidate bitmap, and then supplied to the scaling unit 201. A method for calculating the face detection candidate area bitmap will be described later.

Next, the operation of the imaging apparatus 100 will be described with reference to FIGS. 1 and 2.
First, image capture will be described.
At the time of image recording, light from a subject is incident on an image sensor (not shown) via the optical block 101, and the image formed on the image sensor is converted into an electric signal by photoelectric conversion in the signal conversion unit 102. It becomes an imaging (video) signal. The imaging signals are sequentially supplied to the camera signal processing unit 103 and subjected to digital conversion, image quality correction processing, and the like. The image data that has been subjected to various processes is temporarily stored in the image RAM 109 via the image bus 113.

  The display processing unit 105 receives the image data from the camera signal processing unit 103 via the image RAM 109, generates a display image signal, and supplies it to the display unit 108. As a result, the currently captured image is displayed on the display unit 108, and the photographer can visually check this image and confirm the angle of view.

  In addition, the image signal processing unit 106 sequentially receives the image data output from the camera signal processing unit 103 via the image RAM 109, performs compression encoding processing using, for example, the MPEG method, and generates a moving image file. Records in the storage unit 107.

  Further, in response to pressing of a shutter button provided in an operation unit (not shown), image data for one frame is extracted from the camera signal processing unit 103, and the image signal processing unit 106 performs compression encoding using, for example, the JPEG method. It is also possible to generate a still image file by performing processing and record it in the storage unit 107.

  Furthermore, at the time of image reproduction, the image file stored in the storage unit 107 is read out and decoded by the image signal processing unit 106, supplied to the display processing unit 105, and converted into a display image. Thereby, a moving image or a still image can be reproduced and displayed on the display unit 108.

  At the time of image recording as described above, the face detection unit 104 receives the output image data from the camera signal processing unit 103 via the image RAM 109 and executes face detection processing. The face detection result of the face detection unit 104 is supplied to the control unit 110.

Next, the above-described AE, AF, and AWB control will be described.
The CPU of the control unit 110 controls AE (Auto Exposure), AF (Auto Focus), AWB (Auto White Balance), and the like based on the face detection result of the face detection unit 104. For example, the aperture amount, shutter speed, and white balance gain are adjusted so as to optimize the brightness and color of the detected face. Alternatively, it is possible to perform control such as focusing on the detected face.
The face detection unit 104 receives the image data decoded by the image signal processing unit 106 via the image RAM 109 when the image file in the storage unit 107 is played back, and the face detection unit 104 performs the same processing as the image recording. Detection processing may be executed.

Next, scaling processing, operation processing, face determination processing, and detection candidate region processing, which are main processing operations for face detection, will be described.
First, the scaling process will be described.
The scaling unit 201 generates image data 301 obtained by reducing (or enlarging) the image data 300 on the captured screen at a predetermined reduction rate (or enlargement rate). Further, the face detection candidate area bitmap 302 supplied from the controller 205 is also reduced (enlarged) at the same reduction ratio (or enlargement ratio) as the image data, and the scaled face detection candidate area bitmap is scaled. 303 is generated. FIG. 3 shows image data and a corresponding bit map.
3A and 3B, image data 300 and 301 indicate images when enlarged or reduced with respect to each other. Correspondingly, FIGS. 3C and 3D show corresponding bitmap data when the image data is enlarged or reduced.
The face detection candidate area bitmap is a group of flag data corresponding to pixels (pixels) of image data, and the upper left pixel of the rectangular image data cut out by the image buffer 202 for scanning processing is the face detection candidate. If it is within the area, it is On (1). If it is outside the face detection candidate area, it is Off (0).

FIG. 3A shows image data 300 when enlarged, and FIG. 3C shows a bitmap 302 of face detection candidate areas corresponding to the image data 300. In the detection candidate area bitmap 302, an area indicating data “1” indicates a face detection candidate area, and an area indicating data “0” indicates an area other than the face detection candidate area.
FIG. 3B shows the image data 301 when reduced, and in the detection candidate area bitmap 303 corresponding thereto, the area indicating the data “1” indicates the face detection candidate area, and the data “0”. The area indicating “” indicates an area other than the face detection candidate area.
Comparing FIG. 3 (c) and FIG. 3 (d), the area of the data “1” when enlarged indicates the area of 5 * 5 (where * indicates a multiplication symbol), and when the area is reduced An area of data “1” indicating a face detection candidate area indicates a 3 * 3 area.

Next, the scanning process will be described with reference to FIGS.
In the scanning process (corresponding to step ST-102 in FIG. 12 to be described later), the scaled image data is scanned while moving a predetermined size rectangular area by a predetermined amount in the horizontal direction and the vertical direction, thereby obtaining rectangular image data. Cut out.
The image buffer 202 performs the process shown in FIG. 4 on the image data obtained by the scaling process based on the control of the controller 205. That is, the rectangular area 310 of a predetermined size is moved by the movement amount Δx in the horizontal direction from the upper left end of the screen (corresponding to the rectangular area 311 moved in the horizontal direction), and when reaching the right end of the image data, the rectangular area 310 returns to the left end. Then, the position is changed in the vertical direction by the movement amount Δy, and is moved again in the horizontal direction. By repeating this operation, scanning is performed to the lower right corner of the image data, and rectangular image data is sequentially cut out.
The controller 205 also performs a process of extracting flag data of the face detection candidate area corresponding to the upper left pixel of the rectangular image data cut out by the scanning process.

FIG. 5 shows an image when the screen is reduced by the scaling unit 201. An image obtained by multiplying the image data 300 by Δr by the scaling ratio is shown in image data 300a, and an image obtained by multiplying the image data 300a by Δr is shown in image data 300b. Similarly, the image is reduced by predetermined or arbitrary scaling.
FIG. 5 shows an example of scaling in the case of reduction, but scaling in the case of enlargement can be performed in the same manner. For example, the screen may be a screen in the order of the image data 300c, 300b, 300a, 300 with the screen data 300d as a reference. In addition, since the description regarding the case of enlarging is the reverse of the case of reducing, it is omitted.

  As shown in FIG. 5, the size of the rectangular area 310 scanned in the scanning process is always the same size regardless of the size when the image data is enlarged or reduced. The size of the image data input to the face detection unit 104 is changed by the scaling unit 201, and the image data whose size has been changed is scanned in the rectangular region 310 of a predetermined size, thereby being detected in the rectangular region 310. Since the face size changes, the actual face size can be detected.

Next, the face determination process will be described.
In face determination processing (corresponding to step ST-104 in FIG. 12 described later), predetermined calculation processing is performed on predetermined pixels in the rectangular image data cut out by the scanning processing, and whether or not the rectangular image data includes a face. Determine whether.
When the rectangular image data is cut out from the image data scaled by the scanning process, the face detection core unit 203 selects two pixels determined in advance from among the pixels in the rectangular image data supplied from the image buffer 202. The difference between the luminance values at is calculated.
In addition, the difference between the luminance values of the other two pixels determined by learning is calculated in the same manner, and the detection accuracy value indicating the facialness in the rectangular image data based on the calculated difference between the luminance values. To decide. A predetermined threshold is set for the detection accuracy value. If the detection accuracy value is equal to or greater than the threshold, it is determined that the rectangular image data includes a face. If the detection accuracy value is lower than the threshold, the rectangular image data Is determined not to contain a face.
In addition, although the example which used the luminance value for the face detection was shown here, this is because luminance data such as eyebrow, eyes, nose, and mouth of the face are related to each other, and the color of the face is not necessarily There is no need to detect a face based on data, face shape, or the like.
In addition, the face may be detected by combining conventional face color data, face shape data, and luminance data, and the present invention is not limited to these.

In this way, by performing face determination processing on rectangular image data obtained by scanning the scaled image data in the rectangular area 310, the position of the face in the image data is obtained according to the detection accuracy value. Can do. The same face determination process is sequentially performed on the scaled image further reduced (enlarged) with the reduction rate (enlargement rate) Δr with respect to the image scaled in the scaling process, thereby reducing the size of the face in the image data. Can be sought.
Further, as shown in a later-described branch (step ST-103 in FIG. 12), these face determination processes are performed when the flag data of the face detection candidate area extracted together with the rectangular image data cut out by scanning is On. Only performed in the case of (1), not performed in the case of Off (0). Thereby, the execution of the detection process is limited only to the rectangular image data in the face detection candidate area, and the amount of calculation can be reduced.
The face determination process is repeated until all areas of the image are scanned. Further, the process is repeated until image data of all sizes is processed.

Next, detection candidate area processing will be described.
In the detection candidate area processing, based on the face detection result output by the face detection core unit 203, an image range in which face detection is performed on the next frame image is calculated as a face detection candidate area.

Here, the detection candidate areas described above are defined.
In the face detection candidate area data, as shown in FIG. 6A, the input image is divided into vertical and horizontal m * m pixels (pixels; m is a positive integer), and the horizontal direction is row candidate area data (column_block). The vertical direction is defined as column candidate area data (row_block). The size of m may be set to a small value if the face detection candidate region is set in detail, but the amount of row or column candidate region data increases as the value becomes smaller, which is not practical. . Therefore, m is defined as an appropriate size for use. In addition, the partitioning may be set to m × n pixels in the vertical and horizontal directions, and m and n (pixels; n and m are positive integers) different values.
Further, the row candidate area data has a 2-bit face detection accuracy value of, for example, 0 to 3 within a predetermined detection accuracy value range per division, and these 2 bits represent one row of the horizontal division. Similarly, the column candidate area data has a face detection accuracy value of 2 bits per section, and these 2 bits represent one column of the section in the vertical direction.

  Note that the face detection accuracy values 0 to 3 are not provided at addresses on independent coordinates of each section on the screen, but are configured on the screen as shown in FIG. 6B. It is set only for one column in the row direction and one column in the column direction corresponding to each section. In principle, face detection accuracy values may be set for all sections, but face detection accuracy values are set for only one row and one column in the row and column directions, and face detection accuracy values in the row and column directions are set. If the face detection candidate region is in a range where the value is greater than or equal to the predetermined value, the number of times of the arithmetic processing can be reduced, and the image processing can be speeded up accordingly. Detailed examples will be described later.

The face detection accuracy value indicates that the larger the numerical value, the higher the possibility that a face will be detected in the next frame. The face detection accuracy value = 0 indicates that there is no possibility that a face is detected.
A region where the face detection accuracy value in the row candidate region data and the column candidate region data is a face detection accuracy value ≧ 1 is a detection candidate region of the next frame image.

Here, the face detection accuracy value indicates a weighted value from 0 to 3 (2 bits), and varies between frame images.
For example, in the current frame image, when the rectangular area 310 moves the detection candidate area in the horizontal direction or the vertical direction on the screen, the origin (0, 0) of the coordinates of the rectangular area where the face is not detected is detected in the previous frame image. If it overlaps with another section, “1” is subtracted from the face detection accuracy value of the previous frame image. For example, when the face detection accuracy value of the previous frame image is “3”, the face detection accuracy value decreases by 1 to “2”, and when the face detection accuracy value of the previous frame image is “2”, “1” “,” And “0” when the face detection accuracy value of the previous frame image is “1”.
When the face detection accuracy value in the previous frame image is “0, 1, 2”, the face detection accuracy value is “3” in the section where a new face is detected in the current frame image.
In FIG. 6B, one section is composed of m * m pixels (m is a positive integer) on the screen, and the sections of the row candidate areas are k = 0, 1, 2,..., P−. 1 or a screen composed of a plurality of sections when the section of the column candidate area is i = 0, 1, 2,..., N−1 (where p and n represent integers greater than 1). The configuration is shown.

Next, setting of detection candidate areas will be described with reference to FIG.
When the face detection result is received, the candidate area processing unit 204 includes face detection coordinates as shown in FIG. 7 after subtracting 1 from the face detection accuracy values of all sections having one or more face detection accuracy values. The partition number (i) of the row candidate region data and the partition number (k) of the column candidate region data are calculated, and the row candidate region data and the column candidate region data corresponding to the peripheral partition ± n of the partition (i, k) C00 are calculated. Then, the face detection accuracy value = 3 is set. FIGS. 8A to 8D illustrate changes in face detection accuracy value transitions 400 to 403 over time.

In FIG. 7, C00 is a section when n = 0 and indicates C00 of coordinates (i, k), and one section when n = 1 is 1 in the horizontal and vertical directions centering on (i, k), respectively. The section which expanded the section is shown.
This enlarged section is a section indicated by hatching in FIG. 7, and (i−n, k−n), (i, k−n), (i + n, k−n), (i−n, k), (I, k), (i + n, k), (i−n, k + n), (i, k + n), and (i + n, k + n) are configured by nine sections, and the face detection accuracy value is set to Row block i [1: 0 ] = 3, coloumn block k [1: 0] = 3.
As a result, the image data on the screen is regarded as one section by combining nine sections, and the face detection rectangular area 310 is scanned with this section as a new section. Therefore, when the image moves violently between frames, It is possible to reduce losing sight of face detection. Further, the variable n can be changed to 2, 3,.
In this way, by varying the variable n, it is possible to optimize face detection accompanying changes in image movement.

In FIG. 7, the peripheral section ± n represents the size of an area where the probability that a face is detected in the next frame image is high. This is a case where the movement amount of a person's face is within ± n sections (maximum movement amount) between frame images cut out from a moving image in minute time units.
7 and 8 show an example in which n = 1. The numerical value of the peripheral block ± n may be calculated from the maximum amount of movement, or if it is desired to increase the prediction probability, it may be calculated using, for example, a motion prediction algorithm incorporated in a moving image compression technique. good. Further, the value of n may be decomposed vertically and horizontally, and different values may be used for the respective directions.
In FIG. 8B, “2” is obtained by subtracting “1” from the face detection accuracy value “3” of all the sections where the detection accuracy value of the previous frame image was “3”. This is because the probability that a face is detected from the same position is considered to decrease with the passage of time. 8C and 8D, similarly, the face detection accuracy value of the frame image decreases with time. If the probability of face detection still exists while the face detection accuracy value ≧ 1, it is set as a face detection candidate region. When the face detection accuracy value = 0, the face detection candidate area is excluded.
As described above, the process of subtracting the face detection accuracy value is performed when the face cannot be detected due to temporary face conditions (direction, brightness, hairstyle, facial expression, etc.). Inertia is given to the setting of the detection accuracy value. As a result, even if a face cannot be detected temporarily, face detection can be performed without losing sight of the face detection candidate area.

FIG. 8 shows an example in which a plurality of sections formed on the screen are scanned with a rectangular area 310 for face detection.
FIG. 8A shows an example in which the i on the row_block_i [1: 0] is i = 20 and the k on the colum_block_k [1: 0] is k = 15. In face detection, the rectangular area 310 is a position where a face is detected.
Data “0” indicates that the face detection accuracy value = 0 and not detected. On the other hand, data “3” indicates that a face is detected in the current frame image with a face detection accuracy value = 3.
When the detection result is stored, the face detection accuracy value of the coordinate value of each section on the screen is not stored, and 00333000... Corresponding to the y coordinate sequence and 00000000003330. Is stored in the controller 205, for example.

FIG. 8B shows, for example, an image in which the face has moved one section downward and to the left in the next frame image. Similarly to FIG. 8A, the rectangular area 310a is a position where a face is detected, and the area surrounded by i = 10 to 12 and k = 4 to 6 is a face because a new face is detected in this frame image. Detection accuracy value = 3. On the other hand, the face is not detected this time and the face is detected in the previous frame image, i = 11 to 13 is “2” from k = 3, and the face detection accuracy value is “3” to “2”. Decrease by “1”. In the region where i = 13 and k = 4, 5, the face detection accuracy value decreases by “1” and the face detection accuracy value becomes “2”. The face detection accuracy value of an area (section) where no face is detected in the previous frame image and the current frame image is “0”.
As a result, 000000003320000000 and y coordinate sequence 002333000000 are stored in the controller 205 corresponding to the x coordinate sequence.

FIG. 8C shows a face movement state in the frame image following FIG. An example is shown in which one block is moved diagonally from the face position in FIG. As a result, the area of i = 11 to 13 and k = 3 further decreases the face detection accuracy value by “1” and becomes the face detection accuracy value = 1, and the area of i = 10 to 12 and k = 4 Since the face is detected in the image and the face is not detected in the current frame image, the face detection accuracy = 2, i = 13, and the region indicated by k = 4 has not been detected in the previous frame image, so the face detection accuracy Value = 1. When i = 10 and k = 5 and 6, the previous face detection was performed, so the face detection accuracy value = 2, and when i = 11 to 13 and k = 5-7, the current face detection was performed, so the face detection accuracy value = 3. .
As a result, the x coordinate string 0000000002333000... And the y coordinate string 001233300000.
FIG. 8D shows an example in which the face is shifted diagonally to the lower right by one section in the frame image over time. As a result of detection in the same manner as in the above-described example, face detection accuracy values of 00000031233300... Corresponding to the x coordinate sequence and 00012333300000... Corresponding to the y coordinate sequence are obtained. Store in the controller 205.

  As described above, by setting the detected face detection accuracy value corresponding to the progress of the frame, the face detection accuracy value in the face detection area is weighted, and the appearance ratio of the detection accuracy is quantified and the accuracy is high. Detection candidate areas can be set.

The operation of the bitmap conversion of the detection candidate area will be described with reference to FIG.
Upon receiving the face detection candidate area data shown in FIG. 9A, the controller 205 expands the detection candidate area data into a detection candidate area bitmap as shown in FIG. 9B.
.. corresponding to the x coordinate sequence, face detection accuracy values 000000000123333000... corresponding to the y coordinate sequence, face detection accuracy values 00012333000... A region surrounded by the face detection accuracy value 12333 obtained from the y coordinate sequence is a detection candidate region.
FIG. 9B is an example in which the image of FIG. 9A is converted into a bitmap. In FIG. 9B, the bit map of the face detection candidate area becomes On (1) because the face detection probability values of the row candidate area data and the column candidate area data are both face detection accuracy values ≧ 1 (bitmap = 1), the bitmap corresponding to all the pixels in the partition is the target. That is, a region surrounded by row_block> = 1 and column_Block_k> = 1 is a detection candidate region. Off (0) is set for a section where the face detection probability value = 0 (bitmap = 0) of any other row candidate area data or column candidate area data.

The setting for tracking return will be described with reference to FIG.
FIG. 10A shows an example of frame images A-02, A-01, A00, A01, A02, A03,... Over time. Of these frame images, the frame image A-02 shows the entire screen. Then, face detection is performed, and detection accuracy of detection is obtained in the frame image A-01.
Further, the frame image A00 shows the operation when the face is not detected with the bit map = 1 of the detection candidate area detected in the previous frame image A00 as a result of the face detection, and the frame image A01 is the previous frame image and “1” of the bitmap. The operation of switching (inverting) the “0” and “0” areas and setting the face detection area to a different area is shown. The frame image A02 shows an operation of scanning a region that is “1” in the frame image A01, newly detecting a face detection candidate region, and obtaining a detection candidate region (bitmap = 1).

Hereinafter, the tracking return operation described above will be described in detail with reference to FIGS. 10 (b) to 10 (d).
As shown in FIG. 10B, in the frame image A00 where the face is not detected, row_bloc_i> = 1, colum_block_k> = 1 represents bitmap = 1, and other areas represent bitmap = 0, It does not exist in the area of bitmap = 0. That is, in the frame image A00, there is no face image in the area of bitmap = 1, and the face image is lost.
As time passes, as shown in FIG. 10C, in the frame image A01 that performs the face detection operation, row_block_i> = 1, colum_block_k> = 1 is bitmap = 0, and other regions are bitmap = 1. This is a value obtained by inverting the bitmap data in FIG. In this inverted bitmap, the face is in the area of this bitmap = 1.
Further, as time passes, as shown in FIG. 10 (d), in the frame image A02 in which the face is detected, the face is detected in the area of bitmap = 1 in the frame image A01, and row_block_1> = 1, colum_block_k> = 1 Bitmap = 1, and other areas are bitmap = 0. That is, even if the face image is lost once, the face can be detected again by inverting the bitmap and newly detecting a face in the inverted area.

  As described above, even if face detection processing is performed on the face detection candidate area, the face condition (orientation, brightness, hairstyle, facial expression, etc.) changes and the face cannot be detected continuously. When a person's face moves instantaneously outside the detection candidate area and the face cannot be detected, the face cannot be detected again within a period that satisfies the face detection accuracy value ≧ 1, and is tracked in the frame image A00. It may be possible to lose sight of the face. In that case, the controller 205 replaces the detection candidate area bitmap of the face frame image A00 which has been On (1) and Off (0) up to the previous frame image, and replaces the bitmap of the previous frame image with the frame image A01 in the current frame image. And perform face detection processing. As a result, the face that has been lost can be detected again from the frame image A02, and tracking of the face detection can be restored.

The face detection tracking operation when a new face appears will be described with reference to FIG.
FIG. 11A shows a frame image with or without face detection over time. The frame image B00 shows an example in which face detection is performed and a new face appears in the frame image B01 following this.

As shown in FIG. 11B, in the frame image B00, the region where row_block_k> = 1, colum_block_i> = 1 indicates bitmap = 1, the other region indicates bitmap = 0, and the face indicates bitmap = 1. Exists in the area.
FIG. 11C shows an example in which one new face appears in the frame image B01 and two faces exist in the frame image B01 as time passes. The data of the area of bitmap = 1 and the area of bitmap = 0 detected for the face in the frame image B00 are inverted, and the area where the face is not detected is changed from bitmap = 0 to bitmap = 1. Only the area where the bit map = 1 is newly detected is detected. In the frame image B01 in which the face is detected, row_block_k> = 1, colum_block_K> = 1 indicates bitmap = 0, and other areas indicate bitmap = 1, and the face also exists in this bitmap = 1 area. A new face detection is performed in the area where the bit map = 1, and only the detection candidate area where the face is detected in the area is row_block_k> = 1, colum_block_K> = 1, otherwise the bit map = 0. .
Further, as time passes, as shown in FIG. 11D, a rectangular area simplified by combining the areas where two faces exist in the face image B02 detected as a face becomes a new detection candidate area, and row_block_k > = 1, colum_block_K> = 1 indicates bitmap> = 1, and other areas indicate bitmap = 0. That is, two faces to which newly appearing faces are added are present in the area indicated by this bitmap = 1.

  In this manner, as shown in FIG. 11, the controller 205 periodically executes On (1) and Off (0) replacement of the detection candidate region bitmap, performs the frame image B01, and newly appears in the image. It is also possible to operate the face so that tracking can be started in the frame image B02.

Next, the operation of the face detection unit 104 will be described using the flowchart of FIG.
For example, image data is input from the image RAM 109 to the scaling unit 201 of the face detection unit 104 via the image bus 113 (ST-100).

  In step ST-101, the scaling unit 201 performs a scaling process for reducing or enlarging the size of the image data (the number of pixels in each of the horizontal direction and the vertical direction) to be suitable for the face detection process in the face detection core unit 203, The image is supplied to the image buffer 202 (see FIG. 3). In addition, as a detection candidate region having a high possibility of including a face, a bitmap of the face detection candidate region calculated by the candidate region processing unit 204 is received from the controller 205, and the scaling process is performed in the same manner as the image. Data is supplied to the image buffer 202.

  In step ST-102, after the scaling process, in order to detect the face area, the rectangular area (detection area) 310 is shifted in the Δx and Δy directions, and the operation process is performed by scanning the screen.

  In step ST-103, it is determined whether or not the area scanned with the rectangular area is within the face detection candidate area. If the result of the determination is not within the detection candidate area, the process proceeds to step ST-105, where Transition to step ST-104.

  In step ST-104, predetermined calculation processing is performed on predetermined pixels in the rectangular image data cut out by the scanning processing in step ST-102 to determine whether the rectangular image data includes a face. This detailed operation has been described with reference to FIGS. 8 to 11 as described above.

  In step ST-105, it is determined whether or not the rectangular area 310 has been scanned over all areas on the display screen. As a result of determination, when not all the regions are scanned, the process proceeds to step ST-102. When all the areas are scanned, the process proceeds to step ST-106.

  It is determined whether or not image data obtained by scanning all areas on the screen has been processed with all sizes. As a result of the determination, when all the sizes have been processed, the process proceeds to step ST-107, and the face detection result is output. As a result of the determination, when image data of all sizes is not processed, the process proceeds to step ST-101 to perform scaling processing.

  In step ST-107, based on the face detection result output by the face detection core unit 203, an image range for performing face detection on the next frame image is calculated as a face detection candidate region. Then, the process proceeds to step ST-102.

  Thus, the amount of calculation can be reduced by detecting a face using face detection candidate area data and a bitmap. Further, the amount of face detection candidate data by dividing the face detection candidate area data can also be reduced. Seamless face tracking can be performed by controlling the face detection candidate accuracy. Furthermore, by controlling the face detection candidate area data, the face tracking can be restored, and a newly appearing face can also be tracked.

As described above, the present invention has the following advantages.
By narrowing the face detection processing to the detection candidate area, the amount of calculation for processing one frame image can be reduced, and as a result, the face detection processing speed can be increased. When the face detection processing speed increases, fine control over time is possible when the face detection result is used for AF (Auto Focus), AE (Auto Exposure), or AWB (Auto White Balance) control. When fine control is possible, a clear image can be captured without missing a momentary photo opportunity.

  By giving inertia to the setting of face detection candidate area data, tracking can be interrupted even if the face is not detected due to a temporary change in face conditions (direction, brightness, hairstyle, facial expression, etc.) Can be prevented. This makes it possible to track face detection seamlessly. When seamless face detection tracking becomes possible, for example, when a child in an athletic meet is photographed, a face can be detected even for a subject moving around outdoors, and AF, AE, If the AWB control is applied, a clear moving image can be taken.

Detected when face condition (direction, brightness, hairstyle, facial expression, etc.) changes for a long time, or when the amount of movement of the face of a moving person is large and the face deviates from the detection candidate area instantly Tracking may be interrupted due to loss of sight. In this case, it is possible to newly set a region other than the region that has been used as a face detection candidate so far, detect a face that has been lost, and return to the tracking process. As a result, the time during which face tracking is interrupted can be minimized.
Further, by changing the detection area according to the amount of movement of the face of the image, it is possible to follow even when the movement speed of the face is fast.
In this way, if the detection result of the face being tracked is applied to AF, AE, and AWB control, the time during which AF, AE, and AWB cannot be controlled can be minimized, and as a result, It is possible to minimize the time when the moving image is unclear.

  By regularly performing the operation of setting areas other than the face detection candidates so far as detection candidates, face detection and tracking processing can be performed even when the number of detected faces increases or decreases. As a result, if AF, AE, and AWB controls are applied to the detection results of a plurality of faces, a clear image can be taken even for moving images with a plurality of subjects.

  Therefore, the present invention performs face detection on two images before and after the temporally arranged frame images in a frame image obtained by cutting out a moving image of a moving person in minute time units, and the position and size of the face. When the amount of change is examined, if the minute time is considered to be sufficiently small, the amount of change is not large. If this is a frame image obtained by cutting out a moving image in minute time units, the peripheral image area including the face detected in the current frame image is set as a candidate area having a high possibility of including the face, and the face is detected in the next frame. If the search target area is selected, the face detection search range can be narrowed down, and as a result, the number of calculations can be reduced.

  In addition, it is considered that the probability that a face is detected at the exact same position as the face position detected in the current frame image decreases with time. This is because the person moves in the frame image, and it is considered that the amount of change from the position where the face is detected increases with time. However, the probability is not zero until a certain time has passed. Accordingly, a process is performed in which a region having an event that a face is detected is set as a candidate region that is highly likely to include a face until a predetermined time elapses. As a result, even if there is a moment when a face is not detected, the face is searched without losing sight of the position and size of the face, and the face can be detected again when the captured image conditions are restored.

  If a face is not detected within a candidate area that is likely to contain a face even after a certain period of time, it is determined that the face outside the candidate area is more likely to contain a face. A process of switching the search target area outside the candidate area is added. As a result, even if the position and size of the face are lost, it is possible to detect the face anew and return to the face detection tracking operation. The reason why the candidate area is exclusively switched between the inside and the outside is to maintain the effect of narrowing down the face detection search range even when the face detection tracking operation is restored.

  In the present invention, the scaling unit that enlarges or reduces the image data on the screen corresponds to the scaling unit. A plurality of sections having a predetermined number of pixels are provided for the image data obtained by the scaling unit, and a rectangular area of a predetermined size is horizontally or horizontally displayed on the screen provided with the plurality of sections. An image detection unit that scans in the vertical direction and detects a specific image in the scaled image data corresponds to the face detection unit. The candidate area processing unit that sets the detection accuracy value according to the degree of overlap of the screen sections of the specific image obtained by the detection corresponds to the candidate area processing unit. A storage unit that stores the detection accuracy value set by the candidate processing unit corresponds to an image buffer.

FIG. 1 is a block diagram of the imaging apparatus. FIG. 2 is a block configuration diagram of the face detection unit of the imaging apparatus shown in FIG. FIG. 3 shows image data obtained by enlarging or reducing the frame image and corresponding bitmap data. FIG. 4 shows an example in which a frame image is scanned horizontally or vertically in a rectangular area having a specific size. FIG. 5 shows image data when the frame image is reduced. FIG. 6 is a diagram showing a plurality of sections in which image data on the screen is composed of a plurality of pixels. FIG. 7 is a diagram illustrating a configuration example of a detection candidate area. FIG. 8 is a diagram for explaining the face detection operation in each frame image. FIG. 9 is a diagram showing a face detection candidate area data and a detection candidate area bitmap when bitmap conversion is performed. FIG. 10 is a diagram for explaining the operation of switching the bitmap data and detecting the detection candidate area in the frame image. FIG. 11 is a diagram illustrating an operation for detecting a new face image. FIG. 12 is a flowchart for explaining an operation of detecting a face detection candidate area in a specific area of an image.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Imaging device 101 ... Optical block 102 ... Signal conversion part 103 ... Camera signal processing part 104 ... Face detection part 105 ... Display processing part 106 ... Image signal processing part 107 ... Memory | storage part 108 ... Display 109, image RAM, 110, control unit (CPU), 111, ROM, 112, RAM, 113, image bus, 201, scaling unit, 202, image buffer, 203, face detection core unit, 204, candidate area processing Part, 205 ... controller, 300, 300a to 300d ... image data, 310, 310a to 310d, 311 ... rectangular area.

Claims (14)

A scaling unit for enlarging or reducing the image data on the screen;
A plurality of sections having a predetermined number of pixels are provided for the image data obtained by the scaling unit, and a rectangular area of a predetermined size is horizontally or horizontally displayed on the screen provided with the plurality of sections. And an image detection unit that scans in the vertical direction and detects a specific image in the scaled image data,
A candidate area processing unit for setting a detection accuracy value according to the degree of overlap of the screen section of the specific image obtained by the detection;
A storage unit for storing the detection accuracy value set by the candidate area processing unit;
An image detection apparatus. The image detection apparatus according to claim 1, wherein the specific image of the image data represents a face. The image detection apparatus according to claim 2, wherein the detection accuracy value varies according to a frame image having the image data that changes with time. The image detection apparatus according to claim 3, wherein the detection accuracy value is a value representing a degree of overlap in a specific range. The image detection apparatus according to claim 4, wherein the detection accuracy value is set with respect to horizontal or vertical coordinates of a plurality of sections on the screen. The detection accuracy value is stored in the storage unit corresponding to each frame image, and is used to detect a specific image that is not detected in the specific frame image based on the stored data in a subsequent frame image. Item 6. The image detection device according to Item 5. The image detection apparatus according to claim 1, wherein a detection candidate area for detecting the specific image is varied according to a movement amount of the specific image of the image data. The image detection apparatus according to claim 1, wherein the bitmap value of the specific image detected in the frame image is inverted, and the specific image is inspected for areas other than the inverted region. The image detection device according to claim 1, wherein the image detection unit detects the section of the entire frame image at a predetermined frame interval. Enlarging or reducing the image data on the screen by scaling processing;
Providing a plurality of sections having a predetermined number of pixels in the image data enlarged or reduced by the scaling process;
Scanning the plurality of sections provided on the screen in a rectangular area of a predetermined size in the horizontal direction on the screen, or in the horizontal and vertical directions;
Detecting a specific image in the scaled image data by scanning the rectangular area;
A step of setting a detection accuracy value according to the degree of overlap of the section corresponding to the specific image obtained by the detection;
An image detection method comprising: storing the set detection accuracy value. The image detection method according to claim 10, wherein the specific area of the image data is face image data, and the detection accuracy value varies according to a frame image having the image data that changes with time. The detection accuracy value is stored in the storage unit corresponding to each frame image, and is used to detect a specific image that is not detected in the specific frame image based on the stored data in a subsequent frame image. Item 15. The image detection method according to Item 10. The image detection method according to claim 10, wherein the bitmap value of the specific image detected in the frame image is inverted and the specific image is inspected for areas other than the inverted region. The image detection method according to claim 10, wherein the image detection step detects the section of the entire frame image at a predetermined frame interval.

Images