JP2013003860A - Object detection device and object detection program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection device for tracking a front face region of an object regardless of a posture of the object.SOLUTION: An object detection device 1 with image acquisition parts 2 and 5 capable of acquiring a static image and a dynamic image composed of and multiple frame images detects an object whose posture has been changed. The object detection device 1 comprises: a reference information generation part 4 for generating a histogram as a template from a static image of the object taken by the image acquisition part 2; and a detection processing part 6 for detecting an object region from a retrieved frame image on the basis of the retrieved frame image among the multiple frame images composing the dynamic image taken by the image acquisition part 5 and the template generated by the reference information generation part 4.

Description

  The present invention relates to an apparatus for detecting an object such as a face or a sign from a moving image, and more particularly to an object detection apparatus and an object detection program capable of detecting the object even when the posture of the object changes.

  In recent years, a face detection function has been installed in many consumer digital cameras. As a conventional face detection technique, Non-Patent Document 1 discloses a technique based on AdaBoost using Haar-like features. This method uses a classifier configured in cascade. Specifically, based on the AdaBoost method (Non-Patent Document 2) that constructs a strong classifier with higher performance by combining arbitrary weak classifiers, it is constructed by connecting a plurality of classifiers in series. .

  Further, Patent Document 1 discloses a method for extracting a specific area in an image that extracts a specific area of a face. This extraction method is a technique that uses matching with a specific template using a genetic algorithm, and extracts a specific region from a search image for a moving image.

JP 2006-12093 A

P. Viola, and M. Jones: “Rapid Object Detection using a Boosted Cascade of Simple Features”, Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.511-518 (2001). Y. Freund, and R. E. Schapire, “A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting”, Journal of Computer and System Sciences, Vol.55, pp.119-139 (1997).

  Although the AdaBoost method can detect with high accuracy even in a complicated background, it is generally limited to a specific posture of the learned object. To solve this problem, a method of learning a plurality of target postures and using a plurality of discriminators as a result has been proposed. However, learning all postures is difficult. Further, increasing the number of discriminators can cope with face detection in all directions, but there are problems that time is required for learning and calculation cost is high.

  In the method of Patent Document 1, a plurality of pixels included in a specific region as a template are compared with a plurality of pixels in a region of the same size in an image to be searched for one-on-one to evaluate suitability. Therefore, when the subject to be observed faces sideways, the size of the face area changes and the number of pixels constituting the face area changes, so that the face area in the case of landscape orientation cannot be extracted.

  Furthermore, in recent years, many consumer digital cameras have been equipped with a face detection function, but as far as the present inventors know, the face is oriented in a direction deviating from the optical axis of the camera. There are no products that are robust. In the entertainment field and security field, face detection technology is an important elemental technology. In order to realize practical face detection, face search that is robust in the direction of the face is necessary.

  In order to solve such a problem, an object of the present invention is to provide an object detection apparatus and an object detection program for detecting an object region regardless of the posture of the object.

In order to achieve the above object, the first configuration of the present invention includes an image acquisition unit that can acquire a still image and a moving image composed of a plurality of frame images, and detects an object whose posture has been changed. A reference information creating unit that creates a histogram as a template from a still image of an object photographed by the image obtaining unit; and a plurality of frame images constituting a moving image photographed by the image obtaining unit. A detection processing unit that detects an object region from the frame image to be searched based on the frame image to be searched and the template created by the reference information creation unit, wherein the detection processing unit is a genetic algorithm Based on the above, the following processing (α1) to (α4) is performed.
(α1) N individuals including parameters for specifying the object region are generated in the search target image.
(α2) Histograms of object regions specified by chromosomal parameters of each individual are created, and the degree of coincidence between these histograms and the template created by the reference information creation unit is evaluated by the fitness function.
(α3) N new individuals are generated by genetic operations based on selection, crossover, and mutation for N individuals.
(α4) The above (α2) and (α3) are repeated until the generation change limit, and the parameter of the individual having the highest fitness among the individuals of the last generation is set as a solution, and the region specified by the solution is defined as the object region to decide.
Here, the object is not limited to a person's face or body part, but also a creature such as an animal, an insect, or a fish, a real estate such as a signboard or a sign fixed on the land, Includes movable goods such as television.

  In the object detection apparatus of the present invention, when the detection processing unit starts processing of the genetic algorithm for the next frame image, the N individuals obtained by the processing of the genetic algorithm for the previous frame image Using the chromosomes of

  The object detection device of the present invention preferably further includes a reference information update unit that updates the template to another template. For example, the reference information update unit has the same histogram of the object region over a plurality of frame images or When the difference is small, the histogram is set in the template.

In order to achieve the above object, a second configuration of the present invention is an object detection program for detecting an object whose posture has been changed, wherein a computer is used as a template from a still image of an object photographed by an image acquisition unit. Search based on a frame image to be searched and a template created by the reference information creation unit among a plurality of frame images constituting a moving image photographed by a reference information creation unit that creates a histogram and an image acquisition unit The detection processing unit functions as a detection processing unit that detects an object region from a target frame image, and the detection processing unit performs the following processes (α1) to (α4) based on a genetic algorithm.
(α1) N individuals including parameters for specifying the object region are generated in the search target image.
(α2) Histograms of object regions specified by chromosomal parameters of each individual are created, and the degree of coincidence between these histograms and the template created by the reference information creation unit is evaluated by the fitness function.
(α3) N new individuals are generated by genetic operations based on selection, crossover, and mutation for N individuals.
(α4) The above (α2) and (α3) are repeated until the generation change limit, and the parameter of the individual having the highest fitness among the individuals of the last generation is set as a solution, and the region specified by the solution is defined as the object region to decide.

  In the object detection program of the present invention, when the detection processing unit starts the process of the genetic algorithm for the next frame image, the N individuals obtained by the process of the genetic algorithm for the previous frame image Using the chromosomes of

  The object detection program according to the present invention preferably causes the computer to function as a reference information update unit that updates the template to another template. For example, the reference information update unit can detect an object region over a plurality of frame images. When the histograms are the same or the difference between them is small, the histogram is set in the template.

  According to the present invention, a face can be tracked by matching using a histogram template. In particular, the face histogram can be tracked by real-time processing because the face histogram is constant regardless of the face posture.

1 is a block diagram of a face detection device according to a first embodiment of the present invention. (A) is a face area specified by the reference information creation unit according to the first embodiment of the present invention, and (B) to (D) are histogram templates. It is a figure which shows the moving image information which the image acquisition part which concerns on 1st Embodiment of this invention produces | generates. It is a figure which shows the structure of the chromosome of the individual | organism | solid which concerns on 1st Embodiment of this invention. It is a block diagram which shows the detection process part which concerns on 1st Embodiment of this invention. It is a figure explaining operation | movement of the face detection apparatus which concerns on 1st Embodiment of this invention. It is a figure explaining operation | movement of the face detection apparatus which concerns on 1st Embodiment of this invention. (A)-(D) are the figures for demonstrating operation | movement with the template image acquisition part and reference information preparation part which concern on 1st Embodiment of this invention. It is a figure explaining operation | movement of the face detection apparatus which concerns on 1st Embodiment of this invention. It is a figure which shows the example of a screen of the display displayed with the frame of the face area specified by the face detection apparatus which concerns on 1st Embodiment of this invention. (A) is an image of a template, and (B) to (K) are diagrams showing a region specified by each individual and a hysteresis of the region. (A) is a template image, (B) is a histogram of the rectangular area of (A), (C) is a rectangular area specified by the detection process, and (D) is a diagram showing a histogram of the rectangular area of (C). is there. (A)-(i) is a figure which shows the image tracking result using Cr component. It is a block diagram which shows the face detection apparatus which concerns on 2nd Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[A. First Embodiment]
FIG. 1 is a block diagram of a face detection apparatus 1 according to the first embodiment of the present invention. The face detection device 1 includes a first image acquisition unit 2, a storage unit 3, a reference information creation unit 4, a second image acquisition unit 5, and a detection processing unit 6.

  The first image acquisition unit 2 acquires image information necessary for face tracking. Specifically, the template image acquisition unit 2 is configured as an imaging device such as a CCD camera or a CMOS image sensor, and acquires, for example, one still image of the face to be tracked. The image photographed by the first image acquisition unit 2 is stored in the storage unit 3 as a reference image for creating a histogram template TP described later.

  The reference information creation unit 4 reads a reference image from the storage unit 3, processes the reference image, and detects a front face. The face detection method in this case uses a cascade classifier based on the AdaBoost method using Haar-like features. After extracting a rectangular area from the image as the front face, a narrower rectangular face area (hereinafter sometimes referred to as a rectangular area) AR shown in FIG. 2A is extracted.

  The reference information creation unit 4 acquires a histogram from the face area AR obtained in this way. In this embodiment, the image data to be used is not the RGB color system but the YCrCb color system. Here, Y represents luminance, Cr represents red color difference, and Cb represents blue color difference. Unlike the RGB color system, the YCrCb color system separates luminance and color. Therefore, the YCrCb color system is used because it can cope with a change in luminance.

  2B shows a Y component histogram relating to the face area AR, FIG. 2C shows a Cb component histogram relating to the face area AR, and FIG. 2D shows a Cr component histogram relating to the face area AR. These histograms are hereinafter referred to as histogram templates TP.

  The second image acquisition unit 5 captures an image to be tracked in real time. Specifically, the second image acquisition unit 5 is configured as an imaging device such as a CCD camera or a CMOS image sensor, and acquires a moving image of a face to be tracked, that is, a plurality of frame images. As shown in FIG. 3, moving image information including a plurality of frame images F is hereinafter referred to as a video sequence VS. In addition, this embodiment may be comprised so that one imaging device may serve for the double use as the 2nd image acquisition part 5 and the 1st image acquisition part 2. FIG.

  The detection processing unit 6 specifies the position of the face to be tracked in each frame image F constituting the video sequence VS created by the second image acquisition unit 5, that is, the target image in a state where the imaging target is stationary. . Specifically, the position of the face area (see AR ′ in FIG. 10) in the target image is specified. Note that the face area in the target image (hereinafter sometimes referred to as a rectangular area) is distinguished from the rectangular area AR (see FIG. 2A) used when creating the histogram template TP. A description will be given below with AR ′.

  The detection processing unit 6 performs face detection processing based on the frame image F created by the second image acquisition unit 5 and the histogram template TP stored in the storage unit 3. As will be described in detail later, the detection processing unit 6 performs detection processing based on a genetic algorithm (hereinafter sometimes referred to as GA).

  In the detection process of the present embodiment, the face area on the target image, that is, the face area AR ′ to be tracked in the image is represented by a parameter, and template matching with the face area AR ′ specified by this parameter is an optimization problem. As a solution. Specifically, the matching between the histogram of the face area AR ′ specified by the parameter and the histogram template TP is evaluated.

  As parameters, coordinates, size, and rotation angle representing the center of the rectangular area AR ′ that is the search target are used. In the detection process, the original rectangular area AR or the rectangular area AR ′ based on the original rectangular area AR is geometrically transformed on the target image using the parameters, and a new rectangular area AR ′ is specified. In this embodiment, a genetic algorithm is used as a solution for optimizing matching between the histogram of the pixels on the rectangular area AR ′ as a result of conversion and the histogram template TP.

As the chromosome structure of each individual in the genetic algorithm, as shown in FIG. 4, the chromosome CH of this embodiment includes the center coordinates (c _x , c _y ) of the original rectangular area AR or the rectangular area AR ′ before conversion. When, and with the x-axis of the frame demarcating the rectangle and the y-axis direction of the scaling factor m _x, and m _y, the information of the rotation angle angle of the frame demarcating the rectangular. These are finally obtained solutions and are parameters representing the position, size, and rotation angle of the rectangular area AR ′ specified as the face area on the target image to be searched. The possible ranges of these parameters are selected as (1) to (3) below in this embodiment.

The enlargement / reduction ratio was determined in consideration of the size of the template, and the rotation angle was determined in consideration of the angle at which the face can bend in daily life. Each parameter c _x in this embodiment, expressed as _{c y, m x, m y} , respectively angle 8bit, chromosome CH of one individual is represented by a sum of 40bit.

Detection processing unit 6, these parameters _{c x, c y, m x} , m y, respectively genetic information angle handled as (hereinafter. May be referred to as gene). In particular, in order to solve the optimization problem, the detection processing unit 6 appropriately generates a next-generation individual that inherits these parameters as genetic information using a genetic algorithm. The detection processing unit 6 is identified in the generated next generation individuals are identified i.e. each gene (parameter) c _x constituting a chromosome CH _{individuals, c y, m x, m} y, by angle the histogram of the rectangular area AR ', compared to the histogram template TP, the parameters c _x as a _{solution, c y, m x, m} y, to evaluate the suitability of the angle.

  Therefore, as shown in FIG. 5, the detection processing unit 6 includes a preprocessing unit 61, a genetic operation unit 62, a coordinate conversion unit 63, a fitness calculation unit 64, and a determination unit 65.

The preprocessing unit 61 generates N individuals, that is, N chromosomes CH (see FIG. 4). Here, numerical values of the parameters _{c x, c y, m x} , m y, angle constituting each chromosome CH is randomly selected within the range of (1) to (3). In the present embodiment, the number N of individuals is set to 10, but is not limited to that number.

  The genetic operation unit 62 operates after selecting one of the N individuals or by operating the selected two individuals to generate N new individuals. Specifically, the genetic operation unit 62 performs three operations, selection (淘汰, regeneration), crossover, and mutation, as genetic operations for individuals.

Here, crossing the, each genetic information constituting an individual chromosome (see Fig. 4), an operation to replace words parameter _{c x, c y, m x} , m y, the angle, in particular selected This operation replaces the genetic information of two parent individuals. is there. The mutation, a part of the genetic information composing the chromosome, i.e. the parameter c _x, c _y, a m _{x, m y,} operation of changing a part of the angle, the operation in particular changing to a different numerical is there. The numerical value specifying the parameter by mutation is selected at random.

  In this embodiment, the crossover rate is selected to be 0.7 and the mutation rate is set to 0.05, but it is not limited to those values. The crossover method is uniform crossover, the selection method is roulette selection, and the genetic algorithm is set based on the elite preservation strategy. However, the crossover method and the selection method are not limited to this. In the present embodiment, as an elite preservation strategy, one individual with the highest fitness described below (hereinafter referred to as an elite individual) is left in the next generation.

  The genetic information can be manipulated up to 40 times per target image captured by the second image acquisition unit 5, that is, one frame image F, but the generation change number G is not limited to 40 times.

Here, the coordinates of the rectangular region AR ′ specified based on the parameters c _x , c _y , m _x , m _y , and angle constituting the individual CH generated by the genetic operation unit 62, that is, the chromosome CH of the individual are: It is expressed as the following coordinate transformation. Here, all geometric transformations use homogeneous coordinates in order to reduce the complexity of the transformation matrix. Assuming that the point P is the coordinate of the face area on the image before conversion and the point P ^* is the coordinate of the point P after conversion specified by the generated individual, when these points are expressed in homogeneous coordinates, It is represented by (4) and (5).

Furthermore, the point P ^* is expressed by the following equation (6).

  The coordinate conversion unit 63 converts the coordinate point P before the conversion based on the equation (6), and specifies the position of the search target rectangular area AR ′ in the target image. In the case of the first frame, the point P is the coordinate of the rectangular area AR (see FIG. 2A) extracted from the reference image acquired by the first image acquisition unit 2 of FIG. From the next frame, the coordinates of the rectangular area AR ′ specified by the previous frame image.

The fitness calculation unit 64 performs a matching process between the histogram specified by the pixels in the rectangular area AR ′ whose location has been moved by the coordinate conversion unit 63 and the histogram template TP. Specifically, the histogram of the rectangular area AR ′ on the target image is calculated, and the adaptability between the histogram of the rectangular area AR ′ and the histogram of the template TP, that is, the degree of similarity between the two is determined. Note that, when the magnification of the image is changed, the rectangular area AR ′ differs from the size of the face area AR from which the histogram template TP is created, but this is used when creating a histogram of the rectangular area AR ′. The number of pixels in the rectangular area AR ′ is adjusted to the number of pixels included in the face area AR on which the histogram template TP is created. For example, when the face area AR is 10 × 10 pixels and the rectangular area AR ′ is 20 × 20 pixels, the histogram of the rectangular area AR ′ does not use all 400 pixels, but is arranged, for example, every other pixel. It is created using 100 pixels that are the same as the number of pixels of the face area AR.
The fitness fitness between the histogram of the rectangular area AR ′ and the histogram of the template TP is expressed using fitness functions (7) and (8) shown below.

ρ _i is the similarity of the histogram in each component of the color system (i = 1 represents the Y component, i = 2 represents the Cb component, and i = 3 represents the Cr component), m is the number of histogram bins, and p is The histogram of the histogram template TP, q is the histogram of the rectangular area AR ′ on the target image (the frame image to be observed), and N is the number of pixels of the rectangular area AR from which the histogram template TP was created. The higher the fitness, the more similar to the histogram of the template TP. Note that the Bhattacharyya coefficient is used in the calculation of the similarity of the histogram in equation (7).

The determination unit 65 determines which individual specifies the face area to be tracked most from the result calculated by the fitness calculation unit 64 for each of the N individuals. Specifically, the determination unit 65 determines which individual has the highest fitness. The individual with the highest fitness based on the elite conservation strategy is inherited by the next generation with the same genetic information. Further, when the generational change number G is the maximum, for example, in the 40th time of this embodiment, it is determined that the individual with the highest fitness has specified the face area to be tracked. Parameter c _x thus configuring the chromosome CH of selected individuals _{eventually, c y, m x, m} y, based on the angle, to identify the target of the face area AR '.

  The face detection apparatus 1 described above is constituted by a computer, for example. This computer implements the above-described method, that is, the face detection process, by executing a face tracking program as software installed in advance. Specifically, when the computer executes the detection processing program, the computer executes the above-described reference information creation unit 4, the detection processing unit 6, in particular, the preprocessing unit 61, the genetic operation unit 62, the coordinate conversion unit 63, the fitness calculation. Functions as the unit 64 and the determination unit 65.

  A plurality of computers are connected to each other via a LAN, the Internet, a public network, etc., and a reference information creation unit 4, a detection processing unit 6, particularly a preprocessing unit 61, a genetic operation unit 62, a coordinate conversion unit 63, an adaptation The operations of the degree calculation unit 64 and the determination unit 65 may be distributed by a plurality of personal computers. A computer having a conventionally known configuration can be used. The computer is stored in a storage device such as a RAM, a ROM, or a hard disk, an operation device such as a keyboard or a pointing device, and an instruction from the operation device. A central processing unit (CPU) for processing data and software, a display for displaying processing results, and the like. This computer may be a general-purpose device or a dedicated device.

Next, the operation of the face detection apparatus 1 according to this embodiment will be described.
As pre-processing for the face detection device 1 to perform detection processing, a histogram template TP as reference information is acquired in step S1 shown in FIG. Specifically, as shown in FIG. 7, in this process, the first image acquisition unit 2 acquires an input image F1 (see FIG. 8A) necessary for face tracking in step S11. In step S12, the reference information creation unit 4 processes the input image F1, that is, the reference image, that is, detects the front face R (see FIG. 8B) based on the AdaBoost method using the Haar-like feature. To do. Further, in step S13, the reference information creation unit 4 extracts a rectangular face area AR (see FIG. 8C) narrower than the rectangular area as the front face R by the AdaBoost method, and a histogram (FIG. 8 (D)) is calculated to create a histogram template TP (step S14).

After the above pre-processing is completed, the face detection device 1 executes detection processing. First, as shown in FIG. 6, in step S2, the preprocessing unit 61 initializes the genetic algorithm. This initialization setting is performed only when the face detection apparatus 1 starts the detection process. In this initialization setting, the parameters c _x specifying the rectangular area _{AR ', c y, m x} , m y, generates an individual of N to genetic information (for example, N = 10.) The angle. Numerical values are randomly selected for each parameter constituting the chromosome of each individual within the above ranges (1) to (3).

  Next, in step 3, the second image acquisition unit 5 acquires search target image information F (see FIG. 3), and in step S4, the detection processing unit 6 performs matching of the histogram template TP by GA.

In the matching of the histogram template TP by GA, first, as shown in FIG. 9, each of the N individuals generated by the preprocessing unit 61 is evaluated in step S41. Specifically, it calculates the genetic information of a chromosome CH individuals preprocessing unit 61 has generated, i.e. the parameters _{c x, c y, m x} , m y, a histogram of the rectangular region AR in the 'specified by angle Then, the degree of matching between this histogram and the histogram template TP is calculated. At this time, the fitness is calculated based on the fitness function shown in the equations (7) and (8).

Next, in step S42, the determination unit 65 of the detection processing unit 6 determines whether each individual satisfies the termination condition. More specifically, it is determined whether the number of genetic operations for individuals, that is, generational changes, reaches the maximum number G. When the maximum generation alternation number G is reached, an individual having the highest fitness is selected from N individuals in the final generation, and genetic information of the individual, that is, parameters c _x , c _y , m _x , m Treated as a solution to find _y and angle.

  If such an end condition is not satisfied, in step S43, the genetic operation unit 62 performs a genetic operation based on each individual, in other words, exchanges parameters. The genetic manipulation unit 62 generates N new individuals by causing selection (selection, regeneration), crossing, and mutation to N individuals with a certain probability. Also, based on the elite conservation strategy, the elite individuals will remain in the next generation.

  In this way, the genetic operation unit 62 newly generates N individuals. Then, it is determined whether the newly generated individual satisfies the end condition (from step S43 to step S42). At this time, also for a new individual, a histogram in the rectangular area AR ′ is calculated, and the degree of matching between this histogram and the histogram template TP is calculated based on the fitness function shown in equations (7) and (8). .

  Until the end condition is satisfied in step S42, the genetic operation and fitness evaluation in step S43 are repeated. In the present embodiment, the generation change number G per frame as the image information is set to 40 times.

If the end condition is satisfied in this manner by repeating the generation change was performed genetic operations, among the last generation of the population, the largest individual parameters c _x is _{fitness, c y, m x, m} y, As a solution for obtaining the angle, the rectangular area AR ′ specified by the parameter is displayed on the display as shown in FIG. 10, for example.

  Note that each piece of information of the N individuals of the last generation is stored in the storage unit 3 so as to be used as it is when the processing of the next frame image F is started.

  As described above, when face tracking processing is completed for one frame of image information, that is, the area estimated as a face area can be surrounded by a rectangular frame, the next video sequence stored as a moving image in the storage unit 3 A frame image F (see FIG. 3) is read out as image information, and a detection process is performed on the next frame image F (from step S4 to step S3 in FIG. 6). When the detection process for the next frame image F is started, N individuals created as the final generation in the detection process of the previous frame image without performing solid initialization, that is, without recreating the N individuals. Is used as the first generation population in the next frame image.

  Thus, according to the face detection apparatus 1 according to the present embodiment, a face can be tracked by matching using the histogram template TP. In particular, the face detection apparatus 1 can track the face area AR ′ by real-time processing because the face histogram is constant regardless of the posture of the face. Furthermore, the initialization of individuals is performed only once in the initial frame, and the number of individuals and the number of generations are changed by inheriting the individual that has evolved in the previous frame and each parameter that is genetic information for determining fitness to the next frame. Therefore, calculation cost can be reduced and accuracy can be improved.

[B. Experimental example]
Regarding the embodiment of the present invention, the effectiveness of the histogram and each color system will be described below.

B1: Experiment 1. Effectiveness of histogram [B1-1. Experiment contents]
The similarity between the histogram of the rectangular area AR ′ of the target image extracted by the face detection process and the histogram template TP is confirmed, and the relationship between the degree of similarity and face tracking is investigated.

[B1-2. System settings]
The GA parameters used in the experiment were set as follows. The number of individuals was 10, the crossover rate was 0.7, the crossover method was uniform crossover, the mutation rate was 0.05, the selection method was roulette selection, and an elite conservation strategy was used. In addition, the number of generation changes in one frame image is 40 times. A computer equipped with a 3.2 GHz CPU was used for the experiment.

[B1-3. Evaluation method]
Regarding the rectangular area AR ′ of one frame acquired by the face detection device 1 of this system, the similarity between the histogram of the rectangular area AR ′ and the template histogram TP is confirmed, and it is evaluated whether the histogram patterns are similar. Further, the relationship between the histogram matching degree, the position of the face area specified by the system, and the histogram is evaluated from the image created by the system.

[B1-4. Experimental result]
FIG. 11A shows a histogram template TP. FIGS. 11B to 11K show the results of processing the frame by the system, that is, the results showing the 10 individuals (first to 10th individuals) remaining as the final generation in the detection processing related to the frame image. The right area of FIG. 2 shows the rectangular area AR ′ specified by the system, and the left side shows the histogram of the rectangular area AR ′.

When the histogram template TP in FIG. 11A and the histogram in FIG. 11C are compared, it can be confirmed that the histogram patterns are substantially similar. As shown in the right area of FIG. 11C, the rectangular area AR ′ is selected almost accurately in the face area of the subject.
On the other hand, when the histogram of the first individual in FIG. 11B is compared with the histogram template TP in FIG. 11A, the shapes of the patterns of the two histograms are different and a similar shape portion is not found. When the shapes are different in this way, the face area of the subject and the rectangular area AR ′ are shifted as shown in the right area of FIG. Further, in the eighth individual having a histogram whose shape is greatly different from the pattern of the template TP, as shown in FIG. 11G, the rectangular area AR ′ set by the system is at a position greatly deviated from the face of the subject.
Here, Table 1 below shows the distance for evaluating the degree of similarity between the histogram of each individual CH acquired by the system and the histogram template TP.


This distance was calculated from the following equation (9) that expresses the similarity between two histograms as the distance d. The shorter the distance, the more similar, and the longer the distance, the dissimilar.


Here, H ₁ is the histogram of the individual, H ₂ is the histogram of the template TP, and I is the number of bins.
From Table 1, it was confirmed that the distance d was the longest for the individual of the individual number 6 (see FIG. 11G) whose pattern of the histogram is almost similar. On the other hand, it was confirmed that the distance of the individual of the individual number 2 (see FIG. 11C) having a substantially similar histogram pattern was the shortest.
From the above, it was confirmed that face recognition by matching of the histogram template TP is effective, that is, the histogram template TP is effective.

B2: Effectiveness of each color system [B2-1. Experiment contents]
The target image is a moving image sequence obtained by using a Web camera and a subject shaking his / her face up / down / left / right with respect to the Web camera. The size of the target image is 320 × 240 pixels, and the total number of frames is 180. The front face used for acquiring the histogram template TP was a 16 × 21 pixel image as the same subject image.

  In this embodiment, since a histogram is used, it is conceivable that the result depends greatly on the combination of the color system components. Therefore, the effectiveness of detection of each color system component by experiment of 14 patterns of color system components of YCbCr, YCr, YCb, CbCr, Y, Cr, Cb, HSV, HS, HV, SV, H, S, V. Confirm.

  As the evaluation criteria, the center coordinates of the face are determined as correct coordinates on the target image by visual observation in advance. Correct / incorrect determination is performed using the distance between the center coordinates of the detected rectangular area AR ′ and the correct coordinates, which is the experimental result.

[B2-2. System settings]
The GA parameters used in the experiment were set as follows. The number of individuals was 10, the crossover rate was 0.7, the crossover method was uniform crossover, the mutation rate was 0.05, the selection method was roulette selection, and an elite conservation strategy was used. In addition, the number of generation changes in one frame image is 40 times. A computer equipped with a 3.2 GHz CPU was used for the experiment.

[B2-3. Evaluation method]
The coordinates that can be determined as the center of the face with respect to the target image in advance, that is, the correct coordinates are determined visually. Then, the correctness determination is performed by comparing the center coordinates of the rectangular area AR ′ detected from the experimental results, that is, the result coordinates with the correct coordinates.

Since the distance between the correct coordinates and the result coordinates is affected by the size of the face, normalization is performed according to the size of the image of the face area AR from which the histogram template TP is created, not a simple distance. The following formula (10) is used as the calculation formula.

  Here, A represents the correct coordinates, R represents the result coordinates, and width and height represent the width and height of the image of the face area AR from which the histogram template TP was created. In the experiment, in addition to the face area height (distance from chin to eyebrows) of the target image being about 50 pixels, the result coordinates and correct coordinates are used to make the detection failure when detecting the skin color neck If the distance between and is within 10 pixels, a criterion of correct answer was set.

[B2-4. Experimental result]
The histogram of the rectangular area AR (FIG. 12B) which is the template image of FIG. 12A and the histogram of the rectangular area AR ′ identified by the face detection processing of FIG. 12C (FIG. 12D). As shown in FIG. 12, it can be seen that the histogram shapes of the three components, that is, the Y component, the Cr component, and the Cb component are similar. From this, it was found that the GA proposed in the present embodiment can perform matching by evaluating the histogram.

In addition, using the same random number seed, the experiment was conducted using all 14 patterns. As a result, the experiment was carried out by adding four more random seeds to the top six patterns with an accuracy of 70% or more. The results are shown in Table 2 below.

The upper four patterns are Cr, YCr, CrCb, and YCrCb components or combinations, and these are patterns that include the Cr component in 14 patterns. This shows that the Cr component is important. The result with the highest accuracy was the result of using only the Cr component.
Results images using the Cr component are shown in FIGS. It was confirmed that it was robust to changes in the face direction, such as when it turned to the side.

[C. Second Embodiment]
FIG. 14 is a block diagram showing a face detection apparatus 1A according to the second embodiment of the present invention. In addition to the configuration of the face detection device 1 according to the first embodiment described above, the face detection device 1A includes a reference information update unit 7 as shown in FIG. The same components as those of the first embodiment described above are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the present embodiment, the same histogram template TP is not used in the tracking process from the first frame image to the last frame image, but in some cases, the histogram template TP is updated to another histogram template TP in the middle. It is characterized by.

For this reason, in this embodiment, the reference information update part 7 is provided.
When it is confirmed that the subject to be imaged has the same posture over several frame images, the reference information update unit 7 uses a histogram that can be extracted with the posture as a histogram template TP to be used for subsequent tracking. Set.
The reference information update unit 7 replaces the histogram template TP when the histograms are the same over 10 frames. The case where the histograms are the same corresponds to the case where the differences are small as well as the case where they match, as will be described later.

  The reference information updating unit 7 evaluates the similarity of the histograms of the preceding and succeeding frame images based on the distance d of the above-described equation (9). Specifically, the distance d between the histograms of the preceding and following frame images is calculated, and it is further determined whether the distance d between the next frame image and the next frame image is the same as the distance d between the previous frame images. . In the present embodiment, for example, this sameness determination is made by evaluating that the range of error ± 0.05 is the same. The range of error is not limited to this numerical value. For example, significant digits may be set to evaluate the complete coincidence of numerical values as the same. When the reference information updating unit 7 determines that the histograms are the same over 10 frames, the histogram of the rectangular area AR ′ specified by the individual selected in the final generation of the GA processing for the frame image 10 frames before is displayed. Hereinafter, it is handled as a histogram template TP.

  After the reference information updating unit 7 exchanges the histogram template TP, the detection processing unit 6 performs a histogram matching process based on the newly set histogram template TP.

  The face detection apparatus 1A described above is composed of a computer, for example. The computer implements the above-described method, that is, the face detection process by executing a face tracking program as software installed in advance. Specifically, when the computer executes the face detection processing program, the computer executes the above-described reference information creation unit 4, detection processing unit 6, particularly pre-processing unit 61, genetic operation unit 62, coordinate conversion unit 63, fitness level It functions as a calculation unit 64, a determination unit 65, and a reference information update unit 7.

  As described above, according to the face detection processing apparatus 1A according to the second embodiment of the present invention, the reference information update unit 7 can replace the histogram template 7 with a template corresponding to the use environment. For example, when the surroundings of the subject to be imaged become dark or bright, the histogram of the subject's face area changes according to the brightness, so the histogram pattern TP as reference information is adjusted according to the usage environment. Can do. Thereby, the tracking accuracy of the face can be improved.

[D. Other Embodiments]
As described above in detail, the present invention can be implemented in various forms without departing from the spirit of the invention.
In the above embodiment, the reference information update unit automatically creates a histogram of the face area AR using a cascade classifier based on the AdaBoost method using Haar-like features, and uses this histogram as a template TP. However, the histogram of the template is not necessarily limited to the front face of the subject. For example, the template TP at the time of initial setting may be automatically created from a profile, an upward face, a downward face, and the like. In this case, a cascade classifier constituting the reference information update unit is constructed so as to create a histogram according to the orientation of those faces, that is, the posture of the subject.
The numerical values defining the ranges (1) to (3) of the parameters constituting the chromosome are examples.
In the above description, the detection processing device displays the face area on the display as a calculation result on the display, but such display on the display may be omitted.
Moreover, the parameters as genetic information constituting the chromosome are not limited to the above, and some of them may be omitted, and information such as three-dimensional rotation may be used as genetic information.
In genetic operation, selection is not limited to roulette selection, ranking selection or tournament selection is used, and the crossover method is not limited to uniform crossover, and one-point crossover, two-point crossover, or multipoint crossover may be used.
The number of bits representing a chromosome is not limited to 40 bits.
In the above-described embodiment, three elements, that is, the Y component, the Cr component, and the Cb component are used as the histogram. However, histogram matching in face tracking may be performed using a histogram of only the Cr component.
In the determination of the identity of the histogram in the reference information update unit of the second embodiment, the number of frames in which the same histogram continues as the replacement condition of the histogram template TP is not limited to ten frames. The determination of identity may be processed by time. For example, it may be exchanged when the histograms are the same for several seconds.
In the first embodiment and the second embodiment described above, the tracking target is specified as the face of the subject who is the imaging target, but the imaging target is not only a person but also a living thing such as an animal, as well as a signboard or sign fixed on the land. Such a property may be a movable property such as a portable car or a television, and any of them may have a front or side as an object. If the observation object changes in this way, the color component as the image feature amount changes, so the color system is changed according to the color of the front of the object.
In the above-described embodiment, the configuration in which the genetic operation is repeated until the end condition is satisfied has been described. However, the end condition may be a case where the fitness of a certain individual exceeds a predetermined value. For example, in the fifth generation before reaching the maximum generation change number G, if there is an individual whose fitness exceeds a certain threshold in the population, the generation change, that is, the subsequent genetic operation is not performed, Among individuals, the genetic information of the individual having the highest fitness is handled as a solution, and the N individuals of the fifth generation are handled as the first generation individuals of the next frame image.
The present invention performs a process of extracting the front area of an object such as a face from a moving image, but is not limited to sequentially processing the frames constituting the moving image, for example, a detection process is performed every few frames. May be.

1, 1A Face detection device 2 First image acquisition unit 3 Storage unit 4 Reference information creation unit 5 Second image acquisition unit 6 Detection processing unit 7 Reference information update unit 61 Preprocessing unit 62 Genetic operation unit 63 Coordinate conversion unit 64 Fitness Calculation unit 65 Judgment unit AR, AR ′ Facial region CH Chromosome
c _x component of _x center coordinate
c _y- axis y-axis tax m _x x-axis frame scaling factor _y y y-axis frame scaling factor
Angle Rotation angle of the frame that draws the rectangle F Frame P Coordinates before conversion
P ^* Coordinate TP after conversion Histogram template VS Video sequence

Claims (10)

An object detection device that has an image acquisition unit capable of acquiring a still image and a moving image composed of a plurality of frame images and detects an object whose posture has been changed,
A reference information creation unit that creates a histogram as a template from a still image of an object photographed by the image acquisition unit;
Based on the frame image to be searched and the template created by the reference information creation unit among the plurality of frame images constituting the moving image taken by the image acquisition unit, the object region is extracted from the frame image to be searched. A detection processing unit for detecting,
The object detection apparatus, wherein the detection processing unit performs the following processes (α1) to (α4) based on a genetic algorithm.
(α1) N individuals including parameters for specifying the object region are generated in the search target image.
(α2) Histograms of object regions specified by chromosomal parameters of each individual are created, and the degree of coincidence between these histograms and the template created by the reference information creation unit is evaluated by the fitness function.
(α3) N new individuals are generated by genetic operations based on selection, crossover, and mutation for N individuals.
(α4) The above (α2) and (α3) are repeated until the generation change limit, and the parameter of the individual having the highest fitness among the individuals of the last generation is set as a solution, and the region specified by the solution is defined as the object region to decide.   The detection processing unit uses the chromosomes of N individuals obtained by the genetic algorithm processing for the previous frame image when starting the genetic algorithm processing for the next frame image. The object detection device according to claim 1.   The object detection apparatus according to claim 1, further comprising a reference information update unit that updates the template to another template.   The object detection according to claim 3, wherein the reference information update unit sets the histogram in the template when the histogram of the object region is the same or the difference between the plurality of frame images is the same. apparatus.   The object detection apparatus according to claim 1, wherein the histogram is created based on a Cr color system representing a red color difference. An object detection program for detecting an object whose posture has been changed,
Computer
A reference information creation unit that creates a histogram as a template from a still image of an object photographed by the image acquisition unit;
An object region is detected from a frame image to be searched based on a frame image to be searched and a template created by the reference information creating unit among a plurality of frame images constituting a moving image photographed by the image acquisition unit. A detection processing unit,
Function as
An object detection program, wherein the detection processing unit performs the following processes (α1) to (α4) based on a genetic algorithm.
(α1) N individuals including parameters for specifying the object region are generated in the search target image.
(α2) Histograms of object regions specified by chromosomal parameters of each individual are created, and the degree of coincidence between these histograms and the template created by the reference information creation unit is evaluated by the fitness function.
(α3) N new individuals are generated by genetic operations based on selection, crossover, and mutation for N individuals.
(α4) The above (α2) and (α3) are repeated until the generation change limit, and the parameter of the individual having the highest fitness among the individuals of the last generation is set as a solution, and the region specified by the solution is defined as the object region to decide.   The detection processing unit uses the chromosomes of N individuals obtained by the genetic algorithm processing for the previous frame image when starting the genetic algorithm processing for the next frame image. The object detection program according to claim 6. A reference information update unit that updates the computer with another template;
The object detection program according to claim 6, wherein the object detection program is made to function as:   9. The object detection according to claim 8, wherein the reference information update unit sets the histogram in the template when the histogram of the object region is the same or the difference between the plurality of frame images is the same. program.   The face tracking program according to claim 6, wherein the histogram is created based on a Cr color system representing a red color difference.