JP2013003890A - Image processing system, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable more accurate detection of an edge of an upper portion of a head, which is a head characteristic.SOLUTION: An image processing system comprises: means for detecting an edge pixel in a preset head detection area; means for accumulating the edge pixel in each ellipse in an ellipse parameter space of Hough transform; means for selecting candidate ellipses according to the number of edge pixels; and means for selecting an ellipse of a head from among the candidate ellipses using a total index formed by ellipse indexes, so that an edge of an upper portion of a head, which is a head characteristic, can be detected with higher accuracy.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program, and more particularly to a technique suitable for use in detecting a head region from an image.

  In recent years, the function of controlling the camera focus, exposure, and camera pan, tilt, zoom, and other postures by identifying and tracking the position of a designated person in a video shot from a camera or other imaging device Is attracting attention. In such a function, in order to specify the position of a person, it is common to detect a person's face pattern and track its movement. As a technique for detecting a face from an image in this manner, various methods are disclosed in Non-Patent Document 1. In particular, due to the high execution speed and high detection rate, the method described in Non-Patent Document 2 is widely used in face detection research.

  On the other hand, in order to specify the position of a person, it is not sufficient to detect and track the face pattern of the person. In some cases, a face pattern cannot be detected due to a horizontal rotation or reverse inversion of a human face. Therefore, it is an effective method to detect and track the head region instead of the face.

  As a method for detecting the head region, for example, a method disclosed in Non-Patent Document 3 is known. In recent years, as disclosed in Non-Patent Document 4, the ellipse shape of the head is also detected by gradient information on the ellipse circumference and color histogram matching inside the ellipse.

M.M. H. Yang, D.D. J. et al. Kriegman and N.M. Ahuja. “Detecting Faces in Images: A Survey,” IEEE Trans. on PAMI, vol. 24, no. 1, pp. 34-58, January, 2002. P. Viola and M.M. Jones. “Robust Real-time Object Detection,” in Proc. of IEEE Worksshop SCTV, July, 2001. Duda, R.A. O. and P.M. E. Hart, 'Use of the Hough Transformation to Detect Lines and Curves in Pictures,' Comm. ACM, Vol. 15, pp. 11-15 (January, 1972). Stan Birchfield, "Elliptical Head Tracking Using Intensity Gradients and Color Histograms," Proc. IEEE International Conference On Computer Vision and Pattern Recognition (CVPR '98), Santa Barbara, California, pp. 199 232-237, June 1998

  In an image including a person, there are many upper body edges below the head of the person and there are individual differences, and the edge above the head is relatively stable and is a feature of the head. In addition, there are many horizontal edges and vertical edges in the background area. If an attempt is made to detect a head ellipse using a conventional method, these edges may be detected as a head ellipse, and there may be a case where an edge above the head, which is a head feature, cannot be detected correctly. Further, the combination of the background edge and the head edge may be detected as a large elliptical arc, and the upper edge of the head may not be detected correctly.

  An object of the present invention is to make it possible to detect the upper edge of the head, which is a head feature, with higher accuracy in view of the above-described problems.

  The image processing apparatus according to the present invention includes means for detecting edge pixels in a set head detection region, means for accumulating the edge pixels in each ellipse in an elliptic parameter space in the Hough transform, and the number of edge pixels. And a means for selecting a candidate ellipse based on the candidate ellipse, and a means for selecting a head ellipse from the candidate ellipse using a general index composed of ellipticity indices.

  ADVANTAGE OF THE INVENTION According to this invention, the edge above the head which is a head characteristic can be detected with higher precision.

It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 1st Embodiment. It is a flowchart which shows an example of the process sequence in 1st Embodiment. It is a figure which shows the setting of the change range of the ellipse parameter of Hough transform. It is a figure which shows the weight setting by the vertical position (horizontal position) of an edge pixel. It is a block diagram which shows the hardware structural example of the image processing apparatus in embodiment. It is a figure which shows the weight setting of the edge pixel by the position in an ellipse rotation direction. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 3rd Embodiment. It is a flowchart which shows an example of the process sequence in 3rd Embodiment. It is a figure which shows the circular arc length of the elliptical circular arc which an edge pixel occupies. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 4th Embodiment. It is a figure which shows the calculation procedure of the "pixel number / circumference length" parameter | index in 4th Embodiment. It is a figure which shows the calculation procedure of the "pixel number / circular arc length" parameter | index in 4th Embodiment. It is a flowchart which shows an example of the process sequence in 4th Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 5 is a block diagram illustrating a hardware configuration example of the image processing apparatus according to the present embodiment.
In FIG. 5, reference numeral 1001 denotes a CPU, which executes various controls in the image processing apparatus of the present embodiment. Reference numeral 1002 denotes a ROM which stores a boot program executed when the image processing apparatus main body is started up and various data. Reference numeral 1003 denotes a RAM, which stores a control program for processing by the CPU 1001 and provides a work area when the CPU 1001 executes various controls. A keyboard 1004 and a mouse 1005 provide various input operation environments for the user.

  Reference numeral 1006 denotes an external storage device, which includes a hard disk, flexible disk, optical disk, magnetic disk, magneto-optical disk, magnetic tape, and the like. However, the external storage device 1006 is not necessarily a necessary component if the ROM 1002 has all the control programs and various data. Reference numeral 1007 denotes a display, which is composed of a display or the like, and displays the result and the like to the user. Reference numeral 1008 denotes a network interface. Reference numeral 1009 denotes a video interface, which makes it possible to capture a frame image that is separated from the imaging unit 1004 and a coaxial cable. Reference numeral 1011 denotes a bus for connecting the above components.

FIG. 1 is a block diagram illustrating a functional configuration example of an image processing apparatus 100 that detects a head region of a person in the present embodiment.
In FIG. 1, the head detection area setting unit 102 sets a large area including a human head as a head detection area for the input image 101. The edge detection unit 103 performs edge detection on the head detection region set by the head detection region setting unit 102 using an edge detection operator, and obtains an edge image of the head detection region.

  The parameter setting unit 104 sets the change range of each parameter of the head ellipse based on the head detection region set by the head detection region setting unit 102. Specifically, the minimum value and the maximum value of x0 and the minimum value and the maximum value of y0 are set as the change range of the center coordinates (x0, y0) of the ellipse. Further, the minimum value and the maximum value of the elliptical vertical axis length b and the minimum value and the maximum value of the elliptical horizontal axis length a are set as the change ranges of the elliptical vertical axis length b and the elliptical horizontal axis length a. Further, a minimum value and a maximum value of the angle θ are set as a change range of the inclination angle θ of the ellipse. Hereinafter, all combinations of parameter values within the set change range of each parameter of the ellipse are referred to as an ellipse parameter section.

  The weight setting unit 105 sets the weight according to the position of the edge pixel detected by the edge detection unit 103 on each ellipse in the parameter section set by the parameter setting unit 104. The accumulating unit 106 accumulates the weights of the edge pixels set by the weight setting unit 105 in the elliptic parameter section set by the parameter setting unit 104. The head ellipse selection unit 107 selects and outputs the ellipse having the largest cumulative number as the head ellipse 108 based on the number of weighted edge pixels accumulated by the accumulation unit 106.

FIG. 2 is a flowchart illustrating an example of a processing procedure in the present embodiment.
First, in step S101, an input image 101 is input to the image processing apparatus 100 from an image data input unit (not shown). The input image 101 may be a still image or a frame image in a moving image.

  Next, in step S102, the head detection area setting unit 102 sets an area surrounding the head area in the input image 101 as a head detection area in response to an input from the user interface. The head detection area may be a square, polygonal, circular, or elliptical area.

  Next, in step S103, the edge detection unit 103 calculates a gradient image for the head detection region set in step S102 by an edge detection operator such as Sobel, Prewitt, or Laplacian. Then, the gradient image is binarized with a predetermined threshold value to detect edge pixels. In addition to the edge detection operator, the edge pixel may be detected by a canny edge detection method, a zero-crossing edge detection method, a morphology edge detection method, or the like.

  Next, in step S104, the parameter setting unit 104 sets an elliptic parameter change range of the Hough transform, that is, an elliptic parameter space, based on the set size of the head detection area, as shown in FIG. The upper left coordinates of the circumscribed rectangle 200 of the head detection area are (x1, y1), the lower right coordinates are (x3, y3), and the horizontal and vertical sizes are w and h, respectively. First, the parameter setting unit 104 sets the change range of the vertical axis length b and the horizontal axis length a in proportion to the size of the head detection area by the following equation (1). Here, α and β are parameters.

  Next, the range of the center of the ellipse is set by the following equation (2).

  Finally, a minimum value and a maximum value of θ are set as the range of the ellipse rotation angle θ according to the application use case of the image processing apparatus.

  Next, in step S <b> 105, the accumulation unit 106 repeats the processing from step S <b> 106 to step S <b> 110 for all edge pixels in the head detection region set by the head detection region setting unit 102.

  First, in step S106, the accumulating unit 106 repeatedly performs the processing from step S107 to step S109 for the elliptic parameter space set in step S104.

  In step S107, the weight setting unit 105 calculates an ellipse that passes the target edge pixel for each ellipse on the ellipse parameter space. In step S108, the weight setting unit 105 sets the weight based on the position on the ellipse (x0, y0, b, a, θ) that passes the edge pixel in the ellipse parameter space obtained in step S107. Here, x0 is the X-axis coordinate of the ellipse center, y0 is the Y-axis coordinate of the ellipse center, b is the vertical axis length of the ellipse, a is the horizontal axis length of the ellipse, and θ is the rotation angle of the ellipse. As shown in FIG. 4, the coordinates of the edge pixel are (x, y), and the vertical distances from the separation point to the ellipse center (X axis) are d1 and d2, respectively. Since the center position of the ellipse passing through the edge pixel is (x0, y0), the weight is set by the following equation (3).

  However, if the separation point is above the X axis, the vertical distance from the separation point to the X axis is a negative number. If the separation point is below the X axis, the vertical distance from the separation point to the X axis is positive. It is a number. As shown in FIG. 4, the vertical distance from the separation point a and the separation point c to the X axis is a negative number, the vertical distance from the separation point b to the X axis is 0, and the vertical distance from the separation point d to the X axis. The distance is a positive number.

  Next, in step S109, the accumulation unit 106 accumulates the weight of the target edge pixel obtained in step S108 on each ellipse (x0, y0, b, a, θ) that passes the target edge pixel.

  In step S110, the accumulating unit 106 determines whether or not the processing from step S107 to step S109 has been completed for all ellipses within the set ellipse parameter range, and if completed, proceeds to step S111.

  In step S111, the accumulating unit 106 determines whether the processing from step S106 to step 110 has been completed for all edge pixels, corresponding to step S105. If all edge pixels have been processed, the process proceeds to step S112. If not, the processes from step S106 to step 110 are performed for the other edge pixels.

  In step S112, the head ellipse selection unit 107 selects the ellipse having the largest number of weighted edge pixels as the head ellipse 108 based on the number of weighted edge pixels of each ellipse in the ellipse parameter space accumulated by the accumulation unit 106. To do. In step S113, the head ellipse selector 107 outputs the head ellipse 108 selected in step S112.

  In the present embodiment, the accumulating unit 106 sets the weight by the step-like function according to the position of the edge pixel on each ellipse. However, in addition to the step-like function, the accumulating unit 106 has a continuous function, a smooth function with a continuous first derivative, etc. The edge pixel weights may be set with. In this embodiment, the vertical axis length and the horizontal axis length are handled independently as the ellipse parameters, but the vertical axis length and the horizontal axis length may be limited to the same value. Further, although the inclination angle of the ellipse is changed, the inclination angle may not be treated as a parameter, assuming that the ellipse is not inclined.

  In the present embodiment, the head detection area is set according to the input from the user interface, but the face is detected using the face detection technology, and the head detection area is set based on the face area. Also good. Further, when the input data is continuous video, the head detection area of the current frame may be set based on the head area detected in the previous frame.

  In this embodiment, the Hough transform parameter space is set using the head detection region. However, the face detection technology is used to detect the face, and the Hough transform parameter space is set based on the face region. May be. When the input data is continuous video, the parameter space for the Hough transform of the current frame may be set based on the head region detected in the previous frame.

  As described above, according to the present embodiment, the edge pixels are weighted according to the positions where the edge pixels exist in the respective ellipses on the elliptic parameter section in the Hough transform. Then, by accumulating the weights in the ellipse parameter section, it is possible to preferentially detect an upwardly convex arc that is a feature of the edge above the head, and the upper body edge is less likely to affect head detection. In addition, erroneous detection of a horizontal edge or vertical edge in the background area as an elliptical arc can be reduced.

(Second Embodiment)
In this embodiment, the edge pixel weight setting method is different from that of the first embodiment. In addition, since the process of another process part is the same as that of 1st Embodiment, description is abbreviate | omitted. In the present embodiment, parts different from the first embodiment will be described.

  In the present embodiment, as shown in FIG. 6, the weight setting unit 105 moves the edge in the rotation direction of the ellipse (x0, y0, b, a, θ) that passes the edge pixel in the ellipse parameter space obtained in step S107. The weight of the edge pixel is set according to the position of the pixel on the ellipse. At this time, the coordinates of the edge pixel are (x, y), and the horizontal axis of the rotated ellipse is H ′. The vertical distance from the separation point e to H ′ is d1, and the vertical distance from the separation point f to H ′ is d2. The distance from the edge pixel (x, y) to H ′ is obtained by the following equation (4). Here, when the center (x0, y0) of the ellipse is the rotation center and the Y axis is rotated rightward, the angle θ is a positive number.

  Based on the difference between the rotated Y-axis coordinates y ′ and y0, the weight of the edge pixel (x, y) is set by the following equation (5).

(Third embodiment)
FIG. 7 is a block diagram illustrating a functional configuration example of the image processing apparatus 300 according to the present embodiment. Unlike the first embodiment, the present embodiment does not include the edge pixel weight setting unit 105, and includes a candidate ellipse selection unit 306 and a head ellipse selection unit 307 instead of the head ellipse selection unit 107. . Note that the configuration of the head detection region unit 302, the edge detection unit 303, and the parameter setting unit 304 is the same as that of the head detection region setting unit 102, the edge detection unit 103, and the parameter setting unit 104 in FIG. The description is omitted.

  In FIG. 5, the accumulation unit 305 simply accumulates the number of edge pixels in the ellipse parameter section set by the parameter setting unit 304 without weighting each ellipse. The candidate ellipse selection unit 306 sorts the number of edge pixels accumulated in the respective ellipses by the accumulation unit 305, selects N ellipses having the upper pixel number, and sets them as candidate ellipses.

The head ellipse selection unit 307 calculates a comprehensive index from the N candidate ellipses selected by the candidate ellipse selection unit 306 based on the following indices (1) to (5), and determines the top ellipse of the comprehensive index as the head ellipse 308. Elected as.
(1) “Relative pixel count” index for preferentially selecting a large ellipse (2) “Pixel number / circumference length” index for preferentially selecting a small ellipse (3) Ellipse with emphasis on the edge arc at the top of the ellipse “Number of pixels / arc length” index for selecting (4) “Circularity” index for preferentially selecting an ellipse of a predetermined shape

  FIG. 8 is a flowchart illustrating an example of a processing procedure in the present embodiment. Note that the processes from step S301 to S307 are the same as steps S101 to S107 of FIG. 2 described in the first embodiment, and steps S308 to S310 are the same as steps S109 to S111 of FIG. Therefore, detailed description thereof will be omitted.

  In step S311, the candidate ellipse selection unit 306 sorts the number of edge pixels of each ellipse cumulatively calculated in step 308, and selects N ellipses having the upper number of pixels. Here, N is a parameter, which is set according to the size of the elliptic parameter space.

Next, in step S312, the head ellipse selection unit 307 calculates a total index for the selected N candidate ellipses based on the above-described indices (1) to (4). Hereinafter, the number of edge pixels of the N candidate ellipses is n _k (k = 1, 2,..., N), respectively.

  The “relative pixel number” index in (1) described above is a relative ratio of the number of edge pixels, and is calculated by the following equation (6).

The “number of pixels / circumference length” index in (2) is calculated by the following equation (7). Here, a _k and b _k are the major axis and the minor axis of the kth candidate ellipse, respectively. Here, the circumference of the ellipse is proportional to (a _k + b _k ), but may be calculated by the original ellipse circumference calculation formula as shown in the following equation (8).

The “number of pixels / arc length” index in (3) is calculated by the following equation (9). At this time, as shown in FIG. 9, the lowest pixel of the left elliptical arc is A, the lowest pixel of the right elliptical arc is B, and the arc length of the upper elliptical arc from A to B in the counterclockwise direction is Let AB _k . Here, the arc length AB _k is the number of ellipse pixels between the arcs AB when generating the candidate ellipse.

The “circularity” index in (4) described above is calculated by the following equation (10) when the major and minor axes of the candidate ellipse are a _k and b _k , respectively. Here, α is a parameter, which is a value calculated by statistically analyzing the human head shape.

  As described above, the total index is calculated by linear combination of the above-described four indices according to the following equation (11).

  Here, the overall index is a linear combination of the indices, but may be a linear combination of monotonic functions of the indices. Moreover, it is good also as a linear combination of two or three indices among the indices regarding four ellipses. Further, in addition to the linear combination, it may be configured by a general function such as a combination of addition and multiplication.

  In addition, the comprehensive index is not composed of a function of each index, but is evaluated by one index, and when there are a plurality of ellipses having the same evaluation value, the ellipse having the same evaluation value is evaluated by another index. You may choose. Further, when there are a plurality of ellipses having the same evaluation value, the ellipses having the same evaluation value may be repeatedly evaluated using another index. Similarly, you may comprise only two or three indices among four indices.

  In addition, the comprehensive index is not composed of a function of each index, but is selected as a head ellipse when the evaluation value of the candidate ellipse falls within a predetermined range as shown in the following formula (12) in each index. . Further, when there are a plurality of head ellipses within the predetermined range shown in Expression (12), the head ellipse may be further selected using one index. Similarly, here, it is determined whether all the indices related to the four ellipses are used to enter the predetermined range, but it may be configured by only two or three indices among the four ellipticity indices. Further, if there is no ellipse that satisfies the following expression (12), head detection fails.

  In the present embodiment, the head ellipse selection unit 307 selects a head ellipse from candidate ellipses using a combination of four indices. The head ellipse may be selected using only the “number / arc length” index. Similarly, the head ellipse may be selected only by the “circularity” index that preferentially selects an ellipse having a predetermined shape.

  Returning to the description of FIG. 8, in step S <b> 313, the head ellipse selection unit 307 selects the ellipse having the largest evaluation value according to the comprehensive index obtained in step S <b> 312 and sets it as the head ellipse 308. In step S314, the head ellipse selection unit 307 outputs the head ellipse 308 selected in step S313.

(Fourth embodiment)
FIG. 10 is a block diagram illustrating a functional configuration example of the image processing apparatus 400 according to the present embodiment. The difference from the third embodiment is that a weight setting unit 409 is provided. Since the weight setting unit 409 is the same as the weight setting unit 105 of FIG. 1 described in the first embodiment, detailed description thereof is omitted. The configurations of the head detection region setting unit 402 to the accumulation unit 405 are also the same as the head detection region setting unit 102 to the parameter setting unit 104 and the accumulation unit 106 of FIG. 1 described in the first embodiment. Therefore, detailed description is omitted.

  The candidate ellipse selection unit 406 selects the upper cumulative number of ellipses as N candidate ellipses using the weighted cumulative edge pixel number instead of the cumulative edge pixel number. The head ellipse selection unit 407 is the same as the head ellipse selection unit 307 of the third embodiment described above according to the equations (6), (7), (9), and (10) (1). The four indices (4) to (4) are calculated, and these indices are combined to determine the overall index. Then, the ellipse with the highest overall index is selected as the head ellipse 408.

In this embodiment, the weight of the edge pixel set by the weight setting unit 409 may be reflected on the indices related to the four ellipses. In this case, these four indexes are calculated as follows. Hereinafter, the number of weighted edge pixels of the N candidate ellipses is set to w _k (k = 1, 2,..., N), respectively.

  The above-mentioned “relative pixel number” index in (1) is calculated by the following equation (13).

The “number of pixels / circumference length” index in (2) is calculated by the following equation (14). Here, w _xy is the weight of the arc xy, and n _xy is the number of pixels (length) of the arc xy. When the weight is set not by a step function but by a general function, the weight is integrated instead of the sum of the denominators in the following equation (14). Each arc and its weight are shown in FIG.

The “number of pixels / arc length” index in (3) described above is calculated by the following equation (15). Here, w _xy is the weight of the arc xy, and n _xy is the number of pixels (length) of the arc xy. When the weight is set not by a step function but by a general function, the weight is integrated instead of the sum of the denominators in the following equation (15). Each arc is shown in FIG.

  The “circularity” index in (4) described above is the same as in the third embodiment.

  FIG. 13 is a flowchart illustrating an example of a processing procedure in the present embodiment. Note that the processes from step S401 to S411 are the same as steps S101 to S111 of FIG. 2 described in the first embodiment. Steps S412 to S415 are the same as steps S311 to S314 of FIG. 8 described in the third embodiment.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

102 Head Detection Area Setting Unit 103 Edge Detection Unit 104 Parameter Detection Unit 105 Weight Setting Unit 106 Accumulation Unit 107 Head Ellipse Selection Unit

Claims (6)

Means for detecting edge pixels for the set head detection region;
Means for accumulating the edge pixels in each ellipse in an elliptic parameter space in a Hough transform;
Means for selecting a candidate ellipse based on the number of edge pixels;
An image processing apparatus comprising: means for selecting a head ellipse from the candidate ellipses using a comprehensive index composed of elliptic indices.   The elliptic parameter in the Hough transform is a combination of at least one parameter among the X-axis coordinate, the Y-axis coordinate, the vertical axis length, the horizontal axis length, and the rotation angle. The elliptic parameter interval is the minimum of each parameter. The image processing apparatus according to claim 1, wherein the image processing apparatus is a section from a value to a maximum value.   The overall index composed of the ellipticity index is an index that preferentially selects a large ellipse, an index that preferentially selects a small ellipse, an index that selects an ellipse with an emphasis on the edge arc above the ellipse, and a predetermined shape The image processing apparatus according to claim 1, wherein the image processing apparatus is an index configured by combining at least two indices among indices that preferentially select the ellipse.   The image according to claim 3, wherein the index for preferentially selecting the large ellipse is a monotone function of a ratio between the number of edge pixels of each candidate ellipse and the maximum value of the number of edge pixels of each candidate ellipse. Processing equipment. Detecting edge pixels for the set head detection region;
Accumulating the edge pixels on each ellipse in an elliptic parameter space in a Hough transform;
Selecting a candidate ellipse based on the number of edge pixels;
And a step of selecting a head ellipse from the candidate ellipses using a comprehensive index composed of ellipticity indices.   The program for making a computer perform each process of the image processing method of Claim 5.

Images