JP5795916B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to a technique for detecting a subject from an image.

  Conventionally, when a specific target such as a face or a human body is detected from an image of a monitoring camera, the target is detected by pattern matching processing. For example, in Patent Document 1, a plurality of local patterns are extracted using a matching pattern over the entire image while changing the size of the image, and each local feature amount is calculated. And it is determined whether it is a human body from the sum total of the result of having weighted the local feature-value. In addition, these specific object detection processes need to be able to be performed at high speed in order to perform real-time processing in the background of increasing the number of pixels of network camera devices and monitoring. Regarding a method for realizing high-speed processing, Patent Document 2 describes a method of switching the priority of reduced images (layers) used for pattern matching based on past image capturing results.

JP 2009-211311 A Japanese Patent No. 4626493

  However, in particular, surveillance cameras often capture a wide range of depth, and there are scenes in which large and small subjects are mixed in the same screen, and the subject is not always uniform in the screen. Absent.

  In this environment, in order to detect all the detection target objects, it is necessary to use a matching pattern from when the object size is small to when it is large. Therefore, even if the priority of the reduced image (layer) used for the collation process is changed, the detection process cannot be performed at a high speed after all.

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide a technique for enabling a specific object to be detected at high speed regardless of the size of the detection target object in the image.

In order to achieve the object of the present invention, for example, an image processing apparatus of the present invention includes an acquisition means for acquiring a first image,
Detection means for performing subject detection processing on the first to N images when an image generated by recursively reducing the first image is an Nth image (N is a natural number of 2 or more). And detecting means for performing subject detection processing on an image area associated with the m-th image among the image areas included in the m-th image (1 ≦ m ≦ N). To do.

  According to the configuration of the present invention, it is possible to detect a specific object at high speed regardless of the size of the detection target object in the image.

FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing apparatus. The figure explaining a human body detection process. The figure which shows the example of a division | segmentation of an image. The figure which shows the structural example of a correspondence table. The figure which shows the example of a display of GUI. 6 is a flowchart of processing performed by the image processing apparatus. The flowchart of step S603, S605, S607, S609. FIG. 3 is a block diagram illustrating an example of a functional configuration of the image processing apparatus. The figure explaining a human body detection process. 6 is a flowchart of processing performed by the image processing apparatus. The flowchart of step S1007, S1010, S1013. The figure which shows the structural example of a correspondence table. The figure which shows the example of a display of GUI.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below shows an example when the present invention is specifically implemented, and is one of the specific examples of the configurations described in the claims.

[First Embodiment]
The image processing apparatus according to the present embodiment can be applied to an image processing circuit incorporated in an imaging apparatus such as a surveillance camera or a computer apparatus such as a personal computer. First, a functional configuration example of the image processing apparatus according to the present embodiment will be described with reference to the block diagram of FIG.

  The image acquisition unit 101 acquires a moving image or still image supplied from the outside, and sends the acquired moving image or still image to the human body detection unit 102 in the subsequent stage. When the image acquisition unit 101 acquires a moving image, the image of each frame constituting the moving image is sequentially sent to the human body detection unit 102. When a still image is acquired, the image acquisition unit 101 detects the still image as a human body. Sent to the unit 102. Note that the source of the moving image or still image is not particularly limited, and may be a server device or an imaging device that supplies a moving image or still image via a wired or wireless connection. Further, the image is not limited to the outside, and a moving image or a still image may be acquired from a memory in the image processing apparatus. In the following description, a case where one image in the human body detection unit 102 is input regardless of whether the image acquisition unit 101 acquires a moving image or a still image will be described. In the former case, this single image corresponds to each frame constituting the moving image, and in the latter case, this single image corresponds to a still image.

  In order to detect subjects having different sizes from the image (first image) supplied from the image acquisition unit 101, the human body detection unit 102 first reduces the first image recursively, thereby reducing a plurality of images. An image (second to Nth images) is generated. Hereinafter, each of the first to N images may be referred to as a processing layer. Details of the processing layer will be described later. The human body detection unit 102 performs subject detection processing by matching each of the first to N images with a human body matching pattern. At that time, the human body detection unit 102 performs processing on the mth image (1 ≦ m ≦ N). An image area associated with the mth image in advance among the image areas is set as a target image area. Then, a rectangular area of a prescribed size is set at each position in the target image area, and subject detection processing is performed on the rectangular area. In the present embodiment, the case where the subject is a human body will be described. However, the following description can be similarly applied even when the subject is another object. In this embodiment, the well-known pattern matching using the human body matching pattern is used for the subject detection process. However, the subject detection process may be realized using other techniques.

  The storage unit 105 holds (registers) a table (correspondence table) that associates image regions at relatively the same positions in the first to N images with any of the first to N images. When the human body detection unit 102 specifies an image area previously associated with the m-th image among the image areas on the m-th image (1 ≦ m ≦ N) as a target image area, this correspondence table Will be referred to. The storage unit 105 registers the human body matching pattern used in the human body detection process.

  The layer setting unit 104 creates the correspondence table according to the operation input from the input unit 103. The input unit 103 includes a keyboard, a mouse, and the like, and various instructions can be input by a user operation.

  The display unit 109 is configured by a CRT, a liquid crystal screen, or the like, and can display various screens to be described later. Note that the input unit 103 and the display unit 109 may be integrated to form a touch panel screen.

  The external output unit 106 outputs the processing result obtained by the human body detection unit 102. The output destination is not limited to a specific output destination, and may be output to an external device via a wireless or wired connection, or may be output to the display unit 109. Further, the information to be output may be output as it is or may be output with some visible graphics.

  Next, processing for detecting a subject from an image of various sizes (first to N images in the case of this embodiment) by performing pattern matching using a human body matching pattern will be described. . In order to perform pattern matching by arranging a human body matching pattern at each position on each image, first, as shown in FIG. 2A, a human body matching pattern 202 is placed at the upper left corner of the image 201 which is the first image. To do. Here, hereinafter, the first image may be referred to as processing layer 0 (L0). After the human body collation pattern 202 is arranged, the position of the human body collation pattern 202 is moved from the top to the right from there. Each time the image is moved, pattern matching is performed between the human body matching pattern 202 and the image region overlapping the human body matching pattern 202 to detect the human body.

  Next, as shown in FIG. 2B, an image 203 is generated as a second image obtained by reducing the first image by a predetermined magnification (a magnification smaller than 1). Here, hereinafter, the second image may be referred to as processing layer 1 (L1). Then, the human body matching pattern 202 is arranged in the upper left corner of the image 203 that is the second image. After the human body collation pattern 202 is arranged, the position of the human body collation pattern 202 is moved from the top to the right from there. Each time the image is moved, pattern matching is performed between the human body matching pattern 202 and the image region overlapping the human body matching pattern 202 to detect the human body.

  Next, as shown in FIG. 2C, an image 204 is generated as a third image obtained by reducing the second image by a predetermined magnification (a magnification smaller than 1). Hereinafter, the third image may be referred to as processing layer 2 (L2). Then, the human body matching pattern 202 is arranged in the upper left corner of the image 204 that is the third image. After the human body collation pattern 202 is arranged, the position of the human body collation pattern 202 is moved from the top to the right from there. Each time the image is moved, pattern matching is performed between the human body matching pattern 202 and the image region overlapping the human body matching pattern 202 to detect the human body.

  As described above, by repeatedly using the same human body matching pattern for the reduced image, it is possible to detect human bodies of various sizes in the image. Here, the processing layer is a plurality of image sets having different resolutions generated by enlarging or reducing an input image. As described above, the processing layer in the present embodiment is an image set of a plurality of processing layers. The original image is layer 0, and each time the image is reduced, the processing layer 1 and the processing layer 2 are used. Further, as described above, the size of the human body to be detected is related to the processing layer.

  These processes are repeated to detect human bodies of various sizes, and the process ends when the process is continued for a specified time. In this embodiment, the end of the process is determined based on the elapsed time from the start of the process, but the condition for terminating the process is not limited to this. For example, the end of processing may be determined from the number of detected human bodies, the number of times human body detection processing is executed, and the like.

  Further, the reduction ratio for generating the processing layer P (P ≧ 1) may be any number as long as it is less than 1. The closer the reduction ratio is to 1, the higher the accuracy, but the longer the processing time. The lower the reduction ratio, the lower the accuracy, but the shorter the processing time, so the appropriate reduction ratio should be set according to the operating conditions. Is desirable. In this embodiment, a large human body is sequentially detected from an image group obtained by recursively reducing the first image. However, the processing order may be changed so that a large human body is detected from a small human body.

  In the present embodiment, the human body is the subject, but it may be a face, a car, an animal, or the like. Furthermore, if a plurality of types of specific objects can be detected and a plurality of subjects can be detected simultaneously, a plurality of specific object detection processes may be performed on the object.

  In addition, depending on the object attributes used for the set values such as the object size and aspect ratio set separately, only a predetermined process may be selected from a plurality of specific object detection processes and the process may be performed. For example, assume that human body detection and car detection are performed as specific object detection. At this time, the size of the car that can appear in the screen is set as a minimum vehicle size in accordance with a user instruction. When the size of the object is smaller than the minimum vehicle size, the possibility of being detected as a car is low, so the car detection is not performed and only the human body detection is performed. On the other hand, if the size of the object is larger than the minimum vehicle size, the vehicle may be detected, and therefore vehicle detection is performed. Further, since there is a possibility that a plurality of people act as a group and are detected as one object, human body detection is also performed.

  Next, the correspondence table and the procedure for creating it will be described.

  First, as illustrated in FIG. 3, the layer setting unit 104 divides the image acquired by the image acquisition unit 101, that is, the image 301 as the first image into a plurality of rectangular regions (image regions) 302. In FIG. 3, although divided into 3 (vertical) × 4 (horizontal) rectangular regions 302, the number of divisions and the division pattern are not limited thereto. That is, the image may not be divided at equal intervals, or a part of the image, not the entire image, may be divided into a plurality of regions. Further, the shape of the divided area is not limited to a rectangle, but may be an area having an arbitrary shape, or may be a shape such as a polygon set by the user.

  Then, the layer setting unit 104 assigns an identification number to each rectangular area in order to identify each rectangular area 302. In FIG. 3, identification numbers such as (1, 1), (1, 2), (1, 3), (1, 4) are assigned to the uppermost rectangular area 302 in order from the left end. Further, an identification number is assigned to the middle rectangular area 302 in the order of (2, 1), (2, 2), (2, 3), (2, 4) from the left end. Further, identification numbers such as (3, 1), (3, 2), (3, 3), and (3, 4) are assigned to the bottom rectangular region 302 in order from the left end.

  In FIG. 3, the human body 303 in the rectangular area with the identification number (1, 3) is shown across the rectangular area with the identification number (2, 1) and the rectangular area with the identification number (3, 1). Because it is located in the back compared to, it appears smaller. In this embodiment, paying attention to this point, the human body detection process is not performed on all the rectangular areas of the first image, but the area where the human body appears relatively small is detected from the first image. A relatively large area is detected from a further reduced image.

  Next, the layer setting unit 104 displays the first image acquired by the image acquisition unit 101 on the display screen 501 of the display unit 109 as shown in FIG. Then, the user operates the input unit 103 to set frames 502 and 503 that surround the human body shown in the first image. The frames 502 and 503 are sample rectangles indicating the size of the human body to be detected. The frames 502 and 503 have control points at the four corners, and the shape and size of the frames 502 and 503 can be changed in the first image. Note that the user may set an appropriate size and position frame using the input unit 103 without displaying the first image on the display screen 501.

  In any case, when there is a depth as shown in FIG. 5A, it is desirable to set a small frame and a large frame in the display screen. In FIG. 5A, a small frame is set at the back of the display screen. A large frame is set in front of the display screen.

  When the user operates the input unit 103 to input that this setting has been completed, the layer setting unit 104 operates as follows. First, from the size of one or more frames set on the display screen 501, for each rectangular area on the first image, a frame that includes the human body assumed when the human body appears in the rectangular area ( Assumed subject size). In FIG. 5B, the assumed subject size in each rectangular area is obtained by linear interpolation from the size of the frame 502 and the frame 503 and the distance to the frame 502 and the frame 503, and the frame 504 indicated by a dotted line is A frame having the assumed subject size. Of course, the method of obtaining the assumed subject size in each rectangular area is not limited to this. Hereinafter, a frame set by the user (frames 502 and 503 in FIG. 5A) and a frame obtained by interpolation from the frames (frame 504 in FIG. 5B) are collectively referred to as a subject frame.

  When the subject frame in each rectangular area is determined, the layer setting unit 104 determines the correspondence between the identification number of each rectangular area and the processing layer corresponding to the size of the subject frame obtained for the rectangular area, as illustrated in FIG. The above correspondence table to be managed is generated and registered in the storage unit 105.

  The correspondence table illustrated in FIG. 4 corresponds to the first image illustrated in FIG. In the first image shown in FIG. 3, a subject frame having a relatively small size is set for the uppermost rectangular area as shown in FIG. 5B. In this case, for the uppermost rectangular area, The first and second images (processing layers 0 and 1), which are relatively large images, are associated with each other.

  In the first image shown in FIG. 3, a subject frame that is relatively larger in size than the uppermost rectangular area is set in the middle rectangular area as shown in FIG. 5B. In this case, however, the second and third images (processing layers 1 and 2), which are relatively smaller in size than the uppermost rectangular area, are associated with the middle rectangular area.

  In the first image shown in FIG. 3, a subject frame that is relatively larger in size than the middle rectangular area is set for the lowermost rectangular area as shown in FIG. 5B. However, in this case, the lowermost rectangular area is associated with fourth and fifth images (processing layers 3 and 4), which are relatively smaller in size than the middle rectangular area.

  These settings indicate that only the small human body is detected in the area above the image, and the large human body is set not to perform the detection process. Note that the correspondence between the size of the subject frame and the sizes of the first to N images is set in advance, and when the size of the subject frame is determined, the corresponding image is any of the first to N images. Can be specified.

  However, the method for generating the correspondence table as shown in FIG. 4 is not limited to this, and the user may create the correspondence table by operating the input unit 103. For example, since the subject is not shown in the rectangular area with the identification number (1, 1), the processing layer does not have to be associated with this rectangular area. Such an exception operation related to the association may be performed by the user using the input unit 103.

  In this way, at the time of human body detection processing, by using only the processing layer of the correspondence table at the time of detection processing of each region, the human body detection processing of an unnecessary size can be omitted, so that the speed of human body detection processing can be increased. .

  Here, there are various methods for associating the rectangular area with the processing layer. For example, the following method can be considered. For example, not only processing layer 2 is registered as a processing layer corresponding to a rectangular area, but also the processing layers (processing layer 1 and processing layer 3) before and after that are registered as processing layers corresponding to the rectangular area. It may be. Alternatively, a plurality of front and rear layers may be registered instead of the front and rear layers. This is related to the reduction ratio of the image in the human body detection process. Since the possibility of omission of detection increases as the reduction ratio is closer to 0, it is preferable to increase the margin of the processing layer.

  Also, for example, instead of obtaining a frame in each rectangular area from two frames set by the user as described above, one frame and two or more straight lines are set, and this one frame and two or more frames are set. A frame in each rectangular area may be obtained from the straight line. In this case, the vanishing point on the image is obtained from a plurality of straight lines. Then, a frame in each rectangular area is set based on the vanishing point and the set frame position, and a processing layer corresponding to the set frame size is obtained. The calculation method of the frame in each rectangular area can be simply obtained by proportionally determining the human body size at the vanishing point as 0 and the distance from the vanishing point.

  In this case, instead of the GUI of FIG. 5, the GUI illustrated in FIG. 13 may be displayed, and a frame in each rectangular area may be obtained using this GUI. In FIG. 13, the user sets a frame 1302 and a straight line 1303 on the display screen 1301 of the display unit 109. The straight line 1303 may be extracted by a straight line extraction process. A vanishing point 1304 is a point that intersects when two straight lines 1303 are extended.

  Alternatively, many image processing techniques for obtaining a straight line from an image such as Hough transform are known, and using these, two or more straight lines are automatically obtained from an image, vanishing points are obtained, and each image in the image is similarly obtained. The processing layer at the position may be obtained. In addition, although an example in which the vanishing point is obtained from a straight line has been described here, any method may be used as long as it is a method for obtaining the vanishing point in the image.

  Next, processing performed by the image processing apparatus according to the present embodiment will be described with reference to FIG. 6 showing a flowchart of the processing. It is assumed that the correspondence table as described above is already registered in the storage unit 105 at the time when the processing according to this flowchart is started.

  In step S601, a control unit (not shown) of the image processing apparatus 100 determines whether to continue the following processing. In this determination, as described above, if the specified time or more has elapsed since the start of the processing of the flowchart of FIG. 6, it is determined that the processing is to end, and if the specified time has not yet elapsed, It is determined that the process is to be continued. If the result of determination in step S601 is to continue, the process proceeds to step S602; otherwise, the process ends. In step S602, the image acquisition unit 101 acquires an input image as a first image.

  In step S603, the human body detection unit 102 performs human body detection processing on the first image acquired by the image acquisition unit 101 in step S601. Since the human body detection process in each of steps S603, S605, S607, and S609 is a process that is performed along the same flowchart, the process in these steps will be described later.

  When the processing in step S603 ends, in step S604, the human body detection unit 102 generates a second image that is an image obtained by reducing the first image.

  In step S605, the human body detection unit 102 performs human body detection processing on the second image generated in step S604. When the processing in step S605 ends, in step S606, the human body detection unit 102 generates a third image that is an image obtained by reducing the second image.

  In step S607, the human body detection unit 102 performs human body detection processing on the third image generated in step S606. Thereafter, the same processing is performed up to the Nth image (S608, S609).

  In step S610, the external output unit 106 integrates the results of the human body detection process from the first to N images. In this integration process, an area on the first image corresponding to the human body detection area in each of the first to N images is obtained. The external output unit 106 outputs the obtained position and size of the area.

  Next, the human body detection process in each of steps S603, S605, S607, and S609 will be described with reference to FIG. 7 showing a flowchart of the process. Hereinafter, the human body detection process for the m-th image (1 ≦ m ≦ N) will be described. However, m = 1 may be set in step S603, m = 2 may be set in step S605, m = 3 may be set in step S607, and m = N may be set in step S609.

  In step S701, the human body detection unit 102 determines whether or not the movement of the human body matching pattern within the mth image has been completed. That is, it is determined whether or not the human body detection process using the human body matching pattern in the mth image is completed. As a result of the determination, when the movement of the human body matching pattern in the mth image is completed, that is, when the human body detection process using the human body matching pattern is completed in the mth image, the processing according to the flowchart of FIG. Is completed. On the other hand, if the movement of the human body verification pattern in the mth image has not been completed yet, that is, if the human body detection process using the human body verification pattern has not been completed in the mth image, the process proceeds to step S702. move on. In step S <b> 702, the human body detection unit 102 acquires the above correspondence table from the storage unit 105.

  In step S703, the human body detection unit 102 refers to the correspondence table, and determines whether or not the current human body matching pattern on the m-th image is located in a rectangular area having an identification number associated with the m-th image. Judging. As a result of this determination, if it is determined that “it is located within a rectangular area having an identification number associated with the mth image”, the process proceeds to step S704. On the other hand, if it cannot be determined that “it is located within the rectangular area having the identification number associated with the mth image”, the process proceeds to step S705.

  In the example of FIG. 4, when m = 1, the processing is performed only when the human body matching pattern is located in the rectangular area of the identification numbers (1, 2), (1, 3), (1, 4). The process proceeds to step S704, otherwise the process proceeds to step S705.

  In step S <b> 704, the human body detection unit 102 performs human body detection processing by pattern matching between a human body verification pattern and a region that overlaps the human body verification pattern in the m-th image, and outputs the position and size at which the human body is detected.

  In step S705, the human body detection unit 102 moves the human body matching pattern to the next position. If the process proceeds from step S704 to step S705, the human body matching pattern may be moved by several pixels in step S705. On the other hand, even when the process proceeds from step S703 to step S705, the movement may be performed by the same amount, but even if the movement is performed by several pixels, the moved position is still “associated with the mth image. There is a possibility that it is not “in a rectangular area having an identification number”. In such a case, it may be moved to the next rectangular area. By performing such processing, it is possible to omit the human body detection processing in the same rectangular area at once.

  As described above, according to the present embodiment, the subject detection process can be performed at a higher speed regardless of the size of the subject appearing in the image.

[Second Embodiment]
Below, only a different point from 1st Embodiment is demonstrated, and unless it touches in particular below, it shall be the same as that of 1st Embodiment. In the first embodiment, the human body detection process of the target layer is performed on the entire image. However, for example, the moving object detection process may be performed on the original image, and the human body detection process may be performed only on the area where the moving object is detected. Further, in the first embodiment, the target layer is determined and used at the collation scan position in the human body detection process. However, for example, the target layer may be determined and used based on the position of the moving object region.

  A functional configuration example of the image processing apparatus according to the present embodiment will be described with reference to the block diagram of FIG. In FIG. 8, the same constituent elements as those shown in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted.

  The moving object region detection unit 801 detects, as a moving object region, a region in which a moving object is reflected from the first image acquired by the image acquisition unit 101 by the background subtraction method. The content to be detected is information (moving body area information) that defines the moving body area on the first image, such as the position of the moving body area on the first image, the circumscribed rectangle of the moving body area, and the size of the moving body area. In addition, since various techniques can be applied to the method of detecting the moving object region from the first image, not only the background difference method but also other techniques may be applied.

  Then, the moving body region detection unit 801 sends the detected moving body region information to the subsequent human body detection unit 102. The human body detection unit 102 sets an image area that is an image area that is associated with the m-th image in advance and includes the moving body area indicated by the moving body area information, as the target image area. Then, the human body detection unit 102 sets a rectangular area of a specified size at each position in the target image area, and performs human body detection processing on the target image area.

  Note that the human body detection process is performed not only on an image area that completely includes a moving body area but also on an image area that partially includes a moving body area that has been erroneously determined to be undetected by the moving body detection process. The human body can also be detected.

  The human body detection process performed in this embodiment will be described with reference to FIG. If the moving body region obtained by the moving body region detection unit 801 performing the moving body detection process on the image 901 as the first image is the regions 902 and 903, the human body matching pattern 904 is used in the regions 902 and 903. To perform human body detection processing. Of course, the human body detection process is not performed on all the image areas in the areas 902 and 903 but on the image area associated with the image 901. Similar processing is performed for each of the second image, the third image,..., The Nth image.

  Next, processing performed by the image processing apparatus according to the present embodiment will be described with reference to FIG. 10 showing a flowchart of the processing.

  In step S1001, the control unit (not shown) of the image processing apparatus 100 determines whether or not to continue the following processing in the same manner as in step S601. If the result of determination in step S1001 is to continue, the process proceeds to step S1002, and if not continued, this process ends. In step S1002, the image acquisition unit 101 acquires an input image as a first image, and the moving object region detection unit 801 detects a moving object region from the first image.

  In step S1003, the human body detection unit 102 first selects one moving body area that has not yet been subjected to the following processing among the moving body areas detected in step S1002 as a selected moving body area. The selection order is not limited to a specific order. For example, the selection order may be selected from the position of the upper left corner of the first image in the raster scan order.

  In step S1004, the human body detection unit 102 extracts the selected moving body region from the first image as the original image L0. In step S <b> 1005, the human body detection unit 102 specifies a processing layer associated with the identification number of the image area including the selected moving body area from the correspondence table stored in the storage unit 105.

  In step S1006, the human body detection unit 102 determines whether or not the first image is included in the processing layer specified in step S1005. As a result of this determination, if it is included, the process proceeds to step S1007, and if it is not included, the process proceeds to step S1008.

  In step S1007, the human body detection unit 102 performs human body detection processing on the original image L0. Since the human body detection process in each of steps S1007, S1010, and S1013 is a process that is performed along the same flowchart, the process in these steps will be described later.

  In step S1008, the human body detection unit 102 determines whether or not the second image is included in the processing layer specified in step S1005. As a result of the determination, if it is included, the process proceeds to step S1009. If not, the process corresponding to steps S1008 to S1010 is performed on the third image. In this way, the same processing is performed for the fourth image,..., The Nth image (S1011 to S1013).

  In step S1009, the human body detection unit 102 generates a second image that is an image obtained by reducing the first image. In step S1010, the human body detection unit 102 performs a human body detection process on a region corresponding to the original image L0 (that is, a moving body region in the second image) in the second image generated in step S1009.

  In step S1014, the external output unit 106 integrates the results of the human body detection process from the first to N images. In this integration process, an area on the first image corresponding to the human body detection area in each of the first to N images is obtained. The external output unit 106 outputs the obtained position and size of the area.

  In step S1015, the control unit (not shown) determines whether or not human body detection processing has been performed for all moving body regions detected in step S1002. As a result of the determination, if the human body detection process is performed for all the moving body regions, the process proceeds to step S1001. If the moving body region that has not been subjected to the human body detection process still remains, the process proceeds to step S1003. .

  In this embodiment, in step S1008 or step S1011, it is determined whether or not the mth image is included in the processing layer specified in step S1005, and then the mth image is generated. However, for example, the processing in step S1009 or step S1012 may be omitted by creating in advance a reduced image of the layer to be processed in the correspondence table.

  Next, the human body detection process in each of steps S1007, S1010, and S1013 will be described with reference to FIG. 11 showing a flowchart of the process. Hereinafter, the human body detection process for the m-th image (1 ≦ m ≦ N) will be described. However, m = 1 may be set in step S1007, m = 2 may be set in step S1010, and m = N may be set in step S1013.

  In step S1101, the human body detection unit 102 determines whether or not the movement of the human body matching pattern within the selected moving body region in the mth image has been completed. That is, it is determined whether or not the human body detection process using the human body matching pattern is completed in the selected moving body region in the mth image. As a result of this determination, when the processing is completed, the processing according to the flowchart of FIG. 11 is completed. On the other hand, if not completed, the process proceeds to step S1102.

  In step S1102, the human body detection unit 102 performs human body detection processing by pattern matching between a human body verification pattern and a region overlapping with the human body verification pattern in the m-th image, and outputs the position and size at which the human body is detected.

  In step S1103, the human body detection unit 102 moves the human body matching pattern to the next position. In this movement, the human body verification pattern may be moved by several pixels. Of course, the movement amount is not limited to a specific movement amount.

  As described above, according to the present embodiment, the specific object detection process can be performed at high speed regardless of the size of the detection target object in the image. In addition, by performing the moving object detection process on the original image and performing the human body detection process only on the area where the moving object is detected, the human body detection can be performed at high speed.

[Third Embodiment]
Below, only a different point from 1st Embodiment is demonstrated, and unless it touches in particular below, it shall be the same as that of 1st Embodiment. In the first embodiment, a correspondence table as illustrated in FIG. 4 is created in advance and used for human body detection processing. However, for example, a correspondence table may be created based on past human body detection results, and human bodies may be detected using the correspondence table.

  In the present embodiment, the following processing is performed on each of a plurality of input images in advance. First, first to N images are generated from an input image. Then, each of the first to N images is divided into a plurality of rectangular areas as shown in FIG. 3, and human body detection processing is performed on each rectangular area. The count value (initial value is 0) is incremented by 1 for the area where the human body is detected.

  As a result of such processing, a correspondence table as shown in FIG. 12 is obtained. In this correspondence table, the count values in the rectangular areas of the identification numbers (1, 1) to (3, 4) are registered for the first to N images (L0 to Ln), respectively.

  For example, in the rectangular area of the identification number (1, 2) in each of the first image, the second image, the third image, and the fourth image, the human body is 5 times (count value is 5), 4 times (the count value is 4) Three times (count value is 3) and 0 times (count value is 0) are detected. In addition, in the rectangular area of the identification number (3, 2) in each of the first image, the second image, the third image, and the fourth image, the human body is 0 times (count value is 0), 0 times (the count value is 0), once (count value is 1), and 4 times (count value is 4).

  That is, this correspondence table shows that a relatively large human body is often detected above the image, and a relatively small human body is well detected below the image. When such a correspondence table is created, it is registered in the storage unit 105. This series of processing may be performed by an operation from the input unit 103 and an operation by the layer setting unit 104, or may be performed by an external device.

  Also in the present embodiment, human body detection processing is performed in the same manner as in the first embodiment except for the processing in step S703. Hereinafter, step S703 according to the present embodiment will be described.

  In step S703, the human body detection unit 102 refers to the above correspondence table, and the maximum count value among the count values of the first to N images with respect to the identification number of the rectangular area to which the current human body matching pattern on the m-th image belongs. It is determined whether the image corresponding to is the m-th image.

  Here, step S703 will be described with an example. It is assumed that the human body matching pattern is on the rectangular area of the identification number (2, 3) on the third image at the time of step S703. In this case, the count values of the first to N images in the identification number (2, 3) are 0, 3, 5, 0,... (Hereinafter all 0), and the count value of the third image is the largest. The process proceeds to step S704. On the other hand, when the human body matching pattern is on the rectangular area of the identification number (2, 3) on the second image, the count value of the third image is the largest, and in this case, the process proceeds to step S705.

  Instead of the maximum count value, it may be determined whether the image corresponding to the count value equal to or greater than the specified number of times (detected more than the specified number of times) is the m-th image. In this case, the specified number of times is preferably changed according to the learning period (a period during which the plurality of input images are acquired). Alternatively, if it is an image before and after the image of the maximum count value (layer numbers are before and after), the process may proceed to step S704 (in the case of R (2, 3), L1, L2, and L2 are processing layers). .

  As described above, according to the present embodiment, the specific object detection process can be performed at high speed regardless of the size of the detection target object in the image. In addition, by performing detection processing based on past detection results, it is possible to easily set a processing layer used for detection processing.

  Further, the above embodiments may be used in appropriate combination. 1 and 8, some or all of the units other than the input unit 103, the storage unit 105, and the display unit 109 may be configured by hardware or software (computer) Program). In the latter case, the computer program is stored in a memory (not shown) in the image processing apparatus and executed by a control unit such as a CPU included in the image processing apparatus.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

Claims (7)

Acquisition means for acquiring a first image;
Detection means for performing subject detection processing on the first to N images when an image generated by recursively reducing the first image is an Nth image (N is a natural number of 2 or more). And detecting means for performing subject detection processing on an image area associated with the m-th image among the image areas included in the m-th image (1 ≦ m ≦ N). An image processing apparatus. Furthermore,
Holding means for holding a table in which image regions at the same position in the first to N images are associated with any one of the first to N images;
2. The object detection processing according to claim 1, wherein the detection unit performs subject detection processing on an image region associated with the m-th image by the table among the image regions included in the m-th image. Image processing apparatus. Furthermore,
Means for determining, for each image area included in the first image , an assumed subject size assumed when a subject appears in the image area from the size of one or more areas set on the first image When,
Generating the table associating an image area of the first to N images, which is relatively located at the same position, with an image associated with the assumed subject size obtained for the image area of the first to N images; The image processing apparatus according to claim 2, further comprising: a registration unit configured to register with the holding unit. The detection means detects an area in which the moving object is shown in the first image as a moving object area, and is associated with the m-th image among the image areas included in the m-th image and the m-th image. 4. The image processing apparatus according to claim 1, wherein subject detection processing is performed on an image area including an area corresponding to the moving object area. 5. The detection means detects a subject in an image region that is associated with the m-th image and is identified as an image region in which the subject is detected a predetermined number of times or more among the image regions included in the m-th image. The image processing apparatus according to claim 1, wherein processing is performed. An image processing method comprising:
An acquisition step in which the acquisition means acquires the first image;
The detection means performs subject detection processing on the first to N images when the image generated by recursively reducing the first image is an Nth image (N is a natural number of 2 or more). a detecting step, among the image regions included in the m image (1 ≦ m ≦ N), and a said detection step of performing subject detection processing on the image region associated with said m image An image processing method.   A computer program for causing a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 5.

Images