JP2019061407A - Object detection device - Google Patents

Abstract

To detect a position of an individual object with high accuracy from an image photographed in a possible object congestion occurring space regardless of its congestion state.SOLUTION: Density estimating means 50 analyzes an arbitrary area in a photographed image to estimate a congestion degree of an object photographed in the area. Candidate position extraction means 51 uses a single classifier which has learned a feature of a photographed single object image to extract a candidate position having the feature of the single image in the photographed image. Group generation means 52 sets a lower limit regarding a proximity degree of the candidate position to be higher as a congestion degree in the photographed image is higher, and generates a candidate position group including candidate positions closer than the lower limit. Object position determination means 53 determines a position of the object on the basis of the candidate positions belonging to the candidate position group for each candidate position group.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an object detection apparatus for detecting the position of an individual object from a photographed image in which a space where an object such as a human can exist is photographed, and in particular, an individual object from a photographed image in which a space where congestion may occur is photographed. Object detection apparatus for detecting the position of

  In a space such as an event site where congestion may occur, early detection of a suspicious person taking an abnormal action is required to prevent occurrence of panic or the like. In order to respond to this request, for example, it is expected that surveillance cameras will be arranged at various places in the hall to estimate the distribution of people from photographed images and display the estimated distribution to facilitate understanding of the congestion situation by the surveillance staff Be done. Then, at that time, if the position of each person is detected and the position of each person is shown by displaying a model imitating the shape of the person at each detected position, the monitoring efficiency is further improved. I can expect it.

  As one of the methods for detecting the position of each person from the photographed images taken by a plurality of people, searching for the photographed images using a classifier that previously learned the feature amounts of the images taken by a single person There is a method of detecting the position where the image feature of a single person appears from the photographed image according to.

  In the search processing using a classifier, a plurality of candidate positions may be extracted in proximity to each other, and generally, positions of each individual are determined based on a plurality of candidate positions extracted in proximity. For example, the target detection device described in Patent Document 1 below extracts candidate areas where the index value (score of the classifier) exceeds the first threshold, and from a plurality of candidate areas extracted in duplicate at a certain ratio or more Create an area group that Then, the one with the highest score is detected as a target area (a human area) from each area group, or a plurality of candidate areas forming the area group are averaged for each area group to detect a target area.

JP, 2016-040705, A

  However, in a photographed image obtained by photographing an event hall or the like, candidate areas of adjacent persons in a crowded area may be extracted redundantly at a certain ratio or more. Therefore, in the related art, one area is determined from an area group in which a plurality of candidate areas are mixed, which may cause a failure in detection.

  On the other hand, if area groups are generated in a narrow range in order to prevent detection failure, a plurality of area groups will be generated even though only one person is photographed in an area where congestion does not occur, resulting in excessive detection. There is a fear.

  As described above, when the candidate areas (candidate positions) of the object are integrated on a constant basis regardless of the congestion state, the accuracy of detecting the position of the object by the difference in the congestion state or the change in the congestion state for each area There was a problem that decreased.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an object detection apparatus capable of detecting the positions of individual objects with high accuracy regardless of congestion.

  (1) An object detection apparatus according to the present invention is an apparatus for detecting the position of an individual object from a photographed image in which a space in which congestion due to the object may occur is photographed, and analyzes an arbitrary region in the photographed image Using the congestion estimation means for estimating the congestion degree of the object photographed in the area, and the simple discriminator which has learned the features of the simple image in which the single object is photographed, in the photographed image Candidate position extracting means for extracting candidate positions having the following characteristics, and the lower limit regarding the proximity degree of the candidate positions is set higher as the position with higher degree of congestion in the captured image is higher, Group generation means for generating a candidate position group consisting of candidate positions; and the position of the object based on the candidate positions belonging to the candidate position group for each of the candidate position groups And a object position determining means for determining.

  (2) In the object detection device according to (1), the candidate position extraction unit extracts a candidate region having a feature of the single image based on the candidate position, and the group generation unit is configured to The proximity degree is measured by the ratio of overlapping parts of each other, and the lower limit ratio regarding the overlapping part is set larger as the position of the congestion degree in the captured image is higher, and the candidate area overlapping at the lower limit ratio is coped The candidate position group may be generated.

  (3) In the object detection device according to (1), the group generation unit measures the proximity degree by the distance between the candidate positions, and the upper limit related to the distance increases as the congestion degree in the photographed image increases. Can be set small, and the candidate position group consisting of the candidate positions at a distance below the upper limit can be extracted.

  (4) In the object detection device according to (1) to (3), the congestion estimation unit learns features of density images obtained by photographing the space in which the object is present at the predetermined density. The density of the object captured in an arbitrary area in the captured image may be estimated as the degree of congestion using the density estimator.

  (5) In the object detection device according to (4), the group generation unit extracts the candidate position with respect to the density of the object estimated by the congestion estimation unit in an arbitrary area in the captured image. The upper limit number of candidate positions constituting the candidate position group in the area is set according to the density ratio in the area of the candidate positions extracted by the means, and the candidate consisting of the candidate positions not exceeding the upper limit number It can be configured to generate location groups.

  According to the present invention, it is possible to obtain an object detection apparatus capable of detecting the positions of individual objects with high accuracy regardless of congestion.

FIG. 1 is a block diagram showing a schematic configuration of an image monitoring device according to an embodiment of the present invention. FIG. 1 is a schematic functional block diagram of an image monitoring device according to an embodiment of the present invention. It is the figure which represented typically the information of the single-piece | unit discriminator which the single-piece | unit discriminator storage means has memorize | stored. It is a schematic diagram which shows the example of the candidate position for every density class, and a candidate position group. It is a flowchart which shows the general | schematic operation | movement of the image monitoring apparatus concerning embodiment of this invention. It is a flowchart of the outline of candidate position extraction processing. It is a schematic flowchart of candidate position integration processing.

  Hereinafter, as an embodiment of the present invention, an example of an image monitoring apparatus 1 including an example of an object detection apparatus that detects an individual from an image captured of an event hall will be described. Do.

[Configuration of Image Monitoring Device 1]
FIG. 1 is a block diagram showing a schematic configuration of the image monitoring device 1. The image monitoring apparatus 1 includes a photographing unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and a display unit 6.

  The photographing unit 2 is a monitoring camera, and is connected to the image processing unit 5 via the communication unit 3 to photograph the monitoring space at predetermined time intervals to generate a photographed image, and to sequentially transmit the photographed image to the image processing unit 5 It is a photographing means to input. For example, the photographing unit 2 is installed on a pole installed at an event hall with a view over the monitoring space. The field of view may be fixed or may be changed according to a pre-schedule or an external instruction via the communication unit 3. Also, for example, the photographing unit 2 photographs the monitoring space at a frame period of 1 second to generate a color image. Instead of a color image, a monochrome image may be generated.

  The communication unit 3 is a communication circuit, one end of which is connected to the image processing unit 5 and the other end of which is connected to the photographing unit 2 and the display unit 6 via a coaxial cable or a communication network such as LAN (Local Area Network) or the Internet. Be done. The communication unit 3 acquires a captured image from the imaging unit 2 and inputs the image to the image processing unit 5, and outputs the detection result input from the image processing unit 5 to the display unit 6.

  The storage unit 4 is a memory device such as a read only memory (ROM) or a random access memory (RAM), and stores various programs and various data. The storage unit 4 is connected to the image processing unit 5, and inputs / outputs the information with the image processing unit 5.

  The image processing unit 5 is configured by an arithmetic device such as a central processing unit (CPU), a digital signal processor (DSP), and a micro control unit (MCU). The image processing unit 5 is connected to the storage unit 4 and operates as various processing means and control means by reading and executing a program from the storage unit 4 to store various data in the storage unit 4 and from the storage unit 4 read out. The image processing unit 5 is also connected to the imaging unit 2 and the display unit 6 via the communication unit 3, and detects individual persons by analyzing a captured image obtained from the imaging unit 2 via the communication unit 3, The detection result is output to the display unit 6 via the communication unit 3.

  The display unit 6 is a display device such as a liquid crystal display or a CRT (Cathode Ray Tube) display, and is connected to the image processing unit 5 via the communication unit 3 and is a display unit that displays the detection result by the image processing unit 5 . The supervisor visually recognizes the displayed detection results to determine the occurrence of congestion, etc., and takes measures such as changing the staffing as needed.

  In the present embodiment, the image monitoring device 1 in which the number of the imaging unit 2 and the number of the image processing units 5 is one to one is exemplified, but in another embodiment, the number of the imaging units 2 and the image processing unit 5 Can also be many-to-one or many-to-many.

[Function of Image Monitoring Device 1]
FIG. 2 is a functional block diagram showing the functions of the image monitoring device 1. The communication unit 3 functions as an image acquisition unit 30, an object position output unit 31, and the like, and the storage unit 4 functions as a density estimator storage unit 40, a single identifier storage unit 41, and the like. The image processing unit 5 functions as a density estimation unit 50, a candidate position extraction unit 51, a group generation unit 52, an object position determination unit 53, and the like.

  The image acquisition unit 30 sequentially acquires captured images from the imaging unit 2 which is an imaging unit, and sequentially outputs the acquired captured images to the density estimation unit 50 and the candidate position extraction unit 51.

  The density estimator storage means 40 is an estimated density calculation function which learns the image features of each image (density image) obtained by photographing a space where an object (person) exists at the density for each predetermined density. When the feature amount is input, information representing an estimator (density estimator) which calculates and outputs an estimated value (estimated density) of the density of the object captured in the image is stored in advance. That is, the density estimator storage means 40 stores in advance parameters such as the coefficients of the estimated density calculation function as information of the density estimator.

  The density estimating means 50 analyzes an arbitrary area in the photographed image inputted from the image acquiring means 30, and estimates the density of the object photographed in the area as the degree of congestion of the object in the area (congestion degree) Congestion estimation means. Specifically, the density estimation unit 50 extracts a feature amount (feature amount for estimation) for density estimation from a photographed image of an arbitrary region and reads out a density estimator from the density estimator storage unit 40 and extracts the extracted feature amount. The density is estimated by inputting each of the estimation feature quantities into the density estimator. By performing this estimation at a plurality of positions in the photographed image, the distribution of the estimated density (density distribution of the object) in the photographed image can be obtained, and the density estimating unit 50 can estimate the density distribution as candidate position extracting unit 51 Output to

  The process of density estimation and the density estimator will be specifically described.

  The density estimation unit 50 sets a window (extraction window for estimation) at the position of each pixel of the captured image, and extracts the feature amount for estimation from the captured image in each extraction window for estimation. The feature amount for estimation is a feature of Gray Level Co-occurrence Matrix (GLCM).

  It is desirable that the areas in the monitoring space captured by the estimation extraction windows have the same size. That is, preferably, the density estimation unit 50 reads the camera parameters of the imaging unit 2 stored in advance from the camera parameter storage unit (not shown), and the image is captured at an arbitrary pixel of the captured image by homography conversion using the camera parameters. After the photographed image is deformed so that the area in the monitored space becomes the same size, the feature quantity for estimation is extracted.

  The density estimator can be realized by a classifier that identifies multiple classes of images, and can be a classification function learned by the multiclass Support Vector Machine (SVM) method.

The density is, for example, a "background" class in which there are no people, a "low density" class which is higher than 0 person / m ^{2 and not} more than 2 people / m ^{2, and not} less than 2 people / m ^{2 and not} more than 4 people / m ² It can be defined as four classes of "medium density" class and "high density" class higher than 4 persons / m ² .

  The estimated density is a value given in advance to each class, and is a value output as a result of distribution estimation. In the present embodiment, values corresponding to each class are described as “background”, “low density”, “medium density”, and “high density”.

  That is, the density estimator applies the multi-class SVM method to the feature quantities of a large number of images (density images) belonging to the "background" class, the "low density" class, the "medium density" class, and the "high density" class. It is a discriminant function for discriminating density images of each class from other classes obtained by learning. The parameters of the discriminant function derived by this learning are stored as a density estimator. The feature amount of the density image is the same as the feature amount for estimation, and is a GLCM feature.

  The density estimation means 50 acquires estimated density which is the output value by inputting each of the feature quantities for estimation extracted corresponding to each pixel to the density estimator. When the photographed image is deformed and the estimation feature amount is extracted, the density estimating unit 50 deforms the density distribution into the shape of the original photographed image by homography conversion using camera parameters.

  A group of estimated densities for each pixel of the captured image obtained in this manner is a density distribution. Here, although the density condition of the person in each place of the photographed image can be known from the density distribution outputted by the density estimating means 50, the position of the individual person can not be known from the density distribution. On the other hand, candidate position extraction means 51, group generation means 52 and object position determination means 53 provided after density estimation means 50 are means for detecting the position of each person appearing in the photographed image.

  The single identifier storage means 41 stores in advance a identifier (single identifier) that has learned the features of an image (single image) in which a single person (object) is photographed.

  FIG. 3 is a view schematically showing the information of the single discriminator stored in the single discriminator storage means 41. As shown in FIG.

  The simple discriminator calculates the evaluation value (identification score) that represents the likelihood that the image is a simple image when the feature amount of the image is input, and outputs the evaluation score calculation function coefficient and the discrimination score. It is represented by parameters such as a threshold value applied to it.

  The single-piece discriminator can be a discriminator learned by applying the linear SVM method to the feature amount of a learning image consisting of a large number of single-piece images and a large number of unmanned images in which each person is only a person.

  When a linear SVM is used as a learning algorithm, the coefficient of the evaluation value calculation function is a weight vector. The weight vector is a weight for each element of the feature amount, and the value of the inner product of the feature amount of the input image and the weight vector represents the identification score. In learning, when the inner product of the weight vector and the feature amount is larger than 0, it is adjusted so as to be identified as a person, and when less than 0, it is identified as other than a person. Therefore, the threshold for identifying whether the input image is a single image is in principle 0, and the threshold can usually be set to 0. However, the threshold may be set to a value smaller than 0 in order to reduce an error in identifying a single image as not being a single image.

  Note that the feature amounts of the learning image are HOG (Histograms of Oriented Gradients) feature amounts.

  The single-piece classifier stored in the single-piece classifier storage unit 41 is a classifier that has learned image features of a small part of parts constituting a single object as the density is higher. The single discriminator storage means 41 is a single discriminator that is a single discriminator that has learned the image feature of a single person's whole body in association with a value representing a low density class, and is associated with a value representing a medium density class Upper-body discriminator 101, which is a simplex discriminator that has learned the image features of the upper 2/3 of a person, and a simplex discriminator that has learned the image characteristics of the upper 1/3 of a single person in association with a value representing a high density class The near-head identifier 102 is stored.

  The whole-body discriminator 100 is a single-piece discriminator learned using a single-body image of the whole body of a single person, and the upper-body discriminator 101 is a single-piece image of the upper two-thirds of a single person Is a single-body discriminator learned using an image obtained by cutting out the upper 2/3 of a single-body image taken, etc., and the near-head part discriminator 102 is a single-piece image in which the upper third of a single person is photographed ( It is a single-piece discriminator learned using an image etc. which cut out the upper 1/3 of the single-piece picture by which the whole human body was photoed.

  As described above, the single identifier storage unit 41 associates the whole body identifier 100 with the low density class, associates the full body identifier 100 with the medium density class, associates the upper body identifier 101 with the high density class, and the near head identifier. 102 is stored.

  The candidate position extraction means 51 reads out the simplex discriminator from the simplex discriminator storage means 41, extracts the candidate position having the feature of the simplex image in the photographed image using the read out simplex discriminator, and groups the extracted candidate positions It is output to the generation means 52.

  Specifically, first, the candidate position extraction unit 51 sets a plurality of evaluation positions in the captured image at predetermined intervals, and sets a window for identification (an identification extraction window) based on each evaluation position. For example, the candidate position extraction unit 51 sets an evaluation position at an interval of one pixel in the entire photographed image, and sets the extraction window for identification with the position of each pixel as the evaluation position indicating the center of gravity of a person's head. .

  And candidate position extraction means 51 which set the extraction window for identification extracts the feature-value of the image in each extraction window for identification, The identification score of each evaluation position is input by inputting the extracted feature-value to a single-piece | unit discriminator To get

  At this time, in order to make the image used for identification as large as possible while taking into consideration the occlusion due to congestion, the candidate position extraction means 51 makes the extraction window for identification smaller as the density of objects at each evaluation position is higher, and the lower the density is. The discrimination extraction window is set large, and a single discriminator corresponding to the size of the discrimination extraction window is used.

  For that purpose, the candidate position extraction unit 51 sets a window defined in the shape of the upper 1/3 of a single person at each evaluation position and refers to the density distribution input from the density estimation unit 50, and The largest estimated density at is determined as the density of the evaluation position.

  Then, the candidate position extraction means 51 sets the identification extraction window defined in the shape of the whole body of a single person at the evaluation position where the density is low, and for the single identification from the photographed image in the identification extraction window. A feature amount (feature amount for identification) is extracted, and the extracted feature amount is input to the whole-body classifier to acquire an identification score. Further, the candidate position extraction means 51 sets the identification extraction window defined in the shape of the upper part 2/3 of a single person at the evaluation position where the density is medium density, and identifies from the photographed image in the identification extraction window Feature amounts are extracted, and the extracted feature amounts are input to the upper body classifier to obtain an identification score. Further, the candidate position extraction means 51 sets an identification extraction window defined in the shape of the upper 1/3 of a single person at an evaluation position having a high density and identifies it from the photographed image in the identification extraction window Feature quantities are extracted, and the extracted feature quantities are input to the near-head classifier to obtain a discrimination score.

  Thus, the candidate position extraction unit 51 that has acquired the identification score for each evaluation position compares each identification score with a predetermined threshold, and extracts an evaluation position whose identification score is equal to or higher than the threshold as a candidate position. For example, as described above, in the case of using an SVM in which the threshold value for dividing a person from others is defined as 0, the candidate position extraction unit 51 extracts an evaluation position at which a discrimination score larger than 0 is acquired. Then, the candidate position extraction unit 51 generates information (candidate position information) in which the candidate position, the density, the identification score, the threshold value of the single discriminator used, and the extraction window for identification used are associated for each candidate position. Output to

  In many cases, a plurality of candidate positions extracted in this manner are extracted for each individual person. Therefore, the group generation unit 52 generates a group (candidate position group) consisting of one or more candidate positions considered to be of the same person among the candidate positions, and the object position determination unit 53 sets one candidate position for each candidate position group. Integrate into one to determine the position (object position) of each person.

  FIG. 4 is a schematic view showing an example of candidate positions and candidate position groups for each density class. FIG. 4A is an example of the low density region, FIG. 4B is an example of the medium density region, and FIG. 4C is an example of the high density region. The left part of each of FIGS. 4A to 4C shows an example of an identification extraction window corresponding to a plurality of candidate positions extracted in the vicinity of a person. Specifically, in the low density region of FIG. 4A, four candidate positions are extracted in the vicinity of one person 200, and in correspondence with this, four discrimination extractions of shape / size corresponding to the whole body A window 201 is shown. In the medium density region of FIG. 4B, five candidate positions are extracted in the vicinity of two persons 210 and 211 in proximity, and the shape / size 5 corresponding to the upper two thirds of the person correspondingly One identification extraction window 212 is shown. In the high density region of FIG. 4C, seven candidate positions are extracted in the vicinity of three persons 220 to 222 in proximity, and the shape / size 7 corresponding to the upper one-third of the person correspondingly One identification extraction window 223 is shown. The right side of FIGS. 4A to 4C shows an example of a candidate position group generated from the candidate positions shown in the left side of each.

  Here, when the candidate position group is generated in a wide range because people are close to each other in the area where the high density is estimated, an error occurs to generate one candidate position group from a plurality of candidate positions. It fails to detect the position. On the other hand, since candidate positions for the same person can be extracted in a wider range than a region where high density is estimated in a region where low density is estimated, when candidate position groups are generated in a narrow range, candidates for the same person are candidates. An error occurs to generate a plurality of candidate position groups from the positions, and the position of the person is excessively detected.

  Therefore, the group generation unit 52 refers to the density estimated at the candidate position, and generates a candidate position group consisting of candidate positions extracted in a narrower range (integrated range) as the density in the captured image is higher. For example, as shown in the right side of FIG. 4A, as a result of the wide integration range in the low density region, one candidate position group is obtained from all four candidate positions shown by the discrimination extraction window 201 near one person 200. 202 may be generated. On the other hand, as a result of the integration range being set narrow in the medium density region, as shown on the right side of FIG. 4B, the two candidate positions 210, 211 are supported from five candidate positions shown by the extraction window 212 for identification. As shown in the right side of FIG. 4C, in the high density area where the two candidate position groups 213 and 214 are generated and the integration range is set to be narrow, seven candidate positions indicated by the extraction window 223 for identification Three candidate position groups 224-226 may be generated corresponding to the three persons 220-222.

  The integrated range can be defined using any measure indicating the degree of proximity of candidate positions. That is, the group generation unit 52 sets a lower limit on the proximity degree, and generates a candidate position group as being within the integrated range if it is closer to the lower limit or more. Then, at that time, the group generation unit 52 sets the lower limit regarding the proximity degree of candidate positions higher as the density in the captured image is higher, and lowers the lower limit as the density is lower.

  The group generation unit 52 assigns an identifier of the candidate position group to the candidate position information of the candidate position belonging to each of the generated candidate position groups, and outputs information of each candidate position group to the object position determination unit 53.

Specifically, the group generation unit 52 can measure the proximity degree by the overlapping ratio of the extraction windows for identification (candidate regions) set corresponding to the respective candidate positions, and can control the integration range. That is, the lower limit ratio is set for the overlap ratio, and the case where the extraction windows for identification overlap with each other is set as the integrated range. Then, the group generation unit 52 sets a lower limit ratio that increases as the density in the captured image increases, and extracts a candidate position group consisting of a plurality of candidate areas overlapping at the lower limit ratio or more. For example, the overlapping ratio of candidate areas A and B is defined by equation (1), the lower limit ratio is set to 0.5 for a candidate position having a low density, and a candidate position having a medium density For this, the lower limit ratio is set to 0.65, and the lower limit ratio is set to 0.8 for candidate positions with high density. In Equation (1), S _A , S _B and S _{A ∩ B} respectively represent the area of the candidate area A, the area of the candidate area B, and the area of the overlapping portion of the candidate areas A and B.

Furthermore, the group generation unit 52 can use candidate position extraction unit 51 with respect to the density of the object estimated by the density estimation unit 50 in an arbitrary area in the photographed image in order to prevent detection failure due to the deviation of the arrangement of persons. The upper limit number of candidate positions constituting the candidate position group in the area is set according to the density ratio of the candidate positions extracted by the target area, and the candidate position group consisting of candidate positions not exceeding the upper limit number is generated Extract. Specifically, for each density estimated by the density estimation means 50, the group generation means 52 constructs a candidate position group according to the number of extracted candidate positions and the size of the region for which the density is estimated. Set the upper limit number of candidate positions. For example, if two people / m higher than ² Quadruple / m area was estimated to density class within learned ² from the density image is 3.5 m ² equivalent, the number of the person being photographed to the area It is estimated to be 7 to 14 people. Then, for example, when 40 candidate positions are extracted from the area, the number of candidate positions constituting the candidate position group is estimated to be 2.9 to 5.7 on average. Corresponding to this, the group generation unit 52 sets the upper limit number of candidate positions constituting the medium density candidate position group to six.

  The object position determination means 53 determines the candidate position having the largest identification score among the candidate positions constituting the candidate position group as the object position, for example, for each candidate position group. Then, the object position determination means 53 outputs the information of the determined object position to the object position output means 31.

  The object position output unit 31 sequentially outputs the information on the object position input from the object position determination unit 53 to the display unit 6, and the display unit 6 displays the information on the object position input from the object position output unit 31. For example, information on the position of an object is transmitted and received via the Internet and displayed on the display unit 6. By watching the displayed information, the surveillance personnel grasps a point where congestion occurs in the surveillance space, and takes measures such as dispatching or increasing security personnel to the point.

[Operation of Image Monitoring Device 1]
The operation of the image monitoring device 1 will be described with reference to the flowcharts of FIG. 5, FIG. 6 and FIG.

  When the image monitoring apparatus 1 starts operation, the imaging unit 2 installed in the event hall captures an image of the monitoring space at predetermined time intervals and sequentially transmits the captured image to the image analysis center where the image processing unit 5 is installed. Do. Then, the image processing unit 5 repeats the operation according to the flow chart of FIG. 5 every time the photographed image is received.

  First, the communication unit 3 operates as the image acquisition unit 30 and waits for reception of a photographed image from the photographing unit 2. The image acquisition unit 30 which has acquired the photographed image outputs the photographed image to the image processing unit 5 (step S1).

  The image processing unit 5 which receives the photographed image operates as the density estimating unit 50, and estimates the density distribution from the photographed image (step S2). The density estimating means 50 extracts the feature quantity for estimation at the position of each pixel of the photographed image and reads the density estimator from the density estimator storage means 40 of the storage unit 4 and uses the feature quantities for estimation as the density estimator. The density distribution in the photographed image is estimated by inputting and acquiring the estimated density at each pixel of the photographed image.

  The image processing unit 5 that has estimated the density distribution also operates as the candidate position extraction unit 51, and the candidate position extraction unit 51 receives the photographed image from the image acquisition unit 30 and the density distribution from the density estimation unit 50. . The candidate position extraction means 51 which has input these checks whether or not the density distribution includes an estimated density other than the background class (step S3).

  If an estimated density other than the background class is included (YES in step S3), the candidate position extraction unit 51 determines that at least one person is photographed, and the candidate positions of individual objects from the photographed image The process of extracting is performed (step S4). On the other hand, in the case of only the background class (NO in step S3), the process of steps S4 and S5 is omitted, assuming that no person is photographed.

  The candidate position extraction process of step S4 will be described with reference to the flowchart of FIG.

  The candidate position extraction unit 51 sequentially sets the position of each pixel in the captured image as the evaluation position (step S400). Then, the candidate position extraction unit 51 specifies the density of the evaluation position with reference to the density distribution input from the density estimation unit 50 (step S401). Specifically, the candidate position extraction means 51 sets a window defined in the shape of the top 1/3 of a single person at the evaluation position, and specifies the largest estimated density in the window as the density of the evaluation position.

  The candidate position extraction means 51 which specified the density reads the single-body discriminator according to the density from the single-body discriminator storage means 41, sets the extraction window for identification according to the density, and picks up the photographed image in the extraction window for identification The feature amount for identification is extracted from (step S402), and the extracted feature amount for identification is input to the single discriminator according to the density to calculate an identification score (evaluation value) (step S403).

  Then, if the evaluation value of the evaluation position exceeds the predetermined threshold (YES in step S404), the candidate position extraction unit 51 sets the evaluation position as a candidate position of the object, and generates candidate position information (step S405). ). On the other hand, if the evaluation value of the evaluation position does not exceed the predetermined threshold (NO in step S404), the evaluation position is not considered as a candidate position, and the process of step S405 is omitted.

  When the candidate position extraction unit 51 finishes the process of setting a certain pixel as the evaluation position in steps S404 and S405, the candidate position extraction unit 51 confirms whether or not the positions of all the pixels of the captured image have been set as the evaluation position (step S406). If there is an unset pixel (NO in step S406), the process returns to step S400 to process the position of the next pixel.

  On the other hand, when candidate position extraction means 51 sets the positions of all the pixels as evaluation positions and finishes the extraction process of the candidate positions (YES in step S406), the generated candidate position information is output to group generation means 52 The process proceeds to step S5 of FIG. The candidate position extraction unit 51 outputs the generated candidate position information to the group generation unit 52.

  The candidate position integration process of step S5 will be described with reference to the flowchart of FIG.

  The group generation unit 52 generates a list in which candidate position information is arranged in descending order of evaluation value (step S500). The group generation unit 52 sets the lower limit ratio and the upper limit number according to the density of the candidate position at the head of the list (step S501), and initializes the number of members of the candidate position group to "1" (step S502).

  The group generation unit 52 sequentially sets the second and subsequent candidate position information of the list as comparison position information (step S 503), and the extraction window for identification of the candidate position at the head of the list and the candidate position of comparison position information (comparison position) Calculate the overlapping ratio with the extraction window for identification. When the overlapping ratio exceeds the lower limit ratio set in step S501 (YES in step S504), the group generation unit 52 sets the comparison position as the same candidate position group as the candidate position at the top of the list, and the comparison position The information is deleted from the list (step S505), and the number of members is increased by 1 (step S506).

  When the number of members increased in step S506 does not reach the upper limit number set in step S501 (NO in step S507), the group generation unit 52 determines whether the comparison position information is the end of the list or not. (Step S508). If it is determined in step S504 that the overlapping ratio is equal to or less than the lower limit ratio (NO in step S504), the processing in steps S505 to S507 is omitted and the determination in step S508 is performed.

  When the comparison position information does not indicate the end of the list (NO in step S508), the group generation unit 52 repeats the processing of steps S503 to S508, and when the end of the list is reached (YES in step S508), the current list head candidate End the extraction of candidate position groups for the position.

  Further, when the number of members increased in step S506 reaches the upper limit number (YES in step S507), the group generation unit 52 determines the current candidate position of the top of the list even if the comparison position information is not the end of the list. Finish extracting candidate position groups for.

  When the candidate position group for the candidate position at the top of the list is generated by the group generation unit 52, the object position determination unit 53 determines the candidate position with the largest evaluation value in the candidate position group, that is, the candidate position at the top of the list (Step S509).

  Further, the group generation unit 52 deletes candidate position information on the top of the list for which the generation processing S501 to S508 of the candidate position group is completed (step S510). If candidate position information remains in the list after the deletion processing in step S510 (NO in step S511), the group generation unit 52 returns the processing to step S501, and the candidate position group for the candidate position at the top of the new list Generate. On the other hand, when the list is empty (YES in step S511), the candidate position integration process S5 ends, and the process proceeds to step S6 in FIG. The candidate position extraction unit 51 outputs the generated candidate position information to the group generation unit 52.

  Description will be continued with reference to FIG. 5 again. The object position determination means 53 outputs the information on the object position determined in step S5 to the communication unit 3 (step S6), and the communication unit 3 operates as the object position output means 31 to display the information on the object position on the display unit 6. Send.

  When the above process is completed, the process is returned to step S1, and the process is performed on the next captured image.

[Modification]
(1) In the above embodiment, the group generation unit 52 measures the proximity of candidate positions by using the overlapping ratio of the extraction windows for identification (candidate regions) set corresponding to the candidate positions as a measure. Although the distance between the candidate positions is used as a measure instead of the overlapping ratio, the proximity degree can also be measured by the distance between the candidate positions. In this configuration, the group generation unit 52 sets an upper limit for the distance between candidate positions. Then, the group generation unit 52 sets an upper limit distance that is shorter as the congestion degree in the captured image is higher, and extracts a candidate position group consisting of a plurality of candidate positions located at a distance equal to or less than the upper limit distance. In this case, for example, the group generation unit 52 sets the upper limit distance to 60 pixels for candidate positions having a low density and sets the upper limit distance to 40 for candidate positions having a medium density. The pixels are set as pixels, and the upper limit distance is set as 30 pixels for candidate positions having high density, and candidate position groups are extracted for each density.

  (2) In the above embodiment and the modification thereof, extraction of a candidate position group including setting of the upper limit number of candidate positions constituting the candidate position group and processing of the group generation unit 52 The processing to be performed can be omitted.

  (3) In the above embodiment and its modification, the object position determination means 53 determined the candidate position having the largest evaluation value as the object position, but it is also possible to determine the average value or the weighted average value of the candidate positions as the object position Good. That is, the object position determination means 53 determines the average value of the candidate positions constituting the candidate position group as the object position for each candidate position group, or the candidate positions constituting the candidate position group for each candidate position group The weighted average value is determined as the object position by weighting with the evaluation value of the candidate position (the evaluation value shifted so that all become positive if negative evaluation values can be included) and averaged.

  (4) In the above embodiment and its modification, an example in which the object to be detected is a human is shown, but the invention is not limited to this, and the object to be detected is a vehicle, a fixture such as a chair or desk, a cow, a sheep, etc. The animals can also be In addition, a plurality of types of detection targets can be used such as one type of person, a chair, and a desk (detection in a space in which a plurality of types of objects are mixed).

  (5) In the above-described embodiment and the modification thereof, the one that identifies the whole human body, the upper 2/3 and the upper 1/3, is used as a single body identifier that corresponds to each density class. Is an example, and different settings suitable for each can be made according to differences in the detection target, the characteristics of the monitoring space to be photographed, the feature value to be adopted, the type of evaluation value, and the like.

  (6) In the above embodiment and its modification, the density estimator learned by the multiclass SVM method is exemplified, but instead of the multiclass SVM method, a decision tree type random forest method, multiclass adaboost ( It can be various density estimators, such as a density estimator learned by AdaBoost) method or multiclass logistic regression method.

  Alternatively, it may be a density estimator using a discrimination type CNN (Convolutional Neural Network).

  (7) In the above embodiment and its modification, the class of density other than the background estimated by the density estimator is three classes, but the classes may be divided more finely.

  In that case, instead of the single-class classifiers of the three stages (whole body, upper body and vicinity of the head), a single-class classifier of finer stages corresponding to classification is used 41 can be stored. Alternatively, the class and the three-step single classifiers can be associated in many-to-one correspondence and stored in the single classifier storage unit 41.

  (8) In the above embodiment and its modification, the density estimator classified into multiple classes is exemplified, but instead, a regression type density estimator that regresses the value of density (estimated density) from the feature amount and You can also That is, a density estimator which learns parameters of a regression function for obtaining an estimated density from feature amounts by ridge regression method, support vector regression method, regression tree type random forest method, Gaussian process regression, or the like can do.

  Alternatively, it may be a density estimator using a regression type CNN.

  In these cases, the value range of the estimated density, which is output as a continuous value instead of the value of the density class, is stored in the single identifier storage unit 41 in association with the single identifier.

  (9) In the above embodiment and its modification, GLCM features are exemplified as feature quantities to be learned by the density estimator and feature quantities for estimation. However, these may be replaced with GLCM features, and local binary patterns (Local Binary Pattern) may be used. : LBP) A variety of feature quantities such as a feature quantity, a Haar-like feature quantity, an HOG feature quantity, a luminance pattern, etc., or a feature quantity combining a plurality of GLCM features and these You can also.

  (10) In each of the above-described embodiments and their modifications, the density estimation unit 50 serving as the congestion estimation unit estimates the density as the congestion degree of the object. The degree of congestion can also be estimated by analysis. For example, the congestion estimation means divides the photographed image into areas for adjacent pixels having similar colors, counts the divided areas for each predetermined block, and calculates the complexity of the height according to the count value (previously Calculated based on the relationship determined by experiment, the higher the complexity, the greater the count value. Alternatively, the congestion estimation means performs frequency analysis of the captured image for each predetermined block to determine the complexity of the height according to the height of the peak frequency (the higher the peak frequency is, the higher the complexity determined through experiments) Calculated based on the relationship that determines Then, the congestion estimation means estimates the degree of congestion at a height corresponding to the degree of complexity for each block (calculated based on a relationship determined through preliminary experiments, the higher the degree of congestion, the higher the degree of congestion is determined).

  (11) In the above embodiment and the variation thereof, the projection area of the model defined in the shape of the upper 1/3 of the person or the window defined in the shape is set at the candidate position of interest and the area is set In this example, the estimated density at the candidate position is determined by adding up the estimated density in the area, but in order to reduce the amount of processing, the pixel at the candidate position, eight neighboring areas of the candidate position or It can also be a small area such as a near area. Alternatively, in order to increase the accuracy, instead of the area concerned, the projection area of the model defined in the shape of the upper 2/3 of a single person whose representative position is the representative position or the window defined in the shape or the candidate position The projection area of the model defined in the shape of the whole body of a single person taken as a representative position or a large area such as a window defined in the shape can also be used.

  (12) The threshold value to be compared with the discrimination score shown in the above embodiment and the modification thereof may be a different value for each single classifier.

  (13) In the above embodiment and its modification, the single discriminator learned by the linear SVM method is exemplified, but instead of the linear SVM method, learning is performed using various known learning methods such as the Adaboost method. It can also be used as a single identifier. In addition, a pattern matching unit can be used instead of the classifier, and the classification score in that case is the inner product of the average pattern of the feature quantities extracted from the human learning image and the feature quantity of the input image, etc. The function can be a function having the score as an output value and the feature amount of the input image as an input value. Alternatively, a discrimination type CNN may be used as a single discriminator. In particular, when using the R-CNN (Regions with CNN features) method or the like which performs estimation processing of the size of the extraction window for identification in addition to identification processing, the size of the extraction window for identification which is a candidate region is variable. be able to. The R-CNN method is described, for example, in "Rich feature hierarchies for accurate object detection and semantic segmentation", Ross Girshick et al., CVPR 2014.

  (14) Although the HOG feature has been exemplified as the feature to be learned by the simplex discriminator in the above embodiment and the variation thereof, these are local binary pattern feature, Haar-like feature, instead of the HOG feature. It may be various feature quantities such as a luminance pattern, or it may be a feature quantity combining a HOG feature quantity and a plurality of these.

  Reference Signs List 1 image monitoring apparatus, 2 imaging unit, 3 communication unit, 4 storage unit, 5 image processing unit, 6 display unit, 30 image acquisition unit, 31 object position output unit, 40 density estimator storage unit, 41 single classifier storage unit , 50 density estimation means, 51 candidate position extraction means, 52 group generation means, 53 object position determination means, 100 whole body discriminator, 101 upper body discriminator, 102 head near discriminator.

Claims (5)

An object detection apparatus for detecting the position of each of the objects from a captured image obtained by capturing a space in which congestion due to the object may occur.
Congestion estimation means for analyzing an arbitrary area in the photographed image to estimate the degree of congestion of the object photographed in the area;
Candidate position extraction means for extracting a candidate position having a feature of the single image in the captured image using a single classifier that learns the features of the single image in which the single object is captured;
A group generation unit configured to set a lower limit regarding the proximity degree of the candidate positions to a higher position as the congestion degree in the captured image is higher, and generate a candidate position group including the candidate positions closer to the lower limit or more;
Object position determining means for determining the position of the object based on the candidate positions belonging to the candidate position group for each of the candidate position groups;
An object detection apparatus comprising: The candidate position extraction unit extracts a candidate area having a feature of the single image based on the candidate position.
The group generation unit measures the proximity degree by the ratio of overlapping portions of the candidate areas, and sets the lower limit ratio regarding the overlapping portion to a larger value as the congestion degree is higher in the photographed image, Generating the candidate position group corresponding to the overlapping candidate region;
The object detection apparatus according to claim 1, characterized in that   The group generation unit measures the proximity degree based on the distance between the candidate positions, sets the upper limit regarding the distance to a smaller position as the congestion degree is higher in the photographed image, and the candidate position is a distance below the upper limit The object detection apparatus according to claim 1, wherein the candidate position group consisting of   The congestion estimation means may be captured in an arbitrary area in the captured image using a density estimator that learns features of each of the density images obtained by capturing the space in which the object is present at the density for each predetermined density. The object detection apparatus according to any one of claims 1 to 3, wherein the density of the object is estimated as the degree of congestion.   The group generation means is a ratio of the density in the area of the candidate position extracted by the candidate position extraction means to the density of the object estimated by the congestion estimation means in an arbitrary area in the photographed image. According to the present invention, the upper limit number of candidate positions constituting the candidate position group in the area is set according to the condition, and the candidate position group consisting of the candidate positions equal to or less than the upper limit number is generated. The object detection apparatus as described.

Images