JP2019032588A - Image analysis apparatus - Google Patents

Abstract

To solve a problem in which analysis accuracy changes according to the difference in a congestion degree, in an apparatus which analyzes motion of a moving object from a photographed image.SOLUTION: Image acquisition means 30 acquires photographed images at a plurality of time points obtained by photographing a space to be crowded in a predetermined moving object. Density estimation means 50 estimates the density of the moving object photographed to any region in the photographed image, by using a density estimator which has learned an image feature of each density image obtained by photographing the space in which the moving object is present at the density for each predetermined density. Region division means 51 divides each of the division regions obtained by dividing the photographed image for each of plural classes set with respect to the density based on the estimated density, into a plurality of local regions in accordance with a division reference defined for each class. Motion vector calculation means 52 calculates a motion vector in each local region. Behavior to be closely observed detection means 53 analyzes a motion of the moving object in the space from the motion vector of the plurality of local regions.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image analysis apparatus that analyzes the movement of a moving object from an image of a space in which congestion can occur due to a moving object such as a person.

  A motion vector is known as one of basic information for analyzing the motion of a moving object photographed in an image.

  In the following Patent Document 1, a local region centered on the coordinates of feature points is set, and by performing an optical flow analysis using the local region as an analysis unit, a motion vector of a person or an arrangement is calculated, and the person or the arrangement is calculated. An image monitoring device that analyzes the movement of an object is illustrated. Usually, the size of this local region is predetermined.

  Patent Document 2 below exemplifies a crowd analysis device that calculates a motion vector from each of a plurality of spatiotemporal segments generated by performing spatiotemporal segmentation on a time-series image and analyzes a human motion. The spatiotemporal segmentation at that time is performed by combining the spatiotemporal segments according to the criteria shown in Equation (1) in the 0035 paragraph. In the equation, the value of α of the relaxation term α / N determines the ease of coupling of the spatio-temporal segments, and α is a preset value.

  As described above, in the prior art, a reference (hereinafter referred to as a division reference) for dividing a captured image for motion vector analysis is fixedly set.

JP 2013-143068 A JP 2017-068598 A

  In the conventional method, regardless of the degree of congestion (hereinafter referred to as density) of the moving object in the space where the image is captured, the captured image is always divided into local regions based on the same division criterion, and the motion vector is calculated. When the density fluctuates, the accuracy of analyzing the movement of a moving object may decrease.

  In other words, the higher the density of people, the easier the human images are in close contact with each other, and the lower the density of people, the more likely the human images are separated. Therefore, for example, when the density of a person is low, it is preferable to analyze the detailed movement by setting a local region that is about the size of a human part (hand, head, etc.) or smaller than the part. However, even if congestion occurs and the density of people increases, if the motion is analyzed with the setting as it is, confusion with nearby human parts often occurs and erroneous motion vectors are frequently calculated.

  As described above, when motion vectors are always calculated based on the same division criterion, motion vectors are erroneously calculated due to fluctuations in congestion, and the accuracy of analyzing the motion of a moving object is reduced.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an image analysis apparatus capable of analyzing the movement of a moving object with high accuracy from an image of a space in which congestion due to a moving object such as a person may occur. And

  (1) An image analysis apparatus according to the present invention includes an image acquisition unit that acquires a plurality of time-captured images obtained by capturing a space that can be crowded with a predetermined moving object, and the moving object has a predetermined density for each moving object. Density estimation means for estimating the density of the moving object imaged in an arbitrary region in the captured image using a density estimator that has learned the image characteristics of each density image captured of an existing space; Area dividing means for dividing each of the divided areas obtained by dividing the captured image into a plurality of classes set with respect to the density based on the density, into a plurality of local areas in accordance with a division criterion defined for each class; Motion vector calculating means for calculating a motion vector in each of the local regions, and analyzing the motion of the moving object in the space from the motion vectors of the plurality of local regions Comprising a motion analysis means.

  (2) In the image analysis apparatus according to (1), the division criterion is a region having a size determined in advance with respect to the size of the moving object as the local region, and the higher the density the higher the class. The size can be set large.

  (3) In the image analysis device according to (1), the division criterion is a region including pixels similar to each other based on pixel similarity defined by a pixel value and a pixel position, and It can be determined that the higher the density, the larger the size of the local region.

  (4) In the image analysis device according to (3), the division criterion can be set such that the higher the density, the smaller the number of the local regions per unit area.

  (5) In the image analysis apparatus according to (3), the division criterion may set the similarity threshold that is determined to be similar to each other with respect to the pixels as the density is higher.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the image analysis apparatus which can analyze the motion of a moving object with high precision from the image which image | photographed the space where the congestion by a moving object may arise.

1 is a block diagram illustrating a schematic configuration of an image monitoring apparatus according to an embodiment of the present invention. It is a functional block diagram which shows the function of the image monitoring apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the example of a picked-up image, and the example of the division area corresponding to it. It is a schematic diagram of a local area and a motion vector in each of a low congestion area, a medium congestion area, and a high congestion area. FIG. 5 is a schematic process flow diagram of a monitoring operation in the image monitoring apparatus according to the embodiment of the present invention. It is a general | schematic flowchart of an example of a gaze required action detection process in the 1st Embodiment of this invention. It is a schematic diagram which shows the example of the picked-up image which consists of each of a low congestion area | region, a medium congestion area | region, and a high congestion area | region, and the example of a local area | region with respect to it. It is a general | schematic flowchart of an example of a gaze required action detection process in the 2nd Embodiment of this invention.

  Hereinafter, an image monitoring apparatus 1 according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing a schematic configuration of the image monitoring apparatus 1. The image monitoring apparatus 1 is configured using the image analysis apparatus according to the present invention, and includes an imaging unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and a display unit 6.

  The imaging unit 2 is a monitoring camera, and is an imaging unit that is connected to the image processing unit 5 via the communication unit 3 and that captures a monitoring space in which a predetermined object can be crowded at predetermined time intervals and outputs a captured image. is there.

  For example, the imaging unit 2 is installed with a field of view overlooking the monitoring space on a pole installed at the event venue. The visual field may be fixed, or may be changed according to a schedule in advance or an instruction from the outside via the communication unit 3. Further, for example, the imaging unit 2 captures the monitoring space with a frame period of 1 second and generates a color image. A monochrome image may be generated instead of the color image.

  The communication unit 3 is a communication circuit, one end of which is connected to the image processing unit 5 and the other end is connected to the photographing unit 2 and the display unit 6 via a communication network such as a coaxial cable, a LAN (Local Area Network), or the Internet. Connected. The communication unit 3 acquires a captured image from the imaging unit 2 and inputs the acquired image to the image processing unit 5, and outputs the analysis result input from the image processing unit 5 to the display unit 6.

  The storage unit 4 is a memory device such as a ROM (Read Only Memory) or a RAM (Random Access Memory), and stores various programs and various data. The storage unit 4 is connected to the image processing unit 5 and inputs / outputs such information to / from the image processing unit 5.

  The image processing unit 5 is configured by an arithmetic device such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an MCU (Micro Control Unit). The image processing unit 5 is connected to the storage unit 4, operates as various processing units / control units by reading and executing a program from the storage unit 4, stores various data in the storage unit 4, and from the storage unit 4 read out. The image processing unit 5 is also connected to the photographing unit 2 and the display unit 6 via the communication unit 3, and analyzes a captured image acquired from the photographing unit 2 via the communication unit 3, thereby analyzing a human movement. The analysis result and the captured image are output to the display unit 6 via the communication unit 3.

  The display unit 6 is a display device such as a liquid crystal display or a CRT (Cathode Ray Tube) display, and is a display unit that is connected to the image processing unit 5 via the communication unit 3 and displays an analysis result by the image processing unit 5. . The monitor visually recognizes the photographed image while referring to the displayed analysis result to determine the occurrence of abnormal behavior or the like, and takes measures such as changing the personnel arrangement as necessary.

  In the present embodiment, the image monitoring apparatus 1 in which the number of the photographing units 2 and the image processing units 5 is 1: 1 is illustrated, but in another embodiment, the number of the photographing units 2 and the image processing units 5 is illustrated. Can be many-to-one or many-to-many.

  FIG. 2 is a functional block diagram showing functions of the image monitoring apparatus 1. The communication unit 3 functions as the image acquisition unit 30 and the gaze information output unit 31 and the like, and the storage unit 4 functions as the time-series image storage unit 40, the density estimator storage unit 41, the detection reference storage unit 42, and the like. The image processing unit 5 functions as a density estimating unit 50, a region dividing unit 51, a motion vector calculating unit 52, a gaze behavior detecting unit 53 (motion analyzing unit), and the like.

  The image acquisition unit 30 sequentially acquires captured images from the imaging unit 2 that is an imaging unit, sequentially outputs the acquired captured images to the density estimation unit 50, and additionally stores them in the time-series image storage unit 40 sequentially.

  The time-series image storage unit 40 stores the captured images input from the image acquisition unit 30 in time series, and outputs the time-series images in which the captured images are arranged in order of the shooting time to the motion vector calculation unit 52. The time-series image storage means 40 stores at least a captured image of a time interval required by the motion vector calculation means 52, and preferably deletes it when it becomes unnecessary. For example, the time-series image storage means 40 circulates and stores the current time and the captured images from one hour before the current time to four hours before (that is, the latest five frames).

  The density estimator storage unit 41 is an estimated density calculation function that learns the image features of each image (density image) obtained by capturing a space where a moving object (person) exists at a predetermined density. Is input in advance, information representing an estimator (density estimator) that calculates and outputs an estimated value (estimated density) of the density of the moving object photographed in the image is stored in advance. That is, the density estimator storage unit 41 stores parameters such as coefficients of the estimated density calculation function in advance as information on the density estimator.

  The density estimation unit 50 estimates the density of the moving object photographed in the region for any region in the photographed image input from the image acquisition unit 30. Specifically, the density estimation means 50 extracts the density estimation feature quantity (estimation feature quantity) from various locations of the captured image, reads out the density estimator from the density estimator storage means 41, and extracts the estimated value. The density is estimated by inputting each of the feature quantities to the density estimator. Thereby, an estimated density distribution (a density distribution of the moving object) in the captured image is obtained, and the density estimation unit 50 outputs the estimated density distribution to the motion vector calculation unit 52.

  The density estimation process and the density estimator will be specifically described.

  The density estimation means 50 sets a window (estimation extraction window) at the position of each pixel of the captured image, and extracts an estimation feature amount from the captured image in each estimation extraction window. The estimation feature amount is a GLCM (Gray Level Co-occurrence Matrix) feature.

  It is desirable that the area in the monitoring space photographed by each estimation extraction window is the same size. That is, preferably, the density estimation means 50 reads out the camera parameters of the photographing unit 2 stored in advance from a camera parameter storage means (not shown), and is photographed at an arbitrary pixel of the photographed image by homography conversion using the camera parameters. The estimation feature amount is extracted after the captured image is deformed so that the areas in the monitoring space have the same size.

  The density estimator can be realized by a classifier that identifies multi-class images, and can be a discrimination function learned by a multi-class SVM (Support Vector Machine) method.

Density, for example, there is no human "Background" class is 0 people / m higher than ² is two / m ² or less "low density" class, higher than two / m ² 4 persons / m ² or less It can be defined as 4 classes of “medium density” class, “high density” class higher than 4 persons / m ² .

  The estimated density is a value given in advance to each class, and is a value output as a result of distribution estimation. In the present embodiment, values corresponding to each class are expressed as “background”, “low density”, “medium density”, and “high density”.

  That is, the density estimator applies the multi-class SVM method to the feature quantities of a large number of images (density images) belonging to the “background” class, “low density” class, “medium density” class, and “high density” class. It is an identification function for discriminating images of each class obtained from learning and other classes. The parameters of the discriminant function derived by this learning are stored as a density estimator. The feature amount of the density image is the same type as the estimation feature amount and is a GLCM feature.

  The density estimation means 50 acquires the estimated density which is the output value by inputting each of the estimation feature quantities extracted corresponding to each pixel to the density estimator. Note that when the estimated feature value is extracted by deforming the photographed image, the density estimation unit 50 transforms the density distribution into the shape of the original photographed image by homography conversion using camera parameters.

  A collection of estimated densities for each pixel of the captured image thus obtained is a density distribution.

  The area dividing means 51 refers to the density distribution input from the density estimating means 50, divides the captured image according to the density, and sets a plurality of divided areas according to the division criteria determined according to the density. The image is divided into local areas, and the division result is output to the motion vector calculation means 52. Hereinafter, the area divided according to the density is referred to as a divided area.

  Specifically, the region dividing unit 51 first divides the captured image into a plurality of divided regions set for the density based on the density distribution estimated by the density estimating unit 50. In this embodiment, as a class relating to density, among the density classes output by the density estimation means 50, “background” and “low density” are integrated to define one class “low congestion”, and “medium” “Density” and “High density” are defined as classes of “medium congestion degree” and “high congestion degree”, respectively. Corresponding to these three classes, the photographed image is a low-congested area composed of a group of pixels having an estimated density of “background” class and pixels having an estimated density of “low density” class, and the estimated density is “medium density”. The area is divided into three types of divided areas: a medium crowded area composed of a group of pixels of “class” and a high crowded area composed of a group of pixels of which the estimated density is a “high density” class.

  FIG. 3 is a schematic diagram illustrating an example of a captured image and an example of a segmented area corresponding to the captured image. FIG. 3A is an example of a photographed image, and a human image 60 is shown. Further, FIG. 3B shows a segmented area, where a white portion is a low congestion region, a hatched portion is a medium congestion region, and a shaded portion is a high congestion region.

  Next, the area dividing unit 51 divides the divided area into a plurality of local areas in accordance with a density standard, that is, a division criterion determined for each degree of congestion. That is, each segmented area in the captured image is divided into a plurality of local areas according to a division criterion determined for the degree of congestion corresponding to the segmented area.

  In the present embodiment, a captured image is divided into unit blocks, and a local region is defined using the unit block as a unit. For example, the captured image may be divided into a grid at intervals estimated to be about one-eighth the size of a standing person photographed in the captured image, and a rectangular area generated thereby may be used as a unit block. it can. The area dividing means 51 sets a local area composed of a predetermined number of unit blocks according to the degree of congestion in each of the divided areas divided according to the degree of congestion, so that the captured images of the respective divided areas are set to the density. Divide into local areas of appropriate size.

  Specifically, the unit block is defined by dividing the photographed image along the X-axis direction and the Y-axis direction, with the horizontal direction of the photographed image as the X axis and the vertical direction as the Y axis. In the low congestion area, each unit block is set as a local area. Thereby, in the low congestion area, for example, a local area that is as small as a human hand and as large as a human head is set.

  In the middle congestion area, each integrated block obtained by integrating two unit blocks is set as a local area. For example, the integrated block as the local area is composed of two unit blocks adjacent in the Y-axis direction, and is arranged at an interval of 1 block in the X-axis direction and at an interval of 2 blocks in the Y-axis direction in the middle congestion area. As a result, in the medium congestion area, a local area that is as small as a person's head and as large as the upper body of the person is set.

  In the highly congested area, each integrated block obtained by integrating four unit blocks is set as a local area. For example, the integrated block as the local region is composed of 4 unit blocks having a 2 × 2 array arranged two by two in the X-axis direction and the Y-axis direction, and in the highly congested region, the X-axis direction and the Y-axis Arranged at intervals of 2 blocks in each direction. Thereby, in the high congestion area, a local area is set which is at least as large as the upper body of the person and as large as the whole body of the person.

  FIG. 4 is a schematic diagram of local areas and motion vectors at each degree of congestion. FIGS. 4A to 4C show local areas and motion vectors in low congestion areas, medium congestion areas, and high congestion areas, respectively. ing. 4 shows an example in which the local area is set based on the unit block described above. In the low congestion area shown in FIG. 4A, each square in the captured image 70 is a unit block. The unit block is a local region 72a. In the middle crowded area shown in FIG. 4B, each square in the captured image 70 is an integrated block including two unit blocks arranged in the Y-axis direction, and the integrated block is a local area 72b. In the highly congested area shown in FIG. 4C, each square in the photographed image 70 is an integrated block composed of four unit blocks arranged two by two in the X-axis direction and the Y-axis direction, and the integrated block is a local area. 72c.

  As described above, the area dividing unit 51 divides each of the divided areas obtained by dividing the captured image according to the density into a plurality of local areas according to the division criterion determined according to the density. At that time, the area dividing means 51 converts each of the divided areas into local areas having a size based on the size of the moving object, and the larger the density, the larger the predetermined area. To divide.

  The motion vector calculating unit 52 calculates a motion vector in each local region set by the region dividing unit 51 and outputs the calculated motion vector to the gaze action detecting unit 53.

  In low-congestion areas where it is known that people are not approaching by density estimation, motion vectors for small local areas that are about one-eighth of humans are calculated. It can be expected that a motion vector representing a simple motion is calculated with high accuracy without confusion with the local area between others.

  On the other hand, in medium and high congestion areas where it is known that humans are approaching each other by density estimation, the movement of a large local area of about one-quarter to one-half of a person A vector is calculated. By using a larger local area as a calculation unit, it is possible to calculate a motion vector that represents the movement of a group of multiple people that can be included in the local area. Therefore, even if the local area is enlarged, the accuracy of the motion vector is not easily lowered. Therefore, a motion vector can be calculated with high accuracy regardless of the degree of congestion.

  At this time, preferably, the motion vector is calculated from the motion of the image during a set period (analysis time interval) as the estimated density is higher. That is, the motion vector calculation means 52 switches the motion vector analysis time interval in the local area depending on whether each local area belongs to a low congestion area, a medium congestion area, or a high congestion area. For example, the motion vector calculation means 52 sets the analysis time interval to one time interval (one frame interval) in the low congestion region, two time intervals (two frame interval) in the medium congestion region, and four times in the high congestion region. Motion vectors are calculated as intervals (4 frame intervals).

  That is, the motion vector calculation unit 52 reads the current time captured image and the previous one captured image from the time-series image storage unit 40, and the local region (attention local region) belonging to the low congestion region in the current time captured image. ) Set a predetermined search range for each, and detect corresponding local regions that are located within the search range of each local region of interest and have the most similar features from among the local regions set in the captured image one time ago Then, a vector having the centroid of the corresponding local region as the start point and the centroid of the local region of interest as the end point is calculated as the motion vector at the current time in the low congestion region.

  Similarly, the motion vector calculation unit 52 reads the current time captured image and the current image captured two hours ago from the time-series image storage unit 40, and each of the target local regions belonging to the middle crowded region in the current time captured image. A predetermined search range is set, and a corresponding local region that is located within the search range of each local region of interest and that has the most similar features is detected from the local regions set in the captured image two hours ago. A vector connecting the centroids of the local area and the local area of interest is calculated as a motion vector at the current time in the middle congestion area.

  Also, the motion vector calculation means 52 reads the captured image at the current time and the captured image at 4 hours before from the time-series image storage means 40, and determines the predetermined local area belonging to the highly congested area in the captured image at the current time. A corresponding local region that is located within the search range of each local region of interest and that has the most similar features is detected from among the local regions set in the captured image four times before, and the corresponding local region is detected. A vector connecting the centroids of the region and the local region of interest is calculated as a motion vector at the current time in the highly congested region.

  Here, the feature amount can be, for example, an average pixel value (average color or average density). In addition, the search range can be set to an area in which a moving object to be subjected to motion analysis can move. For example, for each local area of interest, a circle with a predetermined radius centered on the center of gravity of the local area of interest can be set as a search range, and the radius is a distance that a person can run and move during, for example, one time Can be predetermined. Here, the increase in the degree of congestion has the effect of reducing the speed at which a person can move. Considering the effect, the analysis time interval is set longer as the congestion degree is higher. On the other hand, due to this effect, the movable range of the moving object in the medium congestion level or high congestion area where the analysis time interval is set longer than the low congestion level does not change in accordance with the analysis time interval. From this viewpoint, as described above, the size of the search range in each class of the congestion degree can be made common.

  The motion vector will be described with reference to FIG. 4A to 4C show captured images 70 at a plurality of times T with respect to a low congestion area, a middle congestion area, and a high congestion area, respectively. In the calculation of the motion vector in the low congestion area shown in FIG. 4A, one hour before (T = t) as the start point of the motion vector 74a of the local area of interest of low congestion in the captured image 70 at the current time (T = t). The corresponding local region is searched for in the captured image 70 of (-1). In addition, in the motion vector calculation in the middle congestion area shown in FIG. 4B, two hours before (T = t) as the start point of the motion vector 74b of the attention local area of the middle congestion degree of the captured image 70 at the current time (T = t). The corresponding local area is searched for in the captured image 70 at t-2), and attention is paid to the degree of congestion in the captured image 70 at the current time (T = t) in the motion vector calculation in the highly congested area shown in FIG. The corresponding local area is searched for in the captured image 70 four times before (T = t−4) as the start point of the local area motion vector 74c.

  The detection criterion storage means 42 stores a predetermined detection criterion for detecting a gaze action requiring attention. This detection criterion is stored for each degree of congestion, and each detection criterion is used for detecting a gaze action requiring attention based on the motion distribution calculated in the corresponding congestion degree region.

  The gaze action detecting means 53 receives motion vectors of a plurality of local areas from the motion vector calculation means 52, and detects the gaze action due to the moving object by analyzing the movement of the moving object in the imaging space from the motion vectors. Then, the detected gaze action information (gaze information) is output to the gaze information output means 31.

  The gaze-behavior detection means 53 calculates the motion distribution by summing up the motion vectors calculated in the area of the congestion degree for each congestion degree, and also detects the detection reference corresponding to the congestion degree from the detection reference storage means 42. Is read out and the motion distribution is compared with the detection criterion to determine whether or not the behavior requiring attention is occurring in the area of the congestion level. For example, the gaze-behavior detection means 53 calculates the motion direction frequency distribution and / or the speed frequency distribution by summing up the motion vectors for each degree of congestion, and stores the detection criterion stored in association with the congestion degree. To detect gaze behavior.

  Here, the gaze-behavior detection means 53, for example, determines the similarity between the gaze pattern and the motion distribution when the associated detection criterion is a gaze pattern and a threshold value that are features of the gaze behavior. When it is calculated and the similarity is equal to or greater than the threshold value, it is determined that the gaze action is occurring. Further, the gaze behavior detecting means 53 calculates the difference between the normal pattern and the motion distribution when the associated detection criterion is a normal pattern and a threshold value that are the feature amount of the normal action. It is determined that the behavior requiring attention is occurring when is greater than or equal to the threshold value.

  When it is determined that the gaze action is required, the gaze-behavior detection unit 53 superimposes the region in which the motion distribution satisfying the detection criterion is calculated, and the event name corresponding to the met detection criterion is superimposed. The image is generated as the gaze information, and the generated gaze information is output to the gaze information output means 31.

  The gaze information output means 31 sequentially outputs the gaze information input from the gaze behavior detection means 53 to the display unit 6, and the display unit 6 includes information included in the gaze information input from the gaze information output means 31. Is displayed. For example, the attention required information is transmitted / received via the Internet and displayed on the display unit 6. The monitoring person determines whether or not the action requiring attention is necessary by visually checking the displayed information, and takes measures such as dispatching a handling person when it is determined that the action is necessary.

  Next, the operation of the image monitoring apparatus 1 will be described. FIG. 5 is a schematic process flow diagram of the monitoring operation in the image monitoring apparatus 1.

  The imaging unit 2 images the monitoring space and sequentially inputs the captured images to the image processing unit 5. The image processing unit 5 operates as the image acquisition unit 30, acquires a captured image from the imaging unit 2 (step S <b> 1), and inputs the acquired image to the storage unit 4. The storage unit 4 functions as the time-series image storage means 40, and stores and accumulates the input photographed image (step S2).

  Since the calculation of the motion vector used for detecting the gaze action requires the photographing of a plurality of predetermined frames, until the predetermined number of photographed images are accumulated in the time-series image storage means 40 (in step S3, “ In the case of “NO”), the image processing unit 5 repeats steps S1 and S2. In the present embodiment, the number of frames is five.

  When a predetermined number of frames of captured images are accumulated in the time-series image storage means 40 (in the case of “YES” in step S3), the image processing unit 5 operates as the density estimation means 50, and the density estimation means 50 is a captured image. An estimation extraction window is set at the position of each pixel, and the estimated density of the moving object at the pixel is calculated based on the estimation feature amount extracted from the captured image in each estimation extraction window (step S4).

  When the density estimation unit 50 obtains the estimated density distribution in the captured image, the image processing unit 5 operates as the region dividing unit 51, and divides the captured image into regions for each degree of congestion (step S5). As a result, the captured image has a low congestion area that is a pixel group with an estimated density of “background” or “low density”, a medium congestion area that is a pixel group with an estimated density of “medium density”, and an estimated density of “high density”. "Is a highly congested region that is a pixel group.

  The image processing unit 5 sequentially sets a region for each degree of congestion as a processing region (step S6), and performs a gazing action detection process (step S7). Steps S <b> 6 and S <b> 7 are repeated until the gaze-behavior detection processing is completed for all regions of the low congestion region, medium congestion region, and high congestion region (in the case of “NO” in step S <b> 8). When all the regions are completed (in the case of “YES” in step S8), when the gaze action is detected (in the case of “YES” in step S9), the gaze information is required for the gaze information output means 31. In step S10, the process returns to step S1. On the other hand, if no gaze action is detected (in the case of “NO” in step S9), step S10 is omitted. When returning to step S1, the image processing unit 5 causes the storage unit 4 to store the photographed image at the current time and the local area information.

  FIG. 6 is a schematic flowchart of the gaze action detection process S7 requiring attention. In the process shown in FIG. 6, the captured image is divided into a grid pattern, and unit blocks used for setting local areas for the low, medium, and high congestion areas are set (step S100).

  When the set processing area is a low congestion area (in the case of “YES” in step S102), the area dividing means 51 sets each unit block as a local area (step S103). Then, the motion vector calculation means 52 reads the captured image one time ago from the storage unit 4 and information on the local area set by the processing one hour ago, and the local area set in the low congestion area in the captured image at the current time. For each region, a motion vector is calculated with an analysis time interval of one time interval (one frame interval) (step S104), and the motion vectors calculated for the low-congestion region are aggregated to calculate the frequency distribution in the moving direction and the speed frequency distribution. Is calculated (step S105).

When the motion vector calculation means 52 calculates the motion distribution for the low congestion area, the gaze behavior detecting means 53 checks whether or not the motion distribution satisfies the detection criterion at the time of low congestion (step S106). Specifically, the gazing behavior detection means 53 reads the detection standard at the time of low congestion from the detection standard storage means 42. That is, the gazing behavior detection means 53 reads the normal pattern of the motion distribution and the threshold value _TL11 . Next, it is determined whether or not each distribution obtained in step S105 satisfies the detection criterion for the gaze action.

For example, the gaze behavior detecting means 53 calculates the degree of difference by comparing each frequency distribution obtained as the motion distribution in step S105 with the corresponding normal pattern. As the degree of difference, the area difference D _L11 between the motion distribution and its normal pattern can be calculated. Then, the area difference D _L11 is compared with the threshold value T _L11, and when D _L11 ≧ T _L11, it is determined that the detection criterion is satisfied (in the case of “YES” in step S106), and when D _L11 <T _L11 Determines that the detection criterion is not satisfied (in the case of “NO” in step S106).

When D _L11 ≧ T _L11 , there is a local area that suddenly accelerates or decelerates in a low-congestion area, and the movement of the hand before snatching, the escape running movement after snatching, or the hand movement before snatching There is a possibility that an approaching action or the like has occurred. As described above, when a distribution satisfying the detection criterion for the gaze action required is detected (in the case of “YES” in step S106), the gaze action detection unit 53 generates and records the gaze information for the distribution. (Step S107), the process proceeds to Step S8 of FIG. For example, the gaze-behavior detection means 53 generates the event name “possibility of snatching” corresponding to the detection criterion satisfied by the distribution and the coordinates of the local area that is the extraction target area as the gaze information. On the other hand, when the distribution does not satisfy the detection criterion (in the case of “NO” in step S106), step S107 is omitted.

  When the processing area set for the gaze-behavior detection process S7 is a medium congestion area ("NO" in step S102 and "YES" in step S108), the area dividing means 51 is as described above. An integrated block composed of two unit blocks arranged in the Y-axis direction is set as a local region (step S109). Then, the motion vector calculation means 52 reads the captured image two hours before and the information on the local area set by the processing two hours before from the storage unit 4, and the local area set as the medium congestion area in the captured image at the current time. For each region, a motion vector is calculated with an analysis time interval of 2 time intervals (2 frame intervals) (step S110), and the motion vectors calculated for the middle crowded region are aggregated to calculate a motion distribution (step S111). For example, the motion vector calculation unit 52 calculates a frequency distribution in the movement direction as the motion distribution of the middle congestion area.

When the motion vector calculation means 52 calculates the motion distribution for the middle congestion area, the gaze required behavior detection means 53 checks whether or not the motion distribution satisfies the detection criterion for the middle congestion (step S112). Specifically, the watch-at-behavior detection means 53 reads out the detection reference at the time of medium congestion from the detection reference storage means 42. That is, the gaze behavior detecting means 53 reads the frequency distributions of a plurality of moving directions having a frequency that the moving direction is biased in a specific direction and the threshold value _TM11 . Further, the frequency distribution in the moving direction with no deviation in the moving direction and the threshold value _TM12 are read out. These frequency distributions correspond to the watch pattern required.

The gaze behavior detecting unit 53 calculates the similarity by comparing the frequency distribution in the movement direction calculated in step S111 with the gaze pattern. For example, as the similarity, the overlapping area S _M11 between the frequency distribution in the moving direction calculated in step S111 and a plurality of patterns having a biased frequency that is a watched pattern thereof, and the overlapping area S with a pattern having a biased frequency. _M12 is calculated.

The gazing behavior detecting means 53 compares the overlapping area S _M11 with the threshold T _M11 . If S _M11 ≧ T _M11 , each person in the person group moves toward a specific position, and the movement directions match, so that the person group may generate a matrix. .

In addition, the gazing behavior detecting means 53 compares the overlapping area S _M12 with the threshold value T _M12 . If S _M12 ≧ T _M12 , each person in the person group moves toward a specific position, and the movement direction is uniform, and therefore the person group takes an action of enclosing toward the specific position. It shows the possibility of troubles such as sudden illness and fighting.

  When a distribution satisfying the detection criteria for such a watch-at-behavior is detected (in the case of “YES” at step S112), the watch-at-behavior detection means 53 generates and records the watch-at-watch information for the distribution ( In step S107), the process proceeds to step S8 in FIG. For example, the gaze-behavior detection means 53 generates an event name such as “enclosed” corresponding to the detection criterion satisfied by the distribution and the coordinates of the local area that is the extraction target area as the gaze information. On the other hand, when the distribution does not satisfy the detection criterion (“NO” in step S112), step S107 is omitted.

  When the processing area set for the gaze behavior detection process S7 is a highly congested area (in the case of “NO” in steps S102 and S108), the area dividing means 51 is in the X-axis direction and the Y-axis as described above. An integrated block composed of four unit blocks having a 2 × 2 array in the direction is set as a local region (step S113). Then, the motion vector calculation means 52 reads the captured image 4 hours before and the local area information set by the processing 4 hours before from the storage unit 4, and the local area set as the highly congested area in the captured image at the current time. For each region, motion vectors are calculated with an analysis time interval of 4 time intervals (4 frame intervals) (step S114), and the motion vectors calculated for highly congested regions are aggregated to calculate a motion distribution (step S115). For example, the motion vector calculation means 52 calculates an average vector (relative motion vector) of a difference vector between the motion vector of each of the plurality of local regions and the motion vector of the local region around the local region, and the plurality of local regions A motion distribution in which each center of gravity is associated with a relative motion vector is calculated. A local region adjacent to the local region of interest may be a local region around the local region of interest, or a local region that includes a centroid within a circle with a predetermined radius from the centroid of the local region of interest It may be a local region.

When the motion vector calculation means 52 calculates the motion distribution for the highly congested area, the gaze behavior detecting means 53 checks whether or not the motion distribution satisfies the detection criterion at the time of high congestion (step S116). Specifically, the gazing behavior detecting unit 53 reads out the detection criterion at the time of high congestion from the detection criterion storage unit 42. That is, the gazing behavior detection means 53 reads the normal pattern of the motion distribution in the highly congested region, the threshold value _TH11 , and the threshold value _TH12 .

The gaze behavior detecting unit 53 calculates the difference by comparing the distribution calculated in step S115 with the normal pattern. For example, the gazing behavior detecting means 53 compares the magnitude of the difference vector between the relative motion vectors in the local area corresponding to the motion distribution calculated in step S115 and the normal pattern with the threshold _TH11, and calculates the difference vector. The total area _DH12 of the local region whose size is _{equal to} or greater than the threshold value _TH11 is calculated. Note that a local region having a centroid closest to the centroid of the local region of interest may be a local region corresponding to the local region of interest.

Main watching action detection unit 53, the total area _{D H12} is compared with a threshold value _{T H12,} determined to be a _{D H12} ≧ _{T H12} satisfy the detection criteria (if at step S116 is _{"YES"), D H12} If < _{TH 12,} it is determined that the detection criterion is not satisfied (in the case of “NO” in step S116).

When D _H12 ≧ T _H12 , there is a movement different from many other movements in the high-congestion area, such as backward running or staying in a group movement of people, before the runaway movement after snatching or snatching There is a possibility that an approaching action or the like has occurred.

When a distribution satisfying the detection criteria for such a watch-at-behavior is detected (“YES” at step S116), the watch-at-behavior detection means 53 generates and records the watch-at-watch information for the distribution ( In step S107), the process proceeds to step S8 in FIG. For example, the gaze-behavior detecting unit 53 requires the event name such as “possibility of snatching” corresponding to the detection criterion satisfied by the distribution, and the size of the difference vector in the highly congested area that is the extraction target area The center-of-gravity coordinates of the local region that is _{equal to} or higher than _TH11 are generated as gaze information. On the other hand, when the distribution does not satisfy the detection criterion (“NO” in step S116), step S107 is omitted.

  As described above, the motion vector of the moving object in the space that is likely to be crowded is calculated in detail in consideration of the movement of the moving object in the segmented area where the degree of congestion is low. In a highly divided area, it is possible to reduce the erroneous calculation caused by the confusion of the parts of the moving object and to calculate with high accuracy. Therefore, it is possible to accurately analyze the movement of a moving object from a captured image obtained by capturing a space where congestion can occur.

[Second Embodiment]
The image monitoring apparatus 1 according to the second embodiment of the present invention is basically the same as the first embodiment except that the processing of the area dividing unit 51 is different from the first embodiment described above. Hereinafter, the second embodiment will be described with a focus on the differences from the first embodiment, with the same reference numerals given to the same configurations as those of the first embodiment, and the above description. .

  In the first embodiment, the area dividing unit 51 divides the divided area into local areas having a predetermined size according to the density. In the second embodiment, the area dividing unit 51 captures the captured content for each captured image. The local area corresponding to is dynamically set. Specifically, the region dividing means 51 is a division criterion for dividing the pixel value (color or density) and the pixel position into local regions made up of pixels similar to each other, and the higher the class defined for the density, the larger the local region Each of the divided areas is divided into local areas in accordance with division criteria that are easily determined.

  The area dividing means 51 of the second embodiment uses the degree of congestion defined as in the first embodiment as the class relating to the density. Therefore, the partitioned areas in the second embodiment are generated in the same manner as in the first embodiment, and three partitioned areas of a low congestion area, a medium congestion area, and a high congestion area can be set in the captured image.

  FIG. 7 is a schematic diagram showing an example of a photographed image composed of areas of respective congestion degrees and an example of a local area corresponding thereto. Specifically, in the upper part of FIG. 7, a captured image including only a low congestion area and a local area corresponding thereto are shown. Similarly, a middle image and a lower image in FIG. 7 show a captured image including only a medium congestion area and a high congestion area, and a local area corresponding thereto.

  For example, the area dividing means 51 divides the captured image of each divided area into a plurality of clusters by applying an SLIC (Simple Linear Iterative Clustering) method to each divided area. Each of the plurality of clusters is a local region.

  In the SLIC method, the number of divisions is determined prior to division, the same number of cluster centers as the determined number of divisions are set as initial values on the target image, and the target image is divided into the same number of clusters as the number of divisions. Is done.

  Corresponding to the characteristics of the SLIC method, the region dividing means 51 determines a smaller number of divisions for each divided region as the density of the divided region is higher, so that a larger local region is more likely to be generated. . That is, each of the segmented areas is divided into local areas according to a division criterion that sets a small number of local areas per unit area.

  For example, the area dividing unit 51 performs the following steps A1 to A6 for each divided area to divide the divided area into clusters.

  (Step A1) The segmented region is divided into substantially equal grids with a divided area corresponding to the degree of congestion of the segmented region, and the center of each block, which is a rectangular region generated by the segmentation, is set as the initial value of the cluster center. . With this processing, the number of divisions (> 1) equal to the number of cluster centers (number of blocks) is determined as the division criterion.

Specifically, the low congestion area, the area S _L and predetermined divided area U _L quotient S _{L /} U _L the integer value obtained by rounding off the region of the extent of the size 1/8 people of the area defined as the division number k _L, the area is each area is divided into blocks of substantially U _L, the center of each block and the initial value of the cluster centers in the area.

Similarly, for a middle crowded area, an integer value obtained by rounding off a quotient S _M / U _M of a predetermined divided area U _M that is approximately ¼ of the area S _M of the area and a person is divided into the areas. defined as the number k _M, the area is each area is divided into blocks of substantially U _M, the center of each block and the initial value of the cluster centers in the area.

As for the high congestion area, the division number of the region area S _H and human predetermined splitting area U _H quotient S _{H /} U _H that region an integer value obtained by rounding off the about the size of half of the k _H is defined, the region is divided into blocks each having an area of approximately U _H , and the center of each block is set as the initial value of the cluster center in the region.

  In calculating the number of divisions, instead of rounding to an integer value, rounding or rounding may be used to make an integer value, and either one may be determined in advance.

  Further, when the number of divisions is less than 2, control is performed so as not to perform division into local regions and calculation of motion vectors.

  (Step A2) An evaluation value is calculated for a combination of each pixel and each cluster center in the partitioned area. As the evaluation value, for example, a weighted sum of the reciprocal of the distance from the pixel to the cluster center and the luminance similarity between the pixel and the cluster center can be used. That is, the evaluation value can be defined by an integrated similarity obtained by integrating the similarity of pixel positions and the similarity of pixel values.

  (Step A3) Each pixel in the segmented region is assigned to the cluster center having the highest evaluation value with the pixel.

  (Step A4) The sum of evaluation values of all pixels is obtained.

  (Step A5) Each cluster center is updated to weighted average coordinates obtained by weighting and averaging the coordinates of the pixels belonging to the cluster center with the evaluation values of the pixels.

  (Step A6) Steps A2 to A5 are repeated using the cluster center update value, and the absolute value of the difference between the sum obtained in step A4 and the sum total obtained in the previous step A4 becomes less than a predetermined value, and the cluster update process is performed. When it is determined that has converged, or when the number of repetitions reaches the specified number, the process is terminated, and the most recently obtained cluster is determined as a local region.

  The upper, middle, and lower diagrams in FIG. 7 are a captured image of a low-congested area, a captured image of a medium-congested area, and a captured image of a highly-congested area, respectively. Is illustrated as being divided into a local region having a size of about one-fourth of the above and a local region having a size of about one-half of a person.

  The process of generating a local area from the above-described segmented area by the area dividing unit 51 in the second embodiment is performed in the gaze required action detection process S7 in the operation shown in FIG. 5 as in the first embodiment. That is, the monitoring operation in the image monitoring apparatus 1 of the second embodiment is performed in the same processing flow as in FIG. 5 described for the first embodiment, but in relation to the difference in processing of the area dividing unit 51, There is a difference from the first embodiment in the details of the gaze action detection process S7 requiring attention.

  FIG. 8 is a schematic flowchart of the gaze-behavior detection processing S7 in the second embodiment.

When the processing area set in step S6 in FIG. 5 is a low-congestion area (in the case of “YES” in step S200), the area dividing means 51 is divided into a size about 1/8 of a person. performs generation processing of the above-described local region using the area U _L, is divided into the number of local regions where the average area of low congestion area 1/8 persons (step S201).

  For the local region, the motion vector calculation means 52 calculates a motion vector and a motion distribution in the same manner as steps S104 and S105 in FIG. 6 of the first embodiment (steps S202 and S203). Then, in the same manner as steps S106 and S107 in FIG. 6, the gazing behavior detecting unit 53 determines whether or not the detection criterion at the time of low congestion is satisfied and records the gazing information (see FIG. 6). Steps S204 and S205).

When the processing area set in step S6 in FIG. 5 is a medium congestion area (in the case of “NO” in step S200 and “YES” in step S206), the area dividing means 51 is ¼ of the person. The above-described local region generation process using the divided area U _M set to a moderate size is performed, and the middle crowded region is divided into a number of local regions whose average area is ¼ (step S207).

  For the local region, the motion vector calculation unit 52 calculates a motion vector and a motion distribution in the same manner as steps S110 and S111 in FIG. 6 (steps S208 and S209). In the same manner as steps S112 and S107 in FIG. 6, it is determined whether or not the detection criterion at the time of medium congestion is satisfied, and the gazing information is recorded (steps S210 and S205).

In addition, when the processing area set in step S6 in FIG. 5 is a highly congested area (“NO” in step S200 and “NO” in step S206), the area dividing unit 51 determines that the person 1 / performs generation processing of the above-described local region using division area U _H as defined in 2 about the size, the average area of high congestion area is divided into the number of local regions comprising 1/2 persons (step S211 ).

  For the local region, the motion vector calculation unit 52 calculates a motion vector and a motion distribution in the same manner as steps S114 and S115 in FIG. 6 (steps S212 and S213). In the same manner as steps S116 and S107 in FIG. 6, it is determined whether or not the detection criterion at the time of high congestion is satisfied, and the gazing information is recorded (steps S214 and S205).

  In the local region generation processing of the present embodiment described above, each local region is likely to be increased if the number of divisions is reduced, and each local region is likely to be reduced if the number of divisions is increased.

  As described above, even if the divided area is divided by a smaller number of divisions as the density of the moving object in the area is higher, at least the local area related to the moving object is larger as the density is higher in the area divided according to the density. It can be expected that the lower the density, the smaller the division.

  For this reason, it is possible to accurately calculate a motion vector of a moving object in the space from a captured image obtained by capturing a space in which congestion may occur, by reducing erroneous calculation due to confusion of the parts of the moving object. Therefore, it is possible to accurately analyze the movement of a moving object from a captured image obtained by capturing a space where congestion can occur.

  As described above, as the second embodiment, a configuration for realizing a method for controlling the size of a local region in accordance with the degree of congestion by using a large number of divisions when generating a local region from a segmented region is described using the SLIC method. did.

  Here, even when a bottom-up region dividing method using a group average method or the like is employed instead of the SLIC method, the size of the local region can be controlled by the number of divisions. In this case, the area dividing means 51 performs the following steps B1 to B5 to divide the divided area into clusters.

(Step B1) The number of divisions (> 1) is determined by dividing the area of the divided region by the divided area corresponding to the density of the divided region. Specifically, the low congestion area, the area S _L and predetermined divided area U _L quotient S _{L /} U _L the integer value obtained by rounding off the region of the extent of the size 1/8 people of the area defined as the number of divisions _{k L.} For the middle crowded area, an integer value obtained by rounding off the quotient S _M / U _M of the divided area U _M set in advance to the area S _M of the area and about ¼ of the person is the division number k _{M of the} area. It is determined. As for the high congestion area, the division number of the region area S _H and human predetermined splitting area U _H quotient S _{H /} U _H that region an integer value obtained by rounding off the about the size of half of the defined as k _H.

  (Step B2) Each pixel in the captured image is set as an initial cluster.

  (Step B3) An evaluation value is calculated for each combination of adjacent clusters. As the evaluation value, for example, a weighted sum of the reciprocal of the distance between the cluster centers and the similarity of the average luminance between the clusters can be used. That is, the evaluation value can be defined by an integrated similarity obtained by integrating the similarity of pixel positions and the similarity of pixel values. Note that the similarity of pixel values may be used as the evaluation value instead of the integrated similarity. Incidentally, in that case as well, the requirement of similarity of pixel positions is included under the condition of “adjacent clusters”.

  (Step B4) The combination of clusters having the maximum evaluation value is integrated into one cluster.

  (Step B5) Steps B3 and B4 are repeated, and if the number of clusters is equal to or less than the number of divisions determined in step B1, the process is terminated, and the most recently obtained cluster is determined as a local region. On the other hand, if the number of clusters is not less than the number of divisions determined in step B1, steps B3 and B4 are further repeated.

  As described above, two methods for controlling the size of the local region based on the number of divisions have been described. However, instead of controlling based on the number of divisions, the local region is controlled based on the height of the threshold for the evaluation value (integrated similarity) described above. Another method for controlling the size of the region can also be adopted.

  That is, the area dividing unit 51 has the integrated similarity exceeding the threshold for each of the divided areas in accordance with the division criterion that sets the threshold for the integrated similarity obtained by integrating the similarity of the pixel value and the similarity of the pixel position as the density increases. Divide into local areas consisting of pixels. That is, the division criterion sets a lower threshold for the integrated similarity that determines that the pixels are similar to each other as the degree of congestion is higher.

  In this case, the area dividing means 51 performs the following steps C1 to C4 for each divided area to divide the divided area into clusters.

  (Step C1) Each pixel in the captured image is set as an initial cluster.

  (Step C2) An evaluation value is calculated for each combination of adjacent clusters. As the evaluation value, for example, the integrated similarity described above can be used.

  (Step C3) The evaluation value calculated in Step C2 is compared with a threshold value, and a combination of clusters having an evaluation value equal to or less than the threshold value is integrated into one cluster. The threshold value is a predetermined value for each degree of congestion of the segmented area, and is a value set lower as the degree of congestion is higher.

  (Step C4) Steps C2 and C3 are repeated if there is at least one combination of clusters whose evaluation value is less than or equal to the threshold value in Step C3, and processing is performed if there is no combination of clusters whose evaluation value is less than or equal to the threshold value in Step C3. End and determine the most recently obtained cluster as the local region.

  In the second embodiment, the number of divisions and the similarity threshold are exemplified as the division criteria. However, other thresholds limit the size range of the local region (the higher the density, the wider the range, A division criterion such as a threshold that limits the number of cluster integrations (a lower density class) or a threshold that limits the number of cluster integrations (a higher density class has a higher upper limit on integration times, and a lower density class has a lower upper limit on integration times) Also, it can be determined that the higher the density, the easier the local area size becomes.

  (1) In each of the embodiments described above, an example in which the object to be detected is a person has been shown. However, the present invention is not limited to this, and the object to be detected may be a vehicle, an animal such as a cow or a sheep, or the like.

  (2) In each of the above-described embodiments and modifications thereof, the density estimator learned by the multi-class SVM method has been exemplified. However, instead of the multi-class SVM method, a decision tree type random forest method, a multi-class Adaboost Various density estimators such as a density estimator learned by (AdaBoost) method or multi-class logistic regression method can be used.

  Alternatively, a density estimator using a discriminating CNN (Convolutional Neural Network) may be used.

  (3) In each of the above embodiments and their modifications, the density classes other than the background estimated by the density estimator are set to three classes. However, the classes may be divided more finely.

  (4) In each of the above-described embodiments and modifications thereof, a density estimator that classifies into multiple classes is illustrated, but instead of this, a regression type density estimation that regresses a density value (estimated density) from a feature quantity It can also be a container. That is, a density estimator that has learned the parameters of the regression function for obtaining the estimated density from the features by ridge regression method, support vector regression method, regression tree-type random forest method or Gaussian Process Regression, etc. can do.

  Alternatively, a density estimator using a regression type CNN may be used.

  (5) In each of the above embodiments and the modifications thereof, the GLCM feature is exemplified as the feature amount learned by the density estimator and the estimation feature amount. However, instead of the GLCM feature, the local binary pattern (Local Binary Pattern (LBP) feature value, Haar-like feature value, HOG feature value, luminance pattern, and other various feature values, or a combination of GLCM features and a plurality of them You can also

  DESCRIPTION OF SYMBOLS 1 Image monitoring apparatus, 2 imaging | photography part, 3 communication part, 4 memory | storage part, 5 image processing part, 6 display part, 30 image acquisition means, 31 gaze information output means, 40 time series image storage means, 41 density estimator memory | storage Means, 42 detection reference storing means, 50 density estimating means, 51 area dividing means, 52 motion vector calculating means, 53 gaze behavior detecting means.

Claims (5)

Image acquisition means for acquiring a plurality of time-captured images of a space that can be crowded with a predetermined moving object;
The moving object photographed in an arbitrary region in the photographed image by using a density estimator that has learned the image characteristics of each of the density images obtained by photographing the space where the moving object exists at the density for each predetermined density Density estimation means for estimating the density of
Area dividing means for dividing each of the divided areas obtained by dividing the captured image into a plurality of classes set with respect to the density based on the estimated density into a plurality of local areas in accordance with a division criterion determined for each of the classes. When,
Motion vector calculating means for calculating a motion vector in each of the local regions;
Motion analysis means for analyzing the motion of the moving object in the space from the motion vectors of the plurality of local regions;
An image analysis apparatus comprising:   2. The division criterion is characterized in that an area having a predetermined size with respect to the size of the moving object is set as the local area, and the size is set to be larger as the class has a higher density. The image analysis apparatus described in 1.   In the division criterion, a region composed of pixels that are similar to each other based on pixel similarity defined by a pixel value and a pixel position is defined as the local region, and the higher the class, the larger the size of the local region. The image analysis apparatus according to claim 1, wherein the image analysis apparatus is defined.   The image analysis apparatus according to claim 3, wherein the division criterion is set such that the higher the density is, the smaller the number of the local regions per unit area is.   The image analysis apparatus according to claim 3, wherein the division criterion is such that a threshold value of the similarity that is determined to be similar to each other with respect to the pixels is set to be lower as the density is higher.

Images