WO2010050110A1 - 移動体検出方法及び移動体検出装置 - Google Patents
移動体検出方法及び移動体検出装置 Download PDFInfo
- Publication number
- WO2010050110A1 WO2010050110A1 PCT/JP2009/004523 JP2009004523W WO2010050110A1 WO 2010050110 A1 WO2010050110 A1 WO 2010050110A1 JP 2009004523 W JP2009004523 W JP 2009004523W WO 2010050110 A1 WO2010050110 A1 WO 2010050110A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- distance
- region
- movement
- moving
- moving body
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/215—Motion-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Definitions
- the present invention relates to a method for detecting a moving object in an image, and in particular, from a moving image composed of a plurality of images, based on motion information of the image, all moving objects such as a person that moves while changing its shape.
- the present invention relates to a method of detecting a moving object by dividing an area that specifies a partial area.
- a method for extracting candidate regions from the image There is a method in combination with a method of applying an object model prepared in advance to a candidate area of an extracted object (see, for example, Patent Document 1 and Non-Patent Document 1).
- Patent Document 1 a silhouette image of a target object such as a person is extracted from a plurality of images as a target object candidate area, and a model related to the target object such as a person in which the part of the target is parameterized in advance from knowledge about the target object
- a method of applying the model to the extracted silhouette image using the above is disclosed.
- a parameterized model can be applied to a moving body such as a person who moves while changing its shape, so that it is possible to detect the moving body and divide the region.
- Non-Patent Document 1 an image obtained by photographing one fixed moving body from a plurality of viewpoints is input, and Euclidean distance between pixel value data in each image and pixel value data of other images is calculated.
- a method is disclosed in which dimensional compression is performed so that images taken from similar viewpoints can be projected so as to be close to each other in a two-dimensional space.
- PCA Principal Component Analysis
- Patent Document 1 has a problem in that a moving body cannot be detected correctly, particularly in a scene where a moving body such as a person passes around the street.
- the moving object detection method represented by the above-mentioned Patent Document 1 needs to extract a target object candidate region from an image as described above. At this time, if the target object candidate area cannot be correctly extracted, it is impossible to accurately apply the model obtained by parameterizing the target object to the target object candidate area. In particular, in a crowded scene, it is difficult to correctly extract an object candidate region. For example, when moving objects of various sizes are mixed on the image, a plurality of moving objects are mistakenly extracted as one moving object, and there are moving objects to be extracted.
- Non-Patent Document 1 image data can be projected onto a low-dimensional space that is efficiently compressed by performing nonlinear processing using the distance between images as an input. Furthermore, data distributed continuously and nonlinearly can be efficiently expressed by geodetic distance conversion and dimensional compression.
- the main purpose of Non-Patent Document 1 is to visualize the similarity between images by projecting a plurality of images into a low-dimensional space, and various postures in a multi-joint object such as a person whose shape changes A method for correctly detecting a moving object while responding to a change is not disclosed.
- the present invention detects a moving body that can correctly divide an area including a moving body such as a person who moves while changing its shape without being affected by the posture or size of the moving body. It is an object to provide a method and a moving body detection apparatus.
- the present invention is a method for detecting a moving object in a moving image by performing region division that specifies all or a part of the moving object in the moving image.
- An image input step for accepting a plurality of pictures constituting the image, and detecting a motion of the image between two temporally adjacent pictures for each block comprising one or more pixels constituting the picture.
- a motion analysis step for calculating a movement trajectory by connecting motions for the plurality of pictures, and a distance for calculating a distance representing similarity between the movement trajectories for the plurality of movement trajectories calculated in the motion analysis step. Of the distances calculated in the calculation step and the distance calculation step, a distance smaller than a predetermined threshold is connected to calculate the distance in the distance calculation step.
- the discontinuity is converted into a geodetic distance, discontinuous points in the obtained geodetic distance distribution are detected, and the region segmentation is performed by making a movement locus separated by a geodetic distance smaller than the detected discontinuous point into one cluster. And an output step for outputting the result of the area division in the area division step.
- the present invention can be realized not only as the above-mentioned moving body detection method, but also as a moving body detection device having the above steps as constituent elements, a program for causing a computer to execute the above steps, a CD-ROM storing the program, etc. It can also be realized as a computer-readable recording medium.
- the above method and apparatus can accurately detect a moving body such as a person who moves while changing its shape and divide the region into regions. Furthermore, it is also possible to predict the movement of the moving object using the results of detection and area division.
- FIG. 1 is a functional block diagram showing a basic configuration of a moving object detection apparatus according to Embodiment 1 of the present invention.
- FIG. 2 is a block diagram showing a hardware configuration of the moving object detection apparatus according to the present invention.
- FIG. 3 is a flowchart showing the basic operation of the moving object detection device according to Embodiment 1 of the present invention.
- FIG. 4 is a diagram illustrating a processing example of the motion analysis unit according to Embodiment 1 of the present invention.
- FIGS. 5A to 5C are diagrams showing an example of the effect of the geodetic distance of the area dividing unit in the first embodiment of the present invention.
- FIGS. 1 is a functional block diagram showing a basic configuration of a moving object detection apparatus according to Embodiment 1 of the present invention.
- FIG. 2 is a block diagram showing a hardware configuration of the moving object detection apparatus according to the present invention.
- FIG. 3 is a flowchart showing the basic operation of the moving object detection device according to Embodi
- FIGS. 6A and 6B are diagrams showing an example of the effect of the geodetic distance of the area dividing unit in the first embodiment of the present invention.
- FIGS. 7A to 7C are diagrams showing a processing example of the area dividing unit according to the first embodiment of the present invention.
- FIG. 8 is a flowchart showing the basic operation of the moving object detection device according to the modification of the first embodiment of the present invention.
- FIGS. 9A to 9D are diagrams showing processing examples of the area dividing unit in the modification of the first embodiment of the present invention.
- FIG. 10 is a functional block diagram showing the basic configuration of the moving object detection device according to the second embodiment of the present invention.
- FIG. 11 is a flowchart showing the basic operation of the moving object detection device according to the second embodiment of the present invention.
- FIGS. 12 (a) to 12 (f) are diagrams showing processing examples of the area dividing unit according to the second embodiment of the present invention.
- FIGS. 13A to 13C are diagrams showing an example of hierarchical clustering processing of the area dividing unit according to the second embodiment of the present invention.
- FIG. 14 is a functional block diagram illustrating a configuration example of the moving object detection device according to the first modification of Embodiments 1 and 2 of the present invention.
- FIGS. 15A and 15B are diagrams showing display examples of the image display unit in the first modification of Embodiments 1 and 2 of the present invention.
- FIG. 16 is a functional block diagram illustrating a configuration example of the moving object detection device according to the second modification of Embodiments 1 and 2 of the present invention.
- FIG. 17 is a diagram showing an example of recording / transmission data in the second modification of Embodiments 1 and 2 of the present invention.
- FIG. 18 is a functional block diagram illustrating a configuration example of the moving object detection device according to the third embodiment of the present invention.
- FIG. 19 is a flowchart showing the basic operation of the moving object detection device according to Embodiment 3 of the present invention.
- FIG. 20 is a diagram showing an example of motion prediction according to Embodiment 3 of the present invention.
- FIGS. 21A to 21C are diagrams showing examples of figures that can be separated by the method according to the second embodiment of the present invention.
- One embodiment of the present invention is a method for detecting a moving body in a moving image by dividing an area that specifies all or a part of the moving body in the moving image, and includes a plurality of moving images.
- An image input step for receiving one picture, and a motion of the image between two temporally adjacent pictures for each block of one or more pixels constituting the picture, and the detected motion is
- the distance calculated in the distance calculation step is connected to the distance calculated in the distance calculation step by connecting distances smaller than a predetermined threshold.
- the movement trajectories separated by geodesic distances smaller than the discontinuous points form one cluster, and therefore, the character space is similar in terms of similarity between the movement trajectories compared to clustering using the Euclidean distance, which is a linear distance.
- Clustering is performed in consideration of general continuity. Therefore, it is reliably discriminated whether each block in the picture belongs to the same object (or part) or separate object (or part).
- the distance between the first movement locus and the second movement locus is converted into a geodetic distance.
- the distance from the first movement trajectory to the second movement trajectory is traced while following the movement trajectory that is separated by a distance smaller than the predetermined threshold among the distances calculated in the distance calculation step. It is preferable to calculate the distance of the route as a geodetic distance.
- the geodetic distance is smaller as the density in the distribution of the plurality of movement trajectories is larger.
- the above-mentioned conversion is performed after weighting so that the high-similarity movement trajectory is identical with higher accuracy by performing distance conversion using the density of the movement trajectory distribution.
- the region dividing step generates a plurality of threshold values used for the region dividing, and calculates each of the plurality of generated threshold values in the distance calculating step.
- the distance calculated in the distance calculation step is converted into the geodetic distance by connecting distances smaller than the threshold among the calculated distances, and discontinuous points in the obtained distribution of the plurality of geodetic distances are detected.
- a region dividing candidate generating step for dividing the region by making a movement locus separated by a geodetic distance smaller than the detected discontinuous point into one cluster, and generating a result of the region dividing as a region dividing candidate; Get instructions on the number of classes, and generate the region division candidates divided into the same or closest number of regions as the number of earned classes Selected from a plurality of regions divided candidates generated in step may comprise a region division candidate selection step of outputting the selected region division candidates as a result of the area division.
- a plurality of values between a maximum value and a minimum value in the plurality of distances calculated in the distance calculation step are generated as the threshold value.
- the first discontinuous point is detected when the thresholds are arranged in order from a larger value to a smaller value for the plurality of distances calculated in the distance calculation step.
- a plurality of values smaller than the detected discontinuous points are generated as the plurality of threshold values, and by setting the threshold values effectively, the shape moves especially while changing its shape. It is possible to detect a moving object such as a person reliably and at a higher speed, and at the same time, correctly perform area division.
- the region division candidate generation step discontinuous points are detected for a plurality of distances calculated in the distance calculation step, and the region division is hierarchically performed based on a threshold value.
- a threshold value e.g., a threshold value for a subject.
- fine moving body extraction based on the detailed movement of the subject.
- a discontinuous point is detected from a large threshold value, and for the divided clusters, Each of them is configured to detect discontinuous points using a threshold value and to divide the region in a hierarchical manner.
- the threshold value is generated effectively, and by setting the threshold value effectively, it is possible to detect the moving object such as a person moving while changing its shape reliably and faster. At the same time, it is possible to correctly perform area division.
- a distance that is the Nth smallest distance from the movement trajectory is specified and specified.
- a plurality of values selected in descending order with respect to a plurality of distances are generated as the plurality of threshold values, and by this setting the threshold values are effectively set, the shape moves particularly while changing its shape. It is possible to detect a moving object such as a person reliably and at a higher speed, and at the same time, correctly perform area division.
- a predetermined number of movement trajectories are selected for each of the plurality of movement trajectories calculated in the motion analysis step, and the selected one is not selected.
- each of the plurality of distances is converted into a geodetic distance, whereby the selected distance and the unselected distance As compared with the linear distance, the similarity / dissimilarity between the movement trajectories is emphasized, and it is possible to correctly express the motion of an object connected by a joint like a person. Become.
- a movement trajectory whose distance is equal to or less than a predetermined threshold is selected and not selected.
- each of the plurality of distances is converted into a geodetic distance, whereby the selected distance and the unselected distance Since the relationship is non-linear, the similarity / dissimilarity between the movement trajectories is emphasized compared to the linear distance, and it is possible to correctly express the motion of an object connected by a joint like a person. .
- a two-dimensional motion vector or affine parameter indicating the motion is calculated as the motion detection, whereby a motion vector or affine parameter is used.
- the distance calculating step in addition to the similarity between the movement trajectories of the blocks, the distance between the blocks in the picture and a straight line connecting the blocks are used as the distance calculation. It is configured to calculate at least one of the angles indicating the inclination of the moving object, so that it is possible to efficiently capture the movement of a moving object whose shape changes with a rotational motion about a joint like a person. It becomes possible.
- the output step includes a display step of displaying the result of the region division obtained in the region division step so as to be superimposed on the picture received in the image input step.
- a display step of displaying the result of the region division obtained in the region division step so as to be superimposed on the picture received in the image input step.
- a moving image including two or more moving objects is received, and in the region dividing step, the region dividing is performed on the two or more moving objects by 2
- the above-described moving body is configured to be detected, so that it is possible to correctly detect a plurality of moving objects even for an image including a plurality of moving bodies that move while changing their shapes.
- the moving object detection method further calculates a movement trajectory representing the area from the movement trajectory of the block constituting the area specified in the area dividing step, and calculates the calculated representative movement. It is configured to include a motion prediction step for predicting the movement of the moving body by predicting that the area moves according to the trajectory.
- the moving body is used by using a trajectory that represents the movement trajectory of a plurality of blocks. By predicting the motion, it is possible to perform motion prediction with high noise resistance.
- the output step specifies a region in the picture accepted in the image input step based on a result of region division in the region division step, and a corresponding region for each of the identified regions
- the result of the division is configured to include a recording / transmission step for recording or transmitting the result to the storage means, and thereby, by separately holding the detected moving body image based on the divided areas, respectively.
- FIG. 1 is a functional block diagram showing the configuration of the moving object detection device 100 according to the first embodiment.
- the moving body detection apparatus 100 includes an image input unit 101, a motion analysis unit 102, a distance calculation unit 103, a region division unit 104, and an output unit 105. And this moving body detection apparatus 100 detects the moving body in a moving image by carrying out the area division
- the image input unit 101 is a processing unit that receives input of a plurality of temporally different images (pictures) constituting a moving image, and is, for example, a video camera or a communication interface connected to the video camera.
- the motion analysis unit 102 detects the motion of the image between two temporally adjacent pictures for each block composed of one or more pixels constituting the picture received by the image input unit 101, and detects the detected motion Is a processing unit that calculates a movement trajectory by connecting a plurality of pictures.
- the distance calculation unit 103 uses the movement trajectory of the block i calculated by the motion analysis unit 102 and the movement trajectory of blocks other than i to capture the change in the shape of the moving object, and calculates the similarity of the block motion. It is a processing unit that calculates a distance to be expressed. For example, when the movement trajectory of N blocks is used, the calculated distance is an N ⁇ N distance matrix.
- the distance between the blocks changes depending on the movement, in particular, an object such as a person that moves while changing its shape like a joint object. It is possible to express movement as a distance matrix.
- the movement locus of block i is referred to as movement locus i.
- the “distance” in this specification includes not only the distance between two points in a two-dimensional space but also the arithmetic distance between multidimensional data, and a single value or a set of a plurality of values. (Distance matrix).
- the region dividing unit 104 is a processing unit that performs region division by clustering a plurality of movement trajectories calculated by the motion analysis unit 102 using a threshold value used for region division. Specifically, the area dividing unit 104 applies geodetic distance conversion to the distance matrix calculated by the distance calculating unit 103 using a threshold value regarding data continuity, that is, calculated by the distance calculating unit 103. By connecting the distances that are smaller than the threshold among multiple distances, each of the multiple distances is converted into a geodetic distance, and discontinuous points are detected in the distribution of the distance between the trajectories after the geodetic distance conversion at each threshold. Then, by dividing the movement trajectories continuously distributed so that the movement trajectories separated by the geodesic distance smaller than the detected discontinuous points become one cluster, the region division for the threshold value is performed.
- the area dividing unit 104 determines whether the distance between the first movement locus and the second movement locus in the conversion from the distance (distance matrix) calculated by the distance calculation unit 103 to the geodetic distance (geodetic distance conversion). In the case of converting the distance into a geodetic distance, the second movement from the first movement trajectory while following the movement trajectory separated by a distance smaller than a predetermined threshold among the distances calculated by the distance calculation unit 103. The distance of the route to the locus is calculated as the geodetic distance.
- the output unit 105 the detection result of the moving object in the moving image performed by the region dividing unit 104, the memory or recording medium writing processing unit that outputs the region dividing result of the image, or the output interface that outputs to the display device Etc.
- each area obtained as a result of area division corresponds to each moving body
- detection of each moving body and area division for dividing a plurality of moving body areas in an image are particularly I do not distinguish. That is, the process of “detecting a moving object” corresponds to the process of “area division”.
- each component (image input unit 101, motion analysis unit 102, distance calculation unit 103, region division unit 104, and output unit 105) constituting the moving body detection apparatus 100 is a camera, as shown in FIG.
- It may be realized by software, or may be realized by hardware such as an electronic circuit.
- the constituent elements other than the input / output unit such as the display device may be realized by a program and data (that is, software) executed by a computer, an electronic circuit, It may be realized by hardware such as a memory and a recording medium, or may be realized by mixing them.
- a program and data that is, software
- step S101 the image input unit 101 receives a plurality of pictures.
- step S102 the motion analysis unit 102 calculates a block motion from at least two pictures.
- pixel motion is calculated as an example of calculating block motion.
- pixel unit processing will be described as an example of block unit processing.
- Non-Patent Document 2 or Non-Patent Document 3 and Non-Patent Document 4 can be used to calculate a motion vector by optical flow calculation.
- the motion analysis unit 102 uses the pictures input at time t and time t + 1 to use the motion vector (u i t , v i t ) of pixel i. Is estimated.
- the frames do not necessarily have to be continuous.
- the motion of the pixels may be obtained using pictures input at time t and time t + n.
- n is an integer of 1 or more.
- an affine parameter may be estimated as the pixel motion instead of the two-dimensional motion vector. At this time, motion information may be obtained for all pixels.
- the image may be divided into grids to obtain motion information only for pixels on the grid at regular intervals, or as described above, the image may be divided into blocks and each block may be obtained. You may also ask for motion information. Furthermore, when calculating a motion vector using Non-Patent Document 2, since the reliability can be calculated, only pixels having motion information with high reliability may be used. Moreover, when calculating a motion vector using the nonpatent literature 3, an occlusion can be estimated. For this reason, only the motion information of pixels that are not shielded may be used.
- Non-Patent Document 4 is known to be capable of high-speed processing, and may be used when high-speed processing is required.
- Non-Patent Document 5 can be used as a method for calculating a motion vector assuming affine deformation.
- the above method estimates the affine parameter A i t corresponding to the motion near the pixel i of the picture input at time t and time t + 1.
- pixel positions x i t and x i t + 1 on the picture at time t and time t + 1 have the following relationship.
- the above method can estimate the motion of the pixel i with higher accuracy than the case of using a method of calculating a motion vector assuming a translational movement, particularly for an object that performs a rotational motion.
- step S103 the motion analysis unit 102 uses the motion information calculated in step S102 to calculate the movement trajectory i from the pixel motion for a plurality of temporally different pictures.
- the movement locus of the pixel i is referred to as a movement locus i.
- the motion of the pixel i is tracked using the motion information 302 calculated in step S102 from the pixel i303 of the input image 301 at time t.
- the movement locus i is calculated as follows using the pixel position (x i t , y i t ) on the picture at the time t when the pixel i passes.
- T is the number of pictures used to calculate the movement trajectory.
- step S104 the distance calculation unit 103 calculates a distance matrix including the similarity of the motion of the pixels using the movement trajectory i calculated by Equation 2 above.
- the linear distance f (i, j) between the movement trajectory i and the movement trajectory j can be calculated as shown in Equation 3 below.
- w is a weighting factor and is a parameter set by the designer.
- the above expression 3 may be modified as the following expression 4.
- ptn ij and mtn ij are shown in the following formulas 5 and 6, respectively.
- Equation 3 in addition to the time average value of the distance between moving tracks shown in Equation 5, the temporal variation in the distance between moving tracks shown in Equation 6 is changed to the linear distance.
- f i, j
- the temporal variation in the distance between the movement trajectories shown in the above expression 6 indicates the similarity of the movement of the pixels, and thus, not only the rigid body whose relationship between the distances between the pixels does not change with time, Changes in the shape of joint objects can be captured. Note that the same effect can be expected by using time-variable components as shown in the following equations 8 to 13 instead of the above equation 6.
- u t i the motion vector from time t to t + 1 (u i t, v i t), ⁇ u t i ⁇ u t i> is the inner product.
- the area dividing unit 104 executes Steps S105 (geodetic distance conversion) and S106 (area dividing). Specifically, in S105, the region dividing unit 104 uses the threshold value R for the linear distance f (i, j) calculated by the above formula 3 or the above formula 4, and uses the following formula 14 for the threshold value. In this way, non-linear processing is performed to calculate f ′ (i, j).
- the region dividing unit 104 selects R moving trajectories j in ascending order of the linear distance from the moving trajectory i when focusing on the moving trajectory i, and does not change the distance from the selected moving trajectory j.
- the distance from the unselected movement trajectory j is changed to infinity.
- the linear distances f (i, j) are selected in ascending order, but the threshold value R may be set as in the following equation.
- the region dividing unit 104 selects and selects a predetermined number of movement trajectories in ascending order of distance for each of the plurality of movement trajectories calculated by the motion analysis unit 102 as shown in the above equation 14.
- each of a plurality of distances may be converted into a geodetic distance, or when focusing on the moving trajectory i as shown in Equation 15 above
- a non-linearity is selected in which the movement trajectory j whose distance is equal to or less than a predetermined threshold is selected and the distance from the unselected movement trajectory is changed to infinity.
- each of a plurality of distances may be converted into a geodetic distance.
- the distance non-linearization is not limited to the above function, and any distance conversion may be used as long as the nonlinear transformation is performed on the distances related to the movement trajectory i and the movement trajectory j.
- linear distance f (i, j) may be weighted by multiplying the weight calculated using Expression 16 and Expression 17 as follows, and then the processing of Expression 14 or Expression 15 may be performed. Good.
- NN indicates that processing is performed for the neighboring points of the movement trajectory.
- a movement trajectory within a certain distance from each of the movement trajectories j and i, or N movements in ascending order of distance Indicates calculation using a trajectory. That is, N a and N b are the number of moving tracks within a certain distance or N. Note that z is set by the designer.
- dispersion may be used as in the following equation 17.
- f (i, j) By converting f (i, j) using the weights of the above equation 16 and the above equation 17, when the movement locus similar to the movement locus i and j is spatially dense (the movement locus having a short distance) In the case of f), f (i, j) is relatively small, and in the case of spatial sparseness (in the case of a moving trajectory with a long distance), f (i, j) j) becomes relatively large. That is, when the area dividing unit 104 converts each of the plurality of distances calculated by the distance calculation unit 103 into a geodetic distance, the greater the density in the distribution of the plurality of movement trajectories calculated by the motion analysis unit 102, the greater the density. Convert to geodetic distance after weighting so that the geodesic distance is small. This enables distance conversion in consideration of the density of the distribution of pixel movement trajectories.
- the region dividing unit 104 calculates a geodetic distance using the non-linearized distance f ′ (i, j) as in the following equation.
- min (x, y) is a function that returns the smaller of the values x and y.
- s is a movement trajectory s, which is a relay point for tracing from the movement trajectory i to the movement trajectory j.
- the relay point s in f ′ (i, s) + f ′ (s, j) is not limited to one point.
- This method is a shortest path search method called the Dijkstra method and is described in Non-Patent Document 6 below.
- FIG. 5A shows a two-dimensional data distribution.
- each data point corresponds to the movement trajectory i shown in Formula 3 or Formula 4 above.
- the distance between the data point i and the data point j is the distance between the data point i and the data point k. It becomes smaller than the distance.
- the distance between the data point i and the data point j is not the Euclidean distance, but the geodetic distance. This is the distance that the data point called is traced as indicated by an arrow.
- the distance between the data point i and the data point j is larger than the distance between the data point i and the data point k.
- the distance between the head pixel i 802 and the hand tip pixel j 803 is a distance indicated by a linear distance 801.
- the threshold R is appropriately set by performing nonlinear processing such as Expression 14 to Expression 17 above, as shown in FIG.
- the distance to the pixel j803 is a distance as a linear sum as indicated by an arrow from the pixel k804 to the pixel j. Therefore, while the linear distance 801 cannot continuously represent the shape of joints of joint objects such as a person as data, the continuity can be expressed by using the shape of joints as a distance.
- the calculation method of geodesic distance is not limited to the said Formula 17.
- step S106 the region dividing unit 104 performs clustering by detecting discontinuous points using g (i, j) obtained by performing geodetic distance conversion corresponding to the threshold value R in step S105.
- g (i, j) there is a discontinuity between the movement locus i and the movement locus j where g (i, j) is infinite.
- FIG. 7A is a diagram showing the movement trajectories a to h
- FIG. 7B is a conceptual diagram of the high-dimensional space representing the movement trajectories a to h shown in FIG. 7A. .
- FIG. 7A is a diagram showing the movement trajectories a to h
- FIG. 7B is a conceptual diagram of the high-dimensional space representing the movement trajectories a to h shown in FIG. 7A. .
- FIG. 7A is a diagram showing the movement trajectories a to h
- FIG. 7B is a conceptual diagram of the high-dimensional
- the number of the movement trajectories a to h is eight, but actually, the movement trajectory corresponding to each pixel may be used, or the movement trajectory obtained in units of blocks is used. May be.
- one point in the high-dimensional space representing the movement trajectories a to h corresponds to one movement trajectory expressed by the above equation 2. That is, a point on the high-dimensional space is a result of tracking pixels not only over an area on a single picture but over a plurality of temporally different pictures. Further, on the high-dimensional space, the distance between the points corresponds not to the Euclidean distance between the vectors but to a geodetic distance as shown in the above equation 20.
- FIG. 7C shows the clustering result.
- the distance between the movement locus a and the movement locus b shown in the above equation 3 or 4 is f (a, b)
- the threshold value is set as R
- the distance f (f, g) is assumed to have a value larger than the threshold value R.
- g (e, f), g (c, d), and g (f, g) are infinite even if the geodetic distance is obtained by the above equation 17.
- the area dividing unit 104 determines that the discontinuous points are between the movement trajectory c and the movement trajectory d, between the movement trajectory e and the movement trajectory f, and between the movement trajectory f and the movement trajectory g.
- the movement trajectories a, b, and c can be traced without passing through discontinuous points. Therefore, the movement trajectories a, b, and c do not take an infinite value.
- the moving trajectory is infinite because it passes through the discontinuous point g (c, d). In this way, the set of the movement trajectory i and the movement trajectory j where the geodetic distance is not infinite is the same cluster, and when it is infinite, it is a different cluster.
- a group in which the geodesic distance is infinite and a group in which the geodesic distance is not infinite can be arranged and separated into a total of four clusters of ⁇ 1 , ⁇ 2 , ⁇ 3 , and ⁇ 4 .
- the threshold value R may be set in accordance with the minimum size of the moving object to be detected and the movement on the image. It is also possible to calculate the detection rate and the false detection rate while changing the threshold value R according to the scene, and select the threshold value R that makes the detection rate and the false detection rate equal, that is, EER (Equal Error Rate). It is also possible to select the threshold value R by giving priority to the detection rate or giving priority to reducing the false detection rate.
- the region dividing unit 104 can determine that the set of movement trajectories whose geodetic distance is not infinite is the same cluster by making the sets of movement trajectories continuous. By making it continuous, clusters can be separated based on discontinuous points.
- the output unit 105 outputs the movement trajectory separated into clusters in S106 to the inside (memory, etc.) or outside (recording medium, display device, etc.) of the apparatus as a result of area division. Thereby, a moving body can be detected.
- the present embodiment by calculating a distance such as the above expression 3 or 4 with respect to the movement trajectory, it is possible to divide the region in consideration of the pixel position and the similarity of motion. Therefore, in the examples shown in FIGS. 7A to 7C, the difference in head movement and arm movement, and the difference in movement between upper and lower thighs are reflected, and the head, arms, and upper thighs ), The lower leg (hard) can be divided into separate clusters. Furthermore, this method can detect and divide each moving body in a scene where there are a plurality of people, as well as detection of body parts and area division of one moving body by the same method. .
- clustering is performed in a non-linear space without the need for a large amount of parameter fitting, thereby correctly segmenting a moving image including a person who moves while changing its shape. It is possible to detect a moving body in the image.
- the mobile body detection apparatus may divide the region by a method different from the region division based on the discontinuity point of the geodetic distance in the present embodiment.
- a method different from the region division in the first embodiment here, a method for performing region division without conversion to a geodetic distance will be described as a modification of the first embodiment.
- the configuration of the moving body detection apparatus basically includes an image input unit 101, a motion analysis unit 102, a distance calculation unit 103, a region division unit 104, and an output unit 105.
- this moving body detection apparatus 100 detects the moving body in a moving image by carrying out the area division
- the area dividing unit 104 according to this modification performs area division by a method different from that in the first embodiment. Hereinafter, description will be made centering on the area dividing unit 104 according to this modification.
- FIG. 8 is a flowchart showing the basic operation of the moving object detection device according to the modification of the first embodiment of the present invention.
- steps S101 to S104 are the same as steps S101 to S104 in FIG.
- the region dividing unit 104 executes steps S205 (clustering) and S206 (labeling).
- step S205 the region dividing unit 104 uses the distance f (i, j) between the movement trajectory i and the movement trajectory j calculated by the above formula 3 or the above formula 4 to move in ascending order of f (i, j).
- Clustering is performed by repeating the process of combining the trajectory i and the movement trajectory j as the same class.
- FIG. 9 (a) the movement trajectories a to h change greatly depending on the posture change even if the moving bodies are the same. However, as long as the objects are connected by joints, it can be assumed that the movement trajectory in the vicinity retains high similarity.
- FIG. 9B is a conceptual diagram of a high-dimensional space representing the movement trajectories a to h.
- a three-dimensional space is used for ease of explanation, but in reality, each element of the vector shown in Equation 2 corresponds to each dimension.
- the number of movement trajectories a to h is eight, actually, a movement trajectory corresponding to each pixel may be used, or a movement trajectory obtained in units of blocks may be used.
- one point in the high-dimensional space representing the movement trajectories a to h corresponds to one movement trajectory shown in the above equation 2. That is, a point on the high-dimensional space is a result of tracking pixels not only over an area on a single picture but over a plurality of temporally different pictures.
- the distance between the points in the high-dimensional space corresponds to a distance as shown in the above formula 3 or 4 instead of the Euclidean distance between the vectors shown in the above formula 2.
- the region dividing unit 104 performs clustering based on the discontinuity of the distance between the movement trajectories by clustering the movement trajectories based on the continuity of the distribution in the high-dimensional space composed of the movement trajectories.
- each cluster can be expected to correspond to an individual moving body or a part of the moving body, and detection of the moving body and area division can be performed.
- each area to be divided is expressed as follows.
- M is the number of areas and is determined empirically according to the scene to be used.
- a process is performed in which the movement trajectory i and the movement trajectory j are set to the same region label ⁇ m in ascending order of f (i, j) excluding itself.
- one of the movement trajectory i or the movement trajectory j is the case already belongs to the region theta k is still to belong to the even region theta k pixels area label is not granted. Further, if the movement trajectory i and the movement trajectory j already belong to different areas, the area labels are integrated.
- step S206 the area dividing unit 104 labels all the movement trajectories with respect to the process performed in step S205, and determines whether or not the number of areas is a predetermined M.
- the region dividing unit 104 sets the movement locus i and the movement locus j to the same region label ⁇ m in ascending order of f (i, j) in step S205. Repeat the process. If the number of movement trajectories belonging to each area is equal to or less than the threshold value N, it may be excluded from the area by treating it as an outlier.
- the area dividing unit 104 assigns the same area label ⁇ 1 to the movement trajectory a and the movement trajectory b.
- the area dividing unit 104 assigns the same area label ⁇ 2 .
- the region dividing unit 104 assigns the same region label ⁇ 3 to the movement locus d and the movement locus e which are the third smallest distance.
- the next smallest distance is a distance f (b, c) between the movement locus b and the movement locus c.
- the area dividing unit 104 assigns the same area label ⁇ 1 as the movement trajectory b to the movement trajectory c.
- the next smallest distance is the distance f (f, g) between the movement locus f and the movement locus g.
- the area dividing unit 104 assigns the same area label ⁇ 3 as the movement locus g to the movement locus f.
- the movement trajectory that is continuous in the high-dimensional space is determined as one cluster, and the distance between the movement trajectories is large.
- Each of the clusters can be separated using as a discontinuous point.
- the movement trajectory belonging to each cluster corresponds to the moving object detection and area division.
- this method can detect and divide each moving body in a scene where there are a plurality of people, as well as detecting body parts and area division of a single moving body by the same method. .
- FIG. 10 is a functional block diagram showing the configuration of the moving object detection apparatus 100a according to the second embodiment.
- the moving body detection apparatus 100a includes an image input unit 101, a motion analysis unit 102, a distance calculation unit 103, a region division unit 104a (region division candidate generation unit 501 and region division candidate selection unit 502), and An output unit 105 is provided. Since the image input unit 101, the motion analysis unit 102, and the distance calculation unit 103 are the same as those in the first embodiment, description thereof is omitted.
- the region dividing unit 104a performs region division by specifying a region composed of blocks having similar movement trajectories based on the distance calculated by the distance calculating unit 103. Although the same as the unit 104, the specific processing is different.
- the area dividing unit 104a includes an area dividing candidate generating unit 501 and an area dividing candidate selecting unit 502.
- the region division candidate generation unit 501 generates a plurality of determination criteria used for region division, and uses the determination criteria for each of the generated plurality of determination criteria to generate a plurality of movements calculated by the motion analysis unit 102.
- This is a processing unit that divides a region by clustering a locus and generates a result of the region division as a region division candidate.
- the region division candidate generation unit 501 applies geodetic distance conversion to the distance matrix calculated by the distance calculation unit 103 using a threshold relating to data continuity, that is, calculated by the distance calculation unit 103.
- Discontinuous points in the distribution of distances between moving trajectories after conversion of geodetic distances at each threshold by converting each of the plurality of distances into geodetic distances by connecting distances smaller than the threshold among the plurality of distances By generating a cluster of moving trajectories that are continuously distributed so that the moving trajectories separated by a geodetic distance smaller than the detected discontinuous point become one cluster, a candidate for region division for the threshold is generated. To do.
- the number of area divisions varies depending on the setting of the threshold.
- the area division candidate selection unit 502 obtains an instruction about the number of classes (or a preset number of classes), and obtains an area division candidate divided into a number of areas close to the obtained number of classes. This is a processing unit that selects from a plurality of region division candidates generated in 501 and outputs the selected region division candidate as a result of region division based on the distance calculated by the distance calculation unit 103. Specifically, the region division candidate selection unit 502 selects the region division result closest to the predetermined number of classes from the region division candidates for each of the threshold values generated by the region division candidate generation unit 501. That is, a region division result with a threshold corresponding to the number of classes is selected.
- the output unit 105 is the same as that in the first embodiment. By the output from the output unit 105, it is possible to obtain a final moving object detection and region division result.
- Steps S101 to S104 are the same as those in the first embodiment, and a description thereof will be omitted.
- the area division candidate generation unit 501 executes Step S601 (geodetic distance conversion) and Step S602 (clustering).
- step S601 the region division candidate generation unit 501 uses K threshold values R k for the linear distance f (i, j) calculated by the above equation 3 or 4, and for each threshold value R k. Then, non-linearization processing is performed as follows to calculate f ′ (i, j).
- the region division candidate generation unit 501 selects R k movement trajectories j in ascending order of the linear distance from the movement trajectory i when paying attention to the movement trajectory i, and changes the distance from the selected movement trajectory j.
- the distance to the unselected movement trajectory j is changed to infinity.
- the linear distance is selected in ascending order, but the threshold value R k may be set as in the following equation.
- the region division candidate generation unit 501 selects a predetermined number of movement trajectories in ascending order of distance for each of the plurality of movement trajectories calculated by the motion analysis unit 102, as shown in Equation 22 above. After non-linearization that changes the distance to the unselected movement trajectory to infinity, each of the plurality of distances may be converted into a geodetic distance. For each of a plurality of calculated movement trajectories, select a movement trajectory whose distance is equal to or less than a predetermined threshold, and perform nonlinearization to change the distance from the movement trajectory not selected to infinity, then the plurality of distances Each of these may be converted into geodetic distances.
- the distance non-linearization is not limited to the above function, and any distance conversion may be used as long as the nonlinear transformation is performed on the distances related to the movement trajectory i and the movement trajectory j.
- the weighting obtained by multiplying the linear distance f (i, j) by the weights calculated using the above-described Expression 16 and Expression 17 may be performed, and then the processing of Expression 22 or Expression 23 may be performed. Good.
- f (i, j) By converting f (i, j) using the weights of the above formula 17 and the above formula 18, when the movement trajectories similar to the movement trajectories i and j are spatially dense (the movement trajectory having a short distance) In the case of f), f (i, j) is relatively small, and in the case of spatial sparseness (in the case of a moving trajectory with a long distance), f (i, j) j) becomes relatively large. That is, when each of the plurality of distances calculated by the distance calculation unit 103 is converted into a geodetic distance, the smaller the geodetic distance in the distribution of the plurality of movement trajectories calculated by the motion analysis unit 102, the smaller the geodetic distance. Convert to geodetic distance after weighting properly. This enables distance conversion in consideration of the density of the distribution of pixel movement trajectories.
- the region division candidate generation unit 501 calculates a geodetic distance as in the following equation.
- min (x, y) is a function that returns the smaller of the values x and y.
- s is a movement trajectory s, which is a relay point for tracing from the movement trajectory i to the movement trajectory j.
- the relay point s in f ′ k (i, s) + f ′ k (s, j) is not limited to one point.
- k corresponds to a plurality of thresholds R k.
- this method is a shortest path search method called the Dijkstra method.
- step S602 the region division candidate generation unit 501 performs clustering by detecting discontinuous points using g k (i, j) obtained by performing geodetic distance conversion corresponding to each threshold value R k. I do.
- FIG. 12A is a diagram showing the movement trajectories a to h
- FIG. 12B is a conceptual diagram of the high-dimensional space representing the movement trajectories a to h shown in FIG. .
- the number of movement trajectories a to h is eight, actually, a movement trajectory corresponding to each pixel may be used, or a movement trajectory obtained in units of blocks may be used.
- one point in the high-dimensional space representing the movement trajectories a to h corresponds to one movement trajectory shown in the above equation 2. That is, a point on the high-dimensional space is a result of tracking pixels not only over an area on a single picture but over a plurality of temporally different pictures. Further, on the high-dimensional space, the distance between the points corresponds not to the Euclidean distance between the vectors but to the geodetic distance as shown in the above equation 21.
- the threshold value R k is a sufficiently large value, for example, when the threshold value R k is larger than the maximum value of f (i, j) in the above equation 16, the geodetic distance g as shown in FIG. k (i, j) does not become infinite in all combinations of i, j. That is, since there is no discontinuous point, it can be determined that there is one cluster.
- the threshold value R k is sufficiently small, specifically, when the threshold value R k is smaller than the minimum value of f (i, j) in the above equation 16, g k in all combinations of i and j. (I, j) becomes infinite. That is, the number of clusters is the same as the number of movement trajectories.
- g 1 (e, f) is infinite even if the geodetic distance is obtained by the above equation 18. Therefore, the region division candidate generation unit 501 determines that the area between the movement locus e and the movement locus f is a discontinuous point. As a result, since the geodesic distance between the movement trajectory a to d and the movement trajectory e does not pass through the discontinuous point, it does not take an infinite value. Conversely, the movement trajectories f to h and the movement trajectories a to e The geodetic distance to each movement locus is infinite because it passes through the discontinuous point g 1 (e, f).
- the region division candidate generation unit 501 sets the pair of the movement trajectory i and the movement trajectory j whose geodetic distance does not become infinite as the same cluster, and sets it as another cluster when it becomes infinite. As a result, as shown in FIG. 12D, it can be separated into two clusters of ⁇ 1 and ⁇ 2 . Furthermore, as shown in FIG. 12E, when the threshold value is R 2 (where R 1 > R 2 ), geodesic distances that are infinite are g 2 (c, d), g 2. Assume that (e, f) and g 2 (f, g).
- the area division candidate generation unit 501 has a discontinuity between the movement locus c and the movement locus d, between the movement locus e and the movement locus f, and between the movement locus f and the movement locus g, respectively.
- the set of geodesic distances and the sets that do not become infinite are arranged, and ⁇ 1 , ⁇ 2 , ⁇ 3 , Separate into four clusters of ⁇ 4 .
- a set of movement trajectories where the geodetic distance is not infinite can be determined as the same cluster by making it continuous, and a set of movement trajectories where the geodetic distance is infinite is made discontinuous, Clusters can be separated based on discontinuities.
- the setting method of the threshold value R k all the movement trajectories are determined to be one cluster by setting K pieces uniformly between the minimum value and the maximum value of f (i, j), or each movement This has the effect of reducing the risk that the trajectory is determined to be an individual cluster.
- the region division candidate generation unit 501 uses a plurality of values between the maximum value and the minimum value at a plurality of distances calculated by the distance calculation unit 103 as a plurality of threshold values for generating a plurality of region division candidates. It may be generated.
- an average value of f (i, j) or a value that is increased or decreased at regular intervals around the median may be used.
- the threshold value corresponding to the discontinuous point can be determined more efficiently. That is, the region division candidate generation unit 501 generates a plurality of region division candidates from a plurality of values obtained by increasing and decreasing the average value or median value of the plurality of distances calculated by the distance calculation unit 103. It may be generated as a plurality of threshold values.
- the value may be decreased at regular intervals from the maximum value, and a smaller value may be determined as the threshold value by K ⁇ 1 based on the value when the discontinuous point is first detected.
- step S601 and step S602 may be repeated while the steps S601 and step S602 to reduce the threshold R k.
- area dividing candidate generating unit 501, the threshold value R 1, at step S601 and step S602, performs clustering movement trajectory, then, in step S601, Fig.
- each is converted into a geodetic distance, and in step S602, from g (i, j) corresponding to each cluster (where i, j is a movement locus belonging to the same cluster), discontinuity is obtained.
- Clustering can be performed by detecting points. In this way, by converting the geodesic distance and detecting the discontinuous points with respect to the movement trajectory belonging to each cluster while reducing the threshold value R k , a hierarchical structure as shown in FIG. Clustering becomes possible.
- clustering is performed by increasing the threshold value in order from the smallest value, thereby enabling hierarchical clustering.
- processing can be performed with a small amount of calculation.
- clustering by the threshold R 2 since it is sufficient for each cluster extracted in the threshold value R 1, is used to calculate f (i, j) (where, i, j can reduce the total size of movement trajectories belonging to the same cluster. Therefore, the calculation amount can be reduced.
- the scene structure can be expressed as a tree structure as shown in FIGS.
- FIGS For example, when clustering is performed on the input of FIG. 13A with the threshold value R 1 , clusters ⁇ 1 and ⁇ 2 are extracted as shown in FIG. 13C. Further, when clustering is performed with a smaller threshold R 2 , a smaller subject ⁇ 3 can be extracted.
- ⁇ 0 corresponds to the background.
- the camera when the camera is fixed, it is possible to extract only moving objects by using only moving trajectories with movement, and when the camera moves, the cluster with the largest number of moving trajectories is used as the background. It is also good.
- clustering is performed with a threshold value smaller than the above example
- clustering may be performed with a smaller threshold for all clusters, or clustering may be performed on a cluster (subject) designated by the user.
- R 3 threshold value
- it can be classified into a right leg, a left leg, and an upper body as shown on the right side of FIG.
- walking is extracted with a large threshold value compared to other parts because the movement of the leg is large.
- the right leg and the left leg are classified into the upper leg ( ⁇ 5 , ⁇ 7 )) and the lower leg ( ⁇ 6 , ⁇ 8 )), respectively, and the upper body to the arm ( ⁇ 3 ) Is extracted.
- the upper body can be clustered into the head ( ⁇ 1 ), chest ( ⁇ 2 ), and abdomen ( ⁇ 4 ).
- clustering based on a hierarchical structure reflecting the discontinuity of motion is possible.
- the clustering that reflects the distribution of all moving trajectories is performed by clustering each cluster in detail with a smaller threshold value for the extracted clusters. Can be realized.
- the region division candidate generation unit 501 detects the first discontinuous point in the case where the plurality of distances calculated by the distance calculation unit 103 are arranged in order from the largest value to the smallest value, and more than the detected discontinuous point.
- a plurality of small values may be generated as a plurality of threshold values for generating a plurality of region division candidates.
- the threshold value R k may be set as follows. First, a movement trajectory j that is the Nth smallest distance from the movement trajectory i may be calculated, and the value may be set as the threshold value R k , and processing may be performed in descending order of R k .
- N is, for example, a value obtained by dividing the number of movement trajectories used for processing by the number of clusters to be set, so that there is an effect that it is easy to set the threshold value R k that is the number of clusters close to the desired number of clusters.
- the region division candidate generation unit 501 specifies the Nth smallest distance from the movement trajectory for each of the plurality of movement trajectories calculated by the motion analysis unit 102, and increases the order of the identified plurality of distances. May be generated as a plurality of thresholds for generating a plurality of region division candidates.
- the region division candidate selection unit 502 selects the clustering result that has the closest number of clusters to the preset number of classes from the plurality of clustering results performed in step S602.
- the clustering result at the threshold R 2 (FIG. 12E) is selected.
- the clustering result at the threshold R 1 is selected, and the output unit 105 outputs the result of labeling each movement trajectory. .
- the number of thresholds is not limited to two, and any number of thresholds may be prepared. If there are not the same number of clustering results as the set number of classes, the cluster with the closest number of clusters may be selected, or the closest number of clusters out of the set number of classes. The clustering result that becomes the closest cluster number may be selected from among the class numbers larger than the set class number.
- the clustering result at the threshold value R 1 and the clustering result at the threshold value R 2 can be obtained as a hierarchical structure. is there.
- it can be obtained as a hierarchical relationship. In this way, the structure of the subject in the image can be extracted as a hierarchical structure.
- this method can detect and divide each moving body in a scene where there are a plurality of people, as well as detection of body parts and area division of one moving body by the same method. .
- clustering is performed based on discontinuous points calculated using geodetic distances for a plurality of threshold values, and the clustering result closest to the specified number of classes is selected, resulting in detection and region segmentation. It can be performed.
- clustering is performed based on the distance between the pixels or the similarity of the movement trajectories, so that the similar movement trajectories are grouped and divided into regions, so that the distance is close and the movement is similar.
- the moving object in the moving image or the part of the moving object is detected regardless of the posture of the joint object. , It is possible to divide an image including a moving body.
- clustering based on discontinuous points in this embodiment can be performed without being affected by the size of the cluster size, a large subject and a small subject may be mixed, or a large moving subject may be small. Even when moving subjects are mixed, it is possible to extract them reliably.
- by performing clustering while reducing the threshold value it is possible to perform hierarchical clustering from coarse clustering to detailed clustering, so that it is possible to hierarchically extract the structure of the scene and the structure of the body of the subject. . Accordingly, there is an effect that the same method can be used from the analysis of a rough image that extracts the position of the subject in the image to the detailed analysis of a specific image region or subject region.
- the extracted hierarchical structure indicates the data structure of all the movement trajectories, it can also represent the structure of the scene in the input image.
- clustering is performed in a non-linear space without the need for a large amount of parameter fitting, thereby correctly segmenting a moving image including a person who moves while changing its shape. It is possible to detect a moving body in the image.
- the moving body detection apparatus 100b includes an image input unit 101, a motion analysis unit 102, a distance calculation unit 103, an area division unit 104, and an output unit 105a.
- the output unit 105a has an image display unit 1001 in addition to the function of the output unit 105 in the first embodiment, and the image display unit 1001 can display the divided area as an image on a monitor or the like. Is possible.
- the image display unit 1001 is a processing unit that displays the result of the region division obtained by the region division unit 104 on the moving image (picture) received by the image input unit 101.
- a display such as an LCD and its display It consists of a display controller.
- FIGS. 15A and 15B are diagrams showing display examples by the image display unit 1001.
- the area on the image corresponding to the area ⁇ m is color-coded and displayed on the monitor or the like so that the divided areas can be distinguished from each other.
- each moving object may be displayed in different colors, or as shown in FIG. 15 (b), it may be displayed separately for each part. That is, as can be seen from FIGS. 15A and 15B, the moving body detection apparatus according to the present invention can detect a plurality of moving bodies by the above-described region division (FIG. 15A). It is also possible to detect a plurality of parts constituting one moving body, that is, to divide the region into one moving body (FIG. 15B).
- the movement trajectory i shown in the above equation 2 corresponds to one of the regions ⁇ m excluding the outlier. Therefore, if a pixel on the picture is selected based on the movement trajectory i belonging to the region ⁇ m and color coding or the like is performed based on the region label m, the divided region can be easily displayed on the image. As a result, in addition to the effect described in the first and second embodiments that a moving image including a person who moves while changing its shape can be correctly divided into regions, the divided regions are displayed on the moving image. Thus, there is an effect that it becomes easy for the user to distinguish each moving body or a part of the moving body, and further its movement.
- the moving body detection apparatus 100c includes an image input unit 101, a motion analysis unit 102, a distance calculation unit 103, a region division unit 104, and an output unit 105b.
- the output unit 105b includes a recording / transmission unit 1201 in addition to the function of the output unit 105 in the first embodiment.
- the recording / transmission unit 1201 identifies the region in the picture received by the image input unit 101 based on the region division result in the region division unit 104, and stores the corresponding region division result for each identified region in the memory. Or a processing unit that records in a recording medium such as a hard disk or transmits to the outside via a communication interface or the like. That is, the recording / transmission unit 1201 separately records and transmits the result image obtained by the area division according to the area label ⁇ m as in the case of image display. It is also possible to compress the motion information by averaging the motion information belonging to each area as follows. Normally, it is necessary to hold motion information for each pixel, but if the following processing is performed, it is only necessary to hold one motion for one region. For example, when the motion vector (u i t , v i t ) of the pixel i is used as the motion information, the motion information averaged for each of the divided areas can be calculated as follows.
- FIG. 17 shows an example of segmenting the processing result shown in FIG. 15A by inputting t images from time T and recording and transmitting the segment area using the result.
- each region label may be attached to one picture at time T and transmitted.
- recording and transmission may be performed according to the hierarchical clustering result. For example, when the data storage capacity or transfer capacity is small, a motion vector corresponding to the clustering result with a larger threshold value may be recorded and transmitted. Motion vectors based on the clustering result may be recorded and transmitted sequentially. Thus, there is an effect that recording and transmission adaptive to the recording time and the transmission capacity can be performed.
- the picture at time T is restored from the pixel position and pixel value corresponding to each region label shown in FIGS. 15 (a) and 15 (b). Furthermore, it is possible to restore the picture from time T + 1 to T + t by moving each pixel at time T using the pixel movement trajectory information.
- the restored image is the background image. May be overwritten. Accordingly, there is an effect that the information can be restored as a picture with a low calculation amount by using the information transmitted and recorded efficiently.
- Embodiments 1 and 2 The difference from Embodiments 1 and 2 is the processing in the distance calculation unit 103.
- the distance calculation unit 103 is different from the first embodiment in that an additional parameter is used in order to perform processing with higher accuracy.
- an additional parameter is used in order to perform processing with higher accuracy.
- the distance calculation unit 103 measures the distance between the movement trajectories on the image and the distance between them, and the angle formed between the pixels on the image (that is, the inclination angle of the straight line connecting the pixels (the straight line and the side of the picture). Using the angle formed by the axis)) and the distance measure for its variation, a distance matrix is calculated for each distance measure. Then, by integrating both matrices, more accurate processing is realized.
- steps S101 to S103 are the same as those in the first embodiment, description thereof is omitted.
- step S104 the distance calculation unit 103 calculates a distance matrix including the similarity of pixel motion using the movement trajectory i calculated by Equation 2 above.
- the second distance scale is expressed by the following expression 27 based on the angle formed by the pixel i and the pixel j. An example using the distance f 2 (i, j) to be transmitted will be described.
- w a is a weighting coefficient and is a parameter set by the designer.
- a_mean ij and a_var ij are shown below.
- the similarity between the movements of the pixels can be expressed by using the angle formed by the pixel i and the pixel j and the fluctuation component thereof. As a result, it is possible to capture not only a rigid body but also a shape change such as a joint object with a rotational motion.
- Equation 31 to Equation 33 may be any distance value that can evaluate the similarity of pixel motion.
- Equation 34 using the affine parameter A i t of the above equation 2 may be used.
- Expressions 31 to 33 below can represent the similarity of pixel motion using the angular difference between the motion vectors of the pixel i and the pixel j and the fluctuation component thereof. This makes it possible to capture changes in motion including rotation.
- Equation 34 below can represent the similarity of pixel motion using the difference in affine parameters between neighboring blocks of pixel i and pixel j and their fluctuation components. As a result, changes in motion including rotation, translation, and scale changes can be captured.
- the distance matrix calculated by the above equation 27 is added to the distance matrix obtained by the above equation 3 as in the following equation 35.
- two distances, the distance between pixels and the angle formed between the pixels, will be described.
- three or more distances may be obtained using a distance expressing the similarity of other pixel motions.
- w is a weighting factor set by the designer.
- step S105 Since the same processing may be performed after step S105 using the distance matrix calculated by the above equation 35, the following description is omitted.
- the distance between the pixels on the image and the distance scale regarding the variation the angle between the pixels on the image and the distance scale regarding the variation, and
- the distance matrix for each distance measure By calculating the distance matrix for each distance measure, integrating the distance matrices, and then clustering based on continuity, the region of the object moving in the moving image with higher accuracy is temporally As a result of tracking, it can be detected and divided into regions. Further, by using the angle formed between the pixels on the picture and the variation thereof as a distance scale, it is possible to more accurately capture the rotational motion particularly caused by the joint motion.
- the moving object is added with a function for predicting the movement of the moving object from the result of detecting the moving object and dividing the area using the moving object detecting method described in the first, second, and second embodiments.
- the body detection device will be described.
- it demonstrates along Embodiment 1, it is realizable similarly in Embodiment 1, 2, and those modifications.
- FIG. 18 is a functional block diagram showing the configuration of the moving object detection device 100d according to the third embodiment.
- the moving body detection apparatus 100d according to the present embodiment adds a motion prediction unit 1401 to the image input unit 101, the motion analysis unit 102, the distance calculation unit 103, the region division unit 104, and the output unit 105.
- a motion prediction unit 1401 to the image input unit 101, the motion analysis unit 102, the distance calculation unit 103, the region division unit 104, and the output unit 105.
- the output unit 105 has a function of predicting the movement of the moving body.
- the motion prediction unit 1401 calculates a representative trajectory from the movement trajectory of the pixels included in each region from the result of the region division, and predicts the motion of the moving object based on the representative trajectory.
- FIG. 19 shows a flowchart of processing according to the third embodiment. Steps S101 to S106 are the same as those in the first embodiment, and a description thereof will be omitted.
- step S1501 the motion prediction unit 1401 uses the region division results obtained in steps S105 and S106 to obtain the representative points of clusters and their representative trajectories as follows.
- the movement trajectory of the pixel belonging to the region ⁇ m is expressed as x Cm .
- a representative movement trajectory is obtained for each cluster region ⁇ m .
- weighting or the like may be performed for each pixel movement trajectory x Cm in the following calculation, or a cluster on an image may be calculated.
- a pixel movement locus corresponding to the center of gravity may be used as a representative movement locus.
- C m is the number of pixels belonging to the region ⁇ m or the number of pixel trajectories.
- FIG. 20 shows an example in which a representative movement locus is obtained for each cluster region ⁇ m based on the above equation 36. However, for ease of viewing, only representative movement trajectories relating to the cluster region ⁇ 1 corresponding to the head and the cluster region ⁇ 8 corresponding to the legs are shown.
- “x” is an element of x m ⁇ corresponding to time t, and indicates a pixel position.
- Equation 3 and Equation 4 above region segmentation by clustering in a non-linear space is performed in consideration of the similarity of pixel motion, so the time average of the movement trajectories of adjacent pixels is simply Compared with the method for obtaining the above, since it is possible to calculate using only the movement trajectory of the pixel having similar motion, the representative movement trajectory can be obtained with higher accuracy. In this way, by obtaining a representative movement trajectory for each cluster region, it is possible to accurately and easily represent the movement of each part.
- step S1502 the motion prediction unit 1401 predicts the position of the moving body at a time earlier than time T from the representative movement trajectory calculated in step S1501.
- acceleration is calculated from a representative movement locus and the position of the moving body after T + 1 is predicted. If three or more time-series images are inputted, it is possible to obtain an acceleration vector s m for each representative movement trajectory x m-as the following equation 37.
- u m t is a motion vector and can be expressed as the following Expression 38.
- Equation 39 the position position pos m (T + t ′) of the moving object at time T + t ′ is expressed as follows for each position of the moving object: It can be predicted as shown in Equation 39.
- the part of the moving body is taken as an example, but the position for each moving body can also be predicted from the detection examples shown in FIGS.
- the output unit 105 outputs the position of the moving body and the position of the moving body part predicted in step S1502. Thereby, prediction in consideration of acceleration is possible. There is an effect that the position of the moving body can be predicted by reflecting the acceleration when the movement is rapidly accelerated or stopped suddenly. Further, an affine parameter may be used instead of the motion vector. Since the affine parameter can express a motion including a rotational motion and is suitable for a rotational motion of an arm or a leg, the position of the joint object can be predicted more accurately.
- the representative movement trajectory can be obtained with high accuracy.
- the present invention includes embodiments realized by arbitrarily combining the characteristic components in all the embodiments and all modifications described so far.
- FIGS. 21A to 21C show examples of figures that can be separated when the method of the second embodiment is used.
- the moving directions of the respective figures ⁇ 1 and ⁇ 2 shown in FIGS. 21A to 21C may be the same or different.
- the condition is that all the pixels belonging to the graphic ⁇ 1 have the same movement, and all the pixels belonging to the graphic ⁇ 2 have the same movement.
- step S601 in the second embodiment geodetic distance conversion having the characteristics shown in FIG. 5 is performed.
- the distance obtained by tracing (traversing) the movement trajectory from the movement trajectory can be calculated. Therefore, the distance along the shape can be calculated with respect to the curved shape so as to be common to FIGS.
- step S602 clustering is performed using a discontinuous point between the movement locus where the geodetic distance is infinite. For this reason, when there is a distance equal to or greater than the threshold between the movement trajectory, it is possible to divide ⁇ 1 and ⁇ 2 respectively based on the discontinuous points.
- region segmentation with geodetic distance conversion it is possible to segment regions based on discontinuous points when there is a moving object that includes a curved shape and there are certain discontinuous points. is there.
- step S601 when the process of step S601 is not performed, the distance between the movement trajectory belonging to ⁇ 1 and the movement trajectory belonging to ⁇ 2 is calculated as the Euclidean distance. Therefore, as in the case described with reference to FIG. The distance between i and point j is greater than the distance between point i and point k. That is, the distance between points belonging to the same class is larger than the distance between points belonging to another class. Further, in the process of step S602, in order not to perform the process of tracing (tracing) data unless the geodetic distance conversion is performed, it is determined whether the distance between the adjacent movement trajectories is discontinuous or continuous. I can't. Therefore, in the example shown in FIGS. 21 (a) to 21 (c), when a mobile object includes a curved shape and there are certain discontinuous points, the region is not divided based on the discontinuous points. difficult.
- continuity related to similarity between moving trajectories is obtained by clustering using geodetic distances as compared to clustering using Euclidean distances that are linear distances. Clustering in consideration is performed, and even if the regions have complicated relations, it is reliably discriminated whether they belong to the same object (or part) or different objects (or parts).
- the present invention is an apparatus for detecting all or part of a moving object in a moving image, that is, a person who moves while changing its shape based on the movement of a block composed of one or more pixels in a plurality of images.
- a moving body detection device that detects a moving body in an image by dividing an image including the moving body of a moving body
- a moving body detection device incorporated in an AV apparatus such as a motion analysis device, a monitoring device, a video camera, or a TV Etc.
Abstract
Description
図1は、実施の形態1における移動体検出装置100の構成を示す機能ブロック図である。図1に示されるように、この移動体検出装置100は、画像入力部101、動き解析部102、距離算出部103、領域分割部104、出力部105を備える。そして、この移動体検出装置100は、動画像中の移動体の全部又は一部の領域を特定する領域分割をすることによって動画像中の移動体を検出する。
次に、上記実施の形態1で行った非線形空間上での領域分割を、実施の形態1とは異なる方法で実現する方法について、実施の形態2として説明する。
次に、本発明の実施の形態1及び2の第1変形例における移動体検出装置について説明する。ここでは、実施の形態1及び2において移動体を検出及び部位を領域分割した結果を画像として表示する機能を付加した例について説明する。ここでは、実施の形態1における変形例について述べるが、実施の形態1の変形例、実施の形態2においても同様に適用可能である。このような第1変形例に係る移動体検出装置100bは、図14の機能ブロック図に示すように、画像入力部101、動き解析部102、距離算出部103、領域分割部104及び出力部105aを備える。ここで、出力部105aは、実施の形態1における出力部105の機能に加えて、画像表示部1001を有し、その画像表示部1001によって、分割した領域を画像としてモニタ等に表示することが可能である。
次に、本発明の実施の形態1及び2の第2変形例における移動体検出装置について説明する。ここでは、実施の形態1及び2において移動体を検出及び部位を領域分割した結果をそれぞれ分割した領域ごとに記録・送信する機能を付加した例について説明する。ここでは、実施の形態1における変形例について述べるが、実施の形態1の変形例、実施の形態2においても同様に適用可能である。このような第2変形例に係る移動体検出装置100cは、図16の機能ブロック図に示すように、画像入力部101、動き解析部102、距離算出部103、領域分割部104及び出力部105bを備える。ここで、出力部105bは、実施の形態1における出力部105の機能に加えて、記録・送信部1201を有する。
次に、本発明の実施の形態1及び2の第3変形例における移動体検出装置について説明する。ここでは、2つ以上の距離尺度を用いることで、検出と領域分割をより高精度に実現する例について説明する。ここでは、実施の形態1における変形例について述べるが、実施の形態1の変形例、実施の形態2においても同様に適用可能である。このような第3変形例に係る移動体検出装置は、実施の形態1と同じ構成であるため、その構成の説明は省略する。
本実施の形態では、実施の形態0、1及び2で説明した移動体検出方法を用いて、移動体を検出及び領域分割を行った結果から、移動体の動きを予測する機能を付加した移動体検出装置について説明する。ここでは、実施の形態1に沿って説明するが、実施の形態1、2、それらの変形例においても同様に実現可能である。
最後に、測地距離変換を用いた領域分割によって分割される画像上の領域の性質について説明する。ここでは、一例として、実施の形態2で説明した方法を用いた場合に、領域分割可能な移動体の性質について補足する。図21(a)~(c)に実施の形態2の方法を用いた場合に分離可能な図形の一例を示す。実施の形態2の方法を用いることで、それぞれ図形θ1とθ2とに領域分割することが可能である。ここでは、図21(a)~(c)に示したそれぞれの図形θ1とθ2の移動方向は、同一であっても、異なっていてもかまわない。ただし、図形θ1に属する画素はすべて同じ動きであり、かつ図形θ2に属する画素はすべて同じ動きであることが条件である。
101 画像入力部
102 動き解析部
103 距離算出部
104、104a 領域分割部
105、105a、105b 出力部
501 領域分割候補生成部
502 領域分割候補選択部
1001 画像表示部
1201 記録・送信部
1401 動き予測部
Claims (21)
- 動画像中の移動体の全部又は一部の領域を特定する領域分割をすることによって動画像中の移動体を検出する方法であって、
動画像を構成する複数枚のピクチャを受け付ける画像入力ステップと、
前記ピクチャを構成する1個以上の画素からなるブロックごとに、時間的に隣接する2枚のピクチャ間での画像の動きを検出し、検出した動きを前記複数枚のピクチャについて連結することで、移動軌跡を算出する動き解析ステップと、
前記動き解析ステップで算出された複数の移動軌跡について、移動軌跡間の類似性を表す距離を算出する距離算出ステップと、
前記距離算出ステップで算出された距離のうち、予め定められた閾値よりも小さい距離を連結することで、前記距離算出ステップで算出された距離を測地距離に変換し、得られた測地距離の分布における不連続点を検出し、検出した不連続点よりも小さい測地距離だけ離れた移動軌跡を一つのクラスタとすることによって前記領域分割をする領域分割ステップと、
前記領域分割ステップで領域分割された結果を出力する出力ステップと
を含む移動体検出方法。 - 前記領域分割ステップでは、前記距離から前記測地距離への変換において、第1の移動軌跡と第2の移動軌跡との間の距離を測地距離に変換する場合には、前記距離算出ステップで算出された距離のうち、前記予め定められた閾値よりも小さい距離だけ離れた移動軌跡をたどりながら前記第1の移動軌跡から前記第2の移動軌跡に至る経路の距離を、測地距離として算出する
請求項1記載の移動体検出方法。 - 前記領域分割ステップでは、前記距離算出ステップで算出された距離を測地距離に変換するときに、前記複数の移動軌跡の分布における密集度が大きいほど小さい測地距離となるような重み付けをしたうえで、前記変換をする
請求項1記載の移動体検出方法。 - 前記領域分割ステップは、前記領域分割に用いられる閾値を複数生成し、生成した複数の閾値のそれぞれについて、前記距離算出ステップで算出された距離のうち、当該閾値よりも小さい距離を連結することで、前記距離算出ステップで算出された距離を前記測地距離に変換し、得られた複数の測地距離の分布における不連続点を検出し、検出した不連続点よりも小さい測地距離だけ離れた移動軌跡を一つのクラスタとすることによって前記領域分割をし、その領域分割の結果を領域分割候補として生成する領域分割候補生成ステップと、
クラス数についての指示を取得し、所得したクラス数と同じもしくは最も近い個数の領域に分割された領域分割候補を前記領域分割候補生成ステップで生成された複数の領域分割候補から選択し、選択した領域分割候補を前記領域分割の結果として出力する領域分割候補選択ステップを含む
請求項1記載の移動体検出方法。 - 前記領域分割候補生成ステップでは、前記距離算出ステップで算出された複数の距離における最大値と最小値との間の複数の値を、前記閾値として、生成する
請求項4記載の移動体検出方法。 - 前記領域分割候補生成ステップでは、前記距離算出ステップで算出された複数の距離について、閾値を大きい値から小さい値の順に並べた場合における最初の不連続点を検出し、検出した不連続点よりも小さい複数の値を、前記複数の閾値として、生成する
請求項4記載の移動体検出方法。 - 前記領域分割候補生成ステップでは、前記距離算出ステップで算出された複数の距離について不連続点の検出を行い、閾値の大小に基づいて階層的に前記領域分割をする
請求項4記載の移動体検出方法 - 前記領域分割候補生成ステップでは、前記距離算出ステップで算出された複数の距離について、閾値を大きい値から不連続点の検出を行い、分割されたクラスタに対して、それぞれ、より閾値を用いて不連続点の検出を行い、階層的に前記領域分割をする
請求項7記載の移動体検出方法 - 前記領域分割候補生成ステップでは、前記距離算出ステップで算出された複数の距離の平均値又は中央値を中心に増加及び減少させて得られる複数の値を、前記複数の閾値として、生成する
請求項4記載の移動体検出方法。 - 前記領域分割候補生成ステップでは、前記動き解析ステップで算出された複数の移動軌跡のそれぞれについて、当該移動軌跡との距離がN番目に小さい距離を特定し、特定した複数の距離について大きい順から選択した複数の値を、前記複数の閾値として、生成する
請求項4記載の移動体検出方法。 - 前記領域分割ステップでは、前記動き解析ステップで算出された複数の移動軌跡のそれぞれについて、前記距離が小さい順に予め定められた個数の移動軌跡を選択し、選択されなかった移動軌跡との距離を無限大に変更する非線形化をした後に、前記複数の距離のそれぞれを測地距離に変換する
請求項1記載の移動体検出方法。 - 前記領域分割ステップでは、前記動き解析ステップで算出された複数の移動軌跡のそれぞれについて、前記距離が予め定められた閾値以下の移動軌跡を選択し、選択されなかった移動軌跡との距離を無限大に変更する非線形化をした後に、前記複数の距離のそれぞれを測地距離に変換する
請求項1記載の移動体検出方法。 - 前記動き解析ステップでは、前記動きの検出として、前記動きを示す2次元動きベクトル又はアフィンパラメータを算出する
請求項1記載の移動体検出方法。 - 前記距離算出ステップでは、前記距離の算出として、前記ブロックの移動軌跡間の類似性に加えて、前記ピクチャでの前記ブロック間の距離及び前記ブロックどうしを接続する直線の傾斜を示す角度の少なくとも1つを算出する
請求項1記載の移動体検出方法。 - 前記出力ステップは、前記領域分割ステップで得られた領域分割の結果を、前記画像入力ステップで受け付けたピクチャに重ねて表示する表示ステップを含む
請求項1記載の移動体検出方法。 - 前記画像入力ステップでは、2つ以上の移動体が含まれる動画像を受け付け、
前記領域分割ステップでは、前記2以上の移動体について前記領域分割をすることで、2以上の移動体を検出する
請求項1記載の移動体検出方法。 - 前記移動体検出方法はさらに、前記領域分割ステップで特定された領域を構成するブロックの移動軌跡から、当該領域を代表する移動軌跡を算出し、算出した代表の移動軌跡に従って当該領域が移動すると予測することで、前記移動体の動きを予測する動き予測ステップを含む
請求項1記載の移動体検出方法。 - 前記出力ステップは、前記領域分割ステップでの領域分割の結果に基づいて、前記画像入力ステップで受け付けたピクチャにおける領域を特定し、特定した領域ごとに、対応する領域分割の結果を、記憶手段に記録する、又は、送信する記録・送信ステップを含む
請求項1記載の移動体検出方法。 - 動画像中の移動体の全部又は一部の領域を特定する領域分割をすることによって動画像中の移動体を検出する移動体検出装置であって、
動画像を構成する複数枚のピクチャを受け付ける画像入力部と、
前記ピクチャを構成する1個以上の画素からなるブロックごとに、時間的に隣接する2枚のピクチャ間での画像の動きを検出し、検出した動きを前記複数枚のピクチャについて連結することで、移動軌跡を算出する動き解析部と、
前記動き解析部で算出された複数の移動軌跡について、移動軌跡間の類似性を表す距離を算出する距離算出部と、
前記距離算出部で算出された距離のうち、予め定められた閾値よりも小さい距離を連結することで、前記距離算出部で算出された距離を測地距離に変換し、得られた測地距離の分布における不連続点を検出し、検出した不連続点よりも小さい測地距離だけ離れた移動軌跡を一つのクラスタとすることによって前記領域分割をする領域分割部と、
前記領域分割部で領域分割された結果を出力する出力部と
を備える移動体検出装置。 - 動画像中の移動体の全部又は一部の領域を特定する領域分割をすることによって動画像中の移動体を検出する移動体検出装置のためのプログラムであって、
請求項1記載の移動体検出方法に含まれるステップをコンピュータに実行させるプログラム。 - 動画像を構成する複数枚のピクチャを受け付ける画像入力ステップと、
前記ピクチャを構成する1個以上の画素からなるブロックごとに、時間的に隣接する2枚のピクチャ間での画像の動きを検出し、検出した動きを前記複数枚のピクチャについて連結することで、移動軌跡を算出する動き解析ステップと、
前記動き解析ステップで算出された複数の移動軌跡について、移動軌跡間の類似性を表す距離を算出する距離算出ステップと、
前記距離算出ステップで算出された距離が小さい順に移動軌跡のペアを同一クラスタとして統合する処理をクラスタ数が規定の数になるまで繰り返すことによって、距離が一定以上離れた不連続点をクラスタの境界として前記領域分割をする領域分割ステップと、
前記領域分割ステップでの領域分割の結果を出力する出力ステップと
を含む移動体検出方法。
Priority Applications (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN2009801118239A CN101983389B (zh) | 2008-10-27 | 2009-09-11 | 移动体检测方法以及移动体检测装置 |
EP09803985.2A EP2330557B1 (en) | 2008-10-27 | 2009-09-11 | Moving body detection method and moving body detection device |
US12/671,793 US8340357B2 (en) | 2008-10-27 | 2009-09-11 | Moving object detection method and moving object detection apparatus |
JP2009549322A JP4456181B1 (ja) | 2008-10-27 | 2009-09-11 | 移動体検出方法及び移動体検出装置 |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2008276157 | 2008-10-27 | ||
JP2008-276157 | 2008-10-27 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2010050110A1 true WO2010050110A1 (ja) | 2010-05-06 |
Family
ID=42128484
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2009/004523 WO2010050110A1 (ja) | 2008-10-27 | 2009-09-11 | 移動体検出方法及び移動体検出装置 |
Country Status (5)
Country | Link |
---|---|
US (1) | US8340357B2 (ja) |
EP (1) | EP2330557B1 (ja) |
JP (1) | JP4456181B1 (ja) |
CN (1) | CN101983389B (ja) |
WO (1) | WO2010050110A1 (ja) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2011080923A1 (ja) * | 2009-12-28 | 2011-07-07 | パナソニック株式会社 | 関節状領域検出装置およびその方法 |
JP2011238139A (ja) * | 2010-05-12 | 2011-11-24 | Fujitsu Ltd | 移動体検出プログラムおよび移動体検出装置 |
WO2012014430A1 (ja) * | 2010-07-27 | 2012-02-02 | パナソニック株式会社 | 移動体検出装置および移動体検出方法 |
US8213681B2 (en) | 2009-01-09 | 2012-07-03 | Panasonic Corporation | Moving object detection method and moving object detection apparatus |
CN102568000A (zh) * | 2010-12-31 | 2012-07-11 | 华晶科技股份有限公司 | 多画面中物体移动的追踪方法 |
WO2012101723A1 (ja) * | 2011-01-26 | 2012-08-02 | パナソニック株式会社 | 関節領域表示装置、関節領域検出装置、関節領域帰属度算出装置、関節状領域帰属度算出装置および関節領域表示方法 |
US8300892B2 (en) | 2009-07-31 | 2012-10-30 | Panasonic Corporation | Moving object detection apparatus and moving object detection method |
US8605946B2 (en) | 2011-03-22 | 2013-12-10 | Panasonic Corporation | Moving object detection apparatus and moving object detection method |
Families Citing this family (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP4782901B2 (ja) * | 2009-12-28 | 2011-09-28 | パナソニック株式会社 | 移動体検出装置および移動体検出方法 |
US20110221664A1 (en) * | 2010-03-11 | 2011-09-15 | Microsoft Corporation | View navigation on mobile device |
JP2011210179A (ja) * | 2010-03-30 | 2011-10-20 | Sony Corp | 移動物体検出装置及び方法、並びにプログラム |
JP5724237B2 (ja) * | 2010-07-27 | 2015-05-27 | オムロンヘルスケア株式会社 | 歩行変化判定装置 |
EP2447882B1 (en) * | 2010-10-29 | 2013-05-15 | Siemens Aktiengesellschaft | Method and device for assigning sources and sinks to routes of individuals |
JP2012151796A (ja) * | 2011-01-21 | 2012-08-09 | Sony Corp | 画像処理装置と画像処理方法およびプログラム |
JP2012212373A (ja) * | 2011-03-31 | 2012-11-01 | Casio Comput Co Ltd | 画像処理装置、画像処理方法及びプログラム |
CN102821241B (zh) * | 2011-06-10 | 2015-06-03 | 华晶科技股份有限公司 | 利用对象移动趋势驱动快门的装置及方法 |
US20130102387A1 (en) * | 2011-10-21 | 2013-04-25 | Microsoft Corporation | Calculating metabolic equivalence with a computing device |
JP2015039085A (ja) * | 2011-12-14 | 2015-02-26 | パナソニック株式会社 | 画像処理装置及び画像処理方法 |
KR20130088493A (ko) * | 2012-01-31 | 2013-08-08 | 삼성전자주식회사 | Ui 제공 방법 및 이를 적용한 영상 수신 장치 |
TWI476702B (zh) * | 2012-03-16 | 2015-03-11 | Pixart Imaging Inc | 使用者辨識系統及辨識使用者的方法 |
KR101901958B1 (ko) * | 2012-03-26 | 2018-11-08 | 한국전자통신연구원 | 휴리스틱 함수의 학습을 이용한 고속 경로를 탐색을 위한 장치 및 그 방법 |
US10140717B2 (en) * | 2013-02-27 | 2018-11-27 | Hitachi Automotive Systems, Ltd. | Imaging apparatus and vehicle controller |
JP6433149B2 (ja) * | 2013-07-30 | 2018-12-05 | キヤノン株式会社 | 姿勢推定装置、姿勢推定方法およびプログラム |
CN104794733B (zh) | 2014-01-20 | 2018-05-08 | 株式会社理光 | 对象跟踪方法和装置 |
CN104881881B (zh) * | 2014-02-27 | 2018-04-10 | 株式会社理光 | 运动对象表示方法及其装置 |
CN105096259B (zh) | 2014-05-09 | 2018-01-09 | 株式会社理光 | 深度图像的深度值恢复方法和系统 |
DE102014209437B3 (de) * | 2014-05-19 | 2015-10-22 | Sirona Dental Systems Gmbh | Verfahren zur Erzeugung magnetresonanztomographischer Aufnahmen von zyklischer Bewegung |
KR20160014418A (ko) | 2014-07-29 | 2016-02-11 | 삼성전자주식회사 | 유저 인터페이스 장치 및 유저 인터페이스 방법 |
US9418546B1 (en) * | 2015-11-16 | 2016-08-16 | Iteris, Inc. | Traffic detection with multiple outputs depending on type of object detected |
US10248839B2 (en) * | 2015-11-30 | 2019-04-02 | Intel Corporation | Locating objects within depth images |
US10346949B1 (en) * | 2016-05-27 | 2019-07-09 | Augmented Pixels, Inc. | Image registration |
US10838067B2 (en) * | 2017-01-17 | 2020-11-17 | Aptiv Technologies Limited | Object detection system |
CN111145100B (zh) * | 2018-11-02 | 2023-01-20 | 深圳富泰宏精密工业有限公司 | 动态影像生成方法及系统、计算机装置、及可读存储介质 |
TWI673653B (zh) * | 2018-11-16 | 2019-10-01 | 財團法人國家實驗研究院 | 移動物體偵測系統及方法 |
US10937169B2 (en) * | 2018-12-18 | 2021-03-02 | Qualcomm Incorporated | Motion-assisted image segmentation and object detection |
TWI739052B (zh) * | 2018-12-26 | 2021-09-11 | 華碩電腦股份有限公司 | 電子裝置及其操作方法 |
CN110135422B (zh) * | 2019-05-20 | 2022-12-13 | 腾讯科技(深圳)有限公司 | 一种密集目标的检测方法和装置 |
CN110132241A (zh) * | 2019-05-31 | 2019-08-16 | 吉林化工学院 | 一种基于时间序列分析的高精度步态识别方法及装置 |
CN110458046B (zh) * | 2019-07-23 | 2020-11-24 | 南京邮电大学 | 一种基于关节点提取的人体运动轨迹分析方法 |
CN110456731B (zh) * | 2019-08-13 | 2022-06-14 | 成都飞机工业(集团)有限责任公司 | 一种数控切削几何刀轨特征提取方法 |
CN111582025B (zh) * | 2020-03-31 | 2023-11-24 | 北京明略软件系统有限公司 | 一种移动物体的识别方法、装置、及存储介质 |
EP3985957B1 (en) * | 2020-10-14 | 2022-11-30 | Axis AB | Method and system for motion segmentation |
CN112288774B (zh) * | 2020-10-22 | 2024-01-30 | 深圳市华宝电子科技有限公司 | 移动侦测方法、装置、电子设备及存储介质 |
TWI779454B (zh) * | 2021-01-08 | 2022-10-01 | 財團法人資訊工業策進會 | 動作辨識裝置及動作辨識方法 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH1166319A (ja) * | 1997-08-21 | 1999-03-09 | Omron Corp | 移動体検出方法及び装置並びに移動体認識方法及び装置並びに人間検出方法及び装置 |
JP2000222584A (ja) * | 1999-01-29 | 2000-08-11 | Toshiba Corp | 映像情報記述方法、映像検索方法及び映像検索装置 |
JP2005332206A (ja) * | 2004-05-20 | 2005-12-02 | Nippon Hoso Kyokai <Nhk> | 映像イベント判別装置及びそのプログラム、並びに、映像イベント判別用学習データ生成装置及びそのプログラム |
JP2007087049A (ja) * | 2005-09-21 | 2007-04-05 | Kddi Corp | 動画像処理装置 |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3780017B2 (ja) | 1994-12-06 | 2006-05-31 | オリンパス株式会社 | 時系列画像解析装置及びその解析方法 |
US7016540B1 (en) * | 1999-11-24 | 2006-03-21 | Nec Corporation | Method and system for segmentation, classification, and summarization of video images |
US6859554B2 (en) * | 2001-04-04 | 2005-02-22 | Mitsubishi Electric Research Laboratories, Inc. | Method for segmenting multi-resolution video objects |
GB0125774D0 (en) * | 2001-10-26 | 2001-12-19 | Cableform Ltd | Method and apparatus for image matching |
US8614741B2 (en) * | 2003-03-31 | 2013-12-24 | Alcatel Lucent | Method and apparatus for intelligent and automatic sensor control using multimedia database system |
US7801330B2 (en) * | 2005-06-24 | 2010-09-21 | Objectvideo, Inc. | Target detection and tracking from video streams |
CN101159855B (zh) * | 2007-11-14 | 2011-04-06 | 南京优科漫科技有限公司 | 基于特征点分析的多目标分离预测方法 |
US8212210B2 (en) * | 2008-02-04 | 2012-07-03 | Flir Systems Ab | IR camera and method for presenting IR information |
JP4763863B1 (ja) * | 2009-12-28 | 2011-08-31 | パナソニック株式会社 | 関節状領域検出装置およびその方法 |
-
2009
- 2009-09-11 JP JP2009549322A patent/JP4456181B1/ja not_active Expired - Fee Related
- 2009-09-11 US US12/671,793 patent/US8340357B2/en not_active Expired - Fee Related
- 2009-09-11 WO PCT/JP2009/004523 patent/WO2010050110A1/ja active Application Filing
- 2009-09-11 EP EP09803985.2A patent/EP2330557B1/en not_active Not-in-force
- 2009-09-11 CN CN2009801118239A patent/CN101983389B/zh not_active Expired - Fee Related
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH1166319A (ja) * | 1997-08-21 | 1999-03-09 | Omron Corp | 移動体検出方法及び装置並びに移動体認識方法及び装置並びに人間検出方法及び装置 |
JP2000222584A (ja) * | 1999-01-29 | 2000-08-11 | Toshiba Corp | 映像情報記述方法、映像検索方法及び映像検索装置 |
JP2005332206A (ja) * | 2004-05-20 | 2005-12-02 | Nippon Hoso Kyokai <Nhk> | 映像イベント判別装置及びそのプログラム、並びに、映像イベント判別用学習データ生成装置及びそのプログラム |
JP2007087049A (ja) * | 2005-09-21 | 2007-04-05 | Kddi Corp | 動画像処理装置 |
Non-Patent Citations (2)
Title |
---|
ANTONINI G ET AL.: "IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY", vol. 16, 1 August 2006, IEEE SERVICE CENTER, article "Counting pedestrians in video sequences using trajectory clustering", pages: 1008 - 1020 |
See also references of EP2330557A4 * |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8213681B2 (en) | 2009-01-09 | 2012-07-03 | Panasonic Corporation | Moving object detection method and moving object detection apparatus |
US8300892B2 (en) | 2009-07-31 | 2012-10-30 | Panasonic Corporation | Moving object detection apparatus and moving object detection method |
JP4763863B1 (ja) * | 2009-12-28 | 2011-08-31 | パナソニック株式会社 | 関節状領域検出装置およびその方法 |
WO2011080923A1 (ja) * | 2009-12-28 | 2011-07-07 | パナソニック株式会社 | 関節状領域検出装置およびその方法 |
US8401243B2 (en) | 2009-12-28 | 2013-03-19 | Panasonic Corporation | Articulated object region detection apparatus and method of the same |
JP2011238139A (ja) * | 2010-05-12 | 2011-11-24 | Fujitsu Ltd | 移動体検出プログラムおよび移動体検出装置 |
US8599252B2 (en) | 2010-07-27 | 2013-12-03 | Panasonic Corporation | Moving object detection apparatus and moving object detection method |
JP4979840B2 (ja) * | 2010-07-27 | 2012-07-18 | パナソニック株式会社 | 移動体検出装置および移動体検出方法 |
WO2012014430A1 (ja) * | 2010-07-27 | 2012-02-02 | パナソニック株式会社 | 移動体検出装置および移動体検出方法 |
CN102568000A (zh) * | 2010-12-31 | 2012-07-11 | 华晶科技股份有限公司 | 多画面中物体移动的追踪方法 |
TWI423170B (zh) * | 2010-12-31 | 2014-01-11 | Altek Corp | 多畫面中物體移動的追蹤方法 |
WO2012101723A1 (ja) * | 2011-01-26 | 2012-08-02 | パナソニック株式会社 | 関節領域表示装置、関節領域検出装置、関節領域帰属度算出装置、関節状領域帰属度算出装置および関節領域表示方法 |
JP4994525B1 (ja) * | 2011-01-26 | 2012-08-08 | パナソニック株式会社 | 関節領域表示装置、関節領域検出装置、関節領域帰属度算出装置、関節状領域帰属度算出装置および関節領域表示方法 |
CN103003846A (zh) * | 2011-01-26 | 2013-03-27 | 松下电器产业株式会社 | 关节区域显示装置、关节区域检测装置、关节区域归属度计算装置、关节状区域归属度计算装置以及关节区域显示方法 |
US8929599B2 (en) | 2011-01-26 | 2015-01-06 | Panasonic Corporation | Articulation region display apparatus, articulation region detecting apparatus, articulation region belongingness calculating apparatus, articulated object region belongingness calculating apparatus, and articulation region display method |
US8605946B2 (en) | 2011-03-22 | 2013-12-10 | Panasonic Corporation | Moving object detection apparatus and moving object detection method |
Also Published As
Publication number | Publication date |
---|---|
EP2330557B1 (en) | 2018-11-07 |
US8340357B2 (en) | 2012-12-25 |
JP4456181B1 (ja) | 2010-04-28 |
US20110228987A1 (en) | 2011-09-22 |
EP2330557A4 (en) | 2013-07-24 |
CN101983389A (zh) | 2011-03-02 |
EP2330557A1 (en) | 2011-06-08 |
JPWO2010050110A1 (ja) | 2012-03-29 |
CN101983389B (zh) | 2012-11-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP4456181B1 (ja) | 移動体検出方法及び移動体検出装置 | |
JP5054206B2 (ja) | 画像処理方法及び画像処理装置 | |
JP4542207B1 (ja) | 移動体検出方法および移動体検出装置 | |
JP4643766B1 (ja) | 移動体検出装置及び移動体検出方法 | |
JP4782901B2 (ja) | 移動体検出装置および移動体検出方法 | |
JP4994525B1 (ja) | 関節領域表示装置、関節領域検出装置、関節領域帰属度算出装置、関節状領域帰属度算出装置および関節領域表示方法 | |
JP5404918B2 (ja) | 移動軌跡算出方法およびその装置、ならびに領域分割方法 | |
JP4979840B2 (ja) | 移動体検出装置および移動体検出方法 | |
JP4668360B2 (ja) | 移動体検出方法及び移動体検出装置 | |
JPWO2011080923A1 (ja) | 関節状領域検出装置およびその方法 | |
Luo et al. | Face tracking in video with hybrid of Lucas-Kanade and condensation algorithm |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 200980111823.9 Country of ref document: CN |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2009549322 Country of ref document: JP |
|
WWE | Wipo information: entry into national phase |
Ref document number: 12671793 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2009803985 Country of ref document: EP |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 09803985 Country of ref document: EP Kind code of ref document: A1 |
|
NENP | Non-entry into the national phase |
Ref country code: DE |