JP2015049702A - Object recognition device, object recognition method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable the reduction of the occurrence of false recognition of an object.SOLUTION: An object recognition device includes storage means for storing a collection of locus patterns of points that constitutes a recognition object as a standard pattern, point locus acquisition means for acquiring the loci of the points on an image, and locus pattern correlation optimization means for selecting a collection of the loci of the points on the basis of the similarity between the loci of the points acquired by the point locus acquisition means and the standard pattern, and extracting the selected collection of the loci of the points as a recognition object on the image.

Description

  The present invention relates to a technique for recognizing a target such as a moving object.

  Various computer processing methods have been proposed as a method for recognizing a specific object from a moving image sequence acquired from a camera. For example, there are methods described in Patent Documents 1 and 2 as methods for detecting an object of a specific category such as a person, a vehicle, and a face from an image. The method described in Patent Document 1 classifies a plurality of movement trajectories into subclasses, calculates a share ratio of the movement trajectory among the subclasses, and derives a subclass similarity. In this method, the subclasses are classified into classes so that a collection of subclasses having higher similarity between subclasses is classified into the same class. In this way, this method can identify blocks corresponding to moving loci included in the same class as moving object areas and detect moving objects in a moving image. Patent Document 2 describes an example of a mobile object detection method that can correctly extract a region without being affected by changes in the shape and size of the mobile object and the influence of shielding. Patent Document 2 describes that region selection candidates are selected by using temporal changes in geodetic distances between moving trajectories or temporal changes in inter-regional geodetic distances of inter-region division candidates in region division candidate selection. Has been.

Patent 5102410 Re-specialized WO2010 / 079556

  However, in the above-described technology, under the condition that the background image always changes as the camera itself moves, such as an in-vehicle camera image, the recognition target may be erroneously recognized (error) due to noise or movement of the background object image. Detection, including detection omission).

  In order to achieve such an object, an aspect of the present invention is an object recognition apparatus, which stores storage trajectories of a set to which trajectory patterns of points constituting a recognition target belong as reference patterns, and points on an image. A point trajectory acquisition means for acquiring a trajectory; a point trajectory set is selected based on the similarity between the point trajectory acquired by the point trajectory acquisition means and the reference pattern; and the selected point trajectory set is recognized on the image Orbital pattern correlation optimization means for extracting as a target.

  In addition, the present invention provides a point trajectory acquisition step for acquiring a trajectory of a point on an image in a computer including storage means for storing a trajectory of a set to which a trajectory pattern of points constituting a recognition target belongs as a reference pattern. An object recognition program for executing a trajectory pattern correlation optimization step of selecting a set of point trajectories based on similarity between a point trajectory and the reference pattern, and extracting the selected point trajectory set as a recognition target on the image; I will provide a.

  In addition, the present invention stores a trajectory of a set to which a trajectory pattern of points constituting a recognition target belongs as a reference pattern, acquires a trajectory of a point on an image, and determines the similarity between the acquired point trajectory and the reference pattern. An object recognition method for selecting a set of point trajectories based on the selected point trajectory and extracting the selected point trajectory set as a recognition target on the image is provided.

  According to the present invention, occurrence of erroneous recognition of an object can be reduced.

It is a figure which shows the example of the physical structure of a moving object recognition system. It is a figure which shows the example of a structure of the function part of a moving object recognition system. It is a figure which shows the example of the flow of learning of the norm pattern of a trajectory correlation pattern optimization part. It is a figure which shows the example of the feature-value expression of the pattern used in a trajectory correlation pattern optimization part. It is a figure which shows the example of the flow of the feature acquisition from the input data in a trajectory correlation pattern optimization part. It is a figure explaining the example of the effect in a trajectory correlation pattern optimization part. It is a figure which shows the example of the flow of a process of a moving object recognition system. It is a figure which shows the example of the flow of the process of optimization based on an orbital correlation pattern. It is a figure explaining the problem in the detection by a general discriminator. It is a figure which shows the example of a structure of a moving object recognition system. It is a figure which shows the example of a structure of a moving object recognition system. It is a figure which shows the example of a structure of a moving object recognition system.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and description thereof will be omitted as appropriate. In addition, each part which comprises the apparatus in the following description is comprised by hardware, such as a logic circuit. Each unit includes a computer control unit, a memory, a program loaded in the memory, a storage unit such as a hard disk for storing the program, a network connection interface, and the like, and may be realized by any combination of hardware and software. Good. Further, unless otherwise specified, the realization method and apparatus are not limited.

<Embodiment 1>
"Constitution"
FIG. 1 is a block diagram showing a configuration example of a moving object recognition system 1 according to an embodiment of the present invention. Referring to FIG. 1, the moving object recognition system 1 includes an image input device 20, an electronic control device 10, and a terminal device 30.

  The image input device 20 is a device that inputs image information to the electronic control device 10. The image input device 20 is an imaging device such as a camera device. The camera device is an apparatus that captures images in real time, and continuously acquires images to be captured. The imaging device is, for example, a video camera that outputs an NTSC (National Television Standards Committee) format or a PAL (Phase Alternating Line) format. The image input device 20 may be an image capture device that reads image information stored in a storage medium, converts it into an NTSC output, PAL output, or other image format that can be read by the electronic control device 10 and outputs the image format. . The image input device 20 in this case can also be realized as a software program that operates inside a CPU (Central Processing Unit) 11 of the electronic control device 10. The image input device 20 is not limited to a fixed camera, and may be a moving camera. The present invention can be applied to a wide range of uses such as a monitoring system using a pan / tilt camera, driver support by a vehicle-mounted camera, environment recognition of a mobile robot, and the like.

  The terminal device 30 operates as a user interface for operating the electronic control device 10 and monitoring the internal state and output of the electronic control device 10. The terminal device 30 is a display or the like that presents an input image, a recognition area, a symbol indicating a registered list of recognition targets, and the like. Further, the terminal device 30 is an input device (for example, a switch board, keyboard, mouse, etc.) for inputting commands to the electronic control device 10 such as start / end of image recognition, designation of a recognition target, selection of display presentation information, and the like. Touch panel, etc.).

  The electronic control device 10 is a computer that performs information processing for recognizing an object in an image. The electronic control device 10 specifies a region of an object (pedestrian, preceding vehicle, etc.) belonging to a specific category as a recognition target in the image based on a predetermined program in accordance with image information sent from the image input device 20. Information processing is performed.

  The electronic control device 10 includes a central processing unit (CPU) 11, a storage device (Mem) 12, a storage device (DB: database) 13, and interfaces (I / F) 14 and 15. In FIG. 1, the storage device (Mem) 12 and the storage device (DB) 13 are described separately, but these may be realized as one storage device.

  The interface 14 is a device that mediates exchange of information among the central processing unit 11, the storage device (Mem) 12, the storage device (DB) 13, and the image input device 20. In FIG. 1, the interface 14 is connected to the image input device 20 and the central processing unit 11. The interface 14 may be directly connected to the storage device (Mem) 12 and the storage device (DB) 13.

  The interface 15 passes the result of information processing performed inside the electronic control device 10 to the terminal device 30 connected to the outside, or receives a command input from the terminal device 30 to the electronic control device 10. Serves as an information broker.

The storage device (Mem) 12 is a device that stores temporary data, and is electrically connected to the central processing unit 11. The storage device (DB) 13 is a device that mainly stores a database (DB), and is electrically connected to the central processing unit 11. In FIG. 1, the storage device (Mem) 12 and the storage device (DB) 13 are built in the electronic control device 10.
These devices may be external storage devices.

  The central processing unit 11 is a device that performs information processing. The central processing unit 11 is electrically connected to the interfaces 14 and 15, the storage device (Mem) 12, and the storage device (DB) 13. The central processing unit 11 refers to the information stored in the storage device (Mem) 12 and the storage device (DB) 13 on the basis of the image information input from the image input device 20 via the interface 14. Information processing is performed to detect an image area such as an object.

  Here, the detection is to identify and identify an area of an object (pedestrian, preceding vehicle, etc.) belonging to a specific category from other areas on the input image. Further, tracking is to acquire image areas of the same individual such as objects belonging to a specific category in association with each other in time series. Recognition is associating data with meaning. Specifically, the recognition means that an application associates a target for detection or tracking with an image region. Recognition includes object detection and tracking.

  In the present embodiment, pedestrians are mainly examples of recognition targets. However, the present invention does not limit the recognition target to pedestrians. The present invention provides an effective technique for recognizing a multi-joint object that moves while accompanying a change in posture by one or more typical and periodic patterns determined to some extent.

  FIG. 2 is a block diagram showing an example of the entire configuration of the moving object recognition system 1 in the present embodiment.

  The electronic control device 10 implements the various functions shown in FIG. 2 by executing a software program in the central processing unit 11. Various functions realized in the electronic control device 10 may be realized as individual devices, function units, or electronic circuits.

  Referring to FIG. 2, the electronic control apparatus 10 in the present embodiment includes a detection sequence acquisition unit 103, a point trajectory acquisition unit 102, a trajectory set estimation unit 104, and a trajectory pattern correlation optimization unit 105. Each of these functions generally operates as follows.

  The detection sequence acquisition unit 103 selects a region (Region of interest, ROI, candidate region) (see FIG. 3) as a candidate image region of a category (pedestrian or the like) as a recognition target for the input image received at each time. Identify from background image. Then, the detection series acquisition unit 103 outputs a numerical value of reliability indicating the position, scale, and probability of the candidate area. As a method of realizing the detection sequence acquisition unit 103, there is a method of using a discriminator by general statistical learning. For example, a statistical classifier such as SVM (Support Vector Machine) or AdaBoost (Adaptive Boosting) using Haar-like features, HoG (Histograms of Oriented Gradients) features, or the like can be used as the feature amount.

  The detection sequence acquisition unit 103 can output only a region having a reliability equal to or higher than the threshold by setting a threshold (threshCandVal1) to the numerical value of the reliability. In addition, when a candidate exceeding a threshold (maxCandNum) that determines the maximum number of candidates to be output is detected, the detection sequence acquisition unit 103 has a higher reliability threshold (threshCandVal1) than that of maxCandNum (threshCandVal1). By resetting threshCandVal2), the maximum number of candidates that can be output can be limited. In order to avoid detection omission as much as possible, the detection sequence acquisition unit 103 may allow overlap of ROIs as shown in the right end of FIG. 9 and may set threshCandVal1 so as to acquire as many candidates as computational resources allow.

  The detection sequence acquisition unit 103 may operate as a real-time discriminator that sequentially acquires detection outputs for input images received at each time and adds them to a sequence of output results that are held. In addition, the detection sequence acquisition unit 103 responds to requests from other functional units only from past data including a predetermined time on the time axis defined by the program from the storage area holding the detection sequence acquired in advance. May operate as a data output device as provided. The detection sequence acquisition unit 103 holds an output result from the predetermined time on the time axis defined by the program to the past, and can arbitrarily read the result sequence.

  The point trajectory acquisition unit 102 acquires a time-space trajectory of a point (pixel) on an image from an image sequence, stores it, and arbitrarily outputs a functional unit, or can read preliminarily stored point trajectory data. It is a functional part. The point is not limited to one pixel but may be a region where a plurality of pixels are collected. For example, the points are acquired in a range in which one or more points are included in the image area to be recognized in the image space. For example, the point trajectory is acquired at such a density that it is statistically guaranteed that one or more point trajectories are acquired for each part that may operate independently in the image region to be recognized.

A point trajectory is a past (T) frame of a point p _i (t) (i = 1, 2,..., M: number of acquired point trajectories) at a predetermined time (t) on the time axis defined by the program. , Tr _i = {p _i (t _k )}, t _k ∈ [t−T, t], or a set of vectors connecting corresponding points, tr _i = {vec _i (t _k ) }, T _k ∈ [t−T−1, t]. A specific method for acquiring a point trajectory is a method based on an optical flow.

The trajectory set estimation unit 104 associates point trajectories (tr _i , i = 1, 2,..., N) that can be acquired at a predetermined time (t) with a specific recognition target. At this time, the trajectory set estimation unit 104 divides the point trajectory into sets (performs segmentation). For example, the trajectory set estimation unit 104 performs segmentation of the point trajectory set based on the trajectory similarity based on the spatial distance between the point trajectories and the geometric similarity of the trajectories. Further, for example, the trajectory set estimation unit 104 sets the point trajectories so that the total sum of trajectory similarity is maximized on the condition that the point trajectories pass through the same detection area in the acquired detection series. Are segmented as a set of point trajectories for each similar trajectory.

For example, the trajectory similarity for two trajectories {p, q} εS included in the set S of the entire point trajectories is defined as w _pq . In addition, the detection trajectory correlation value defined by the sum total weighted by the detection reliability in the detection result is defined as n _pq as the number of times the point trajectory passes through the same detection region acquired in the interval t _k ∈ [tT, t]. . At this time, the estimation of the trajectory set {S ₁ , S _2, ...} Based on the similarity of trajectories is obtained by optimizing the selection of the set that maximizes the sum of the following objective functions. Here, since w _pq and n _pq are defined by a known method, they will not be described in detail.
(Formula 1)

Here, λ is a weighting coefficient, and S ₁ ∪S ₂ ∪S ₃ ... S, S _i ∩S _j = φ, i ≠ j. A known method may be used as a specific implementation method.

  The trajectory pattern correlation optimization unit 105 performs optimization of the point trajectory set acquired by the trajectory set estimation unit 104. That is, the trajectory pattern correlation optimization unit 105 reintegrates point trajectory sets estimated as separate sets due to relative differences in trajectories even though they are point trajectories corresponding to the same recognition target region. I do. For example, the trajectory set estimation unit 104, when the relative motion between the trunk and legs of a pedestrian moving in the lateral direction is large, the trajectories of the two are greatly different, so the same walking that is the same recognition target There is a possibility that an over segmentation problem occurs in which point trajectory groups corresponding to a person's trunk and legs are segmented as different sets. The trajectory pattern correlation optimization unit 105 solves this problem.

  The trajectory pattern correlation optimization unit 105 uses a reference pattern which is a set of point trajectory patterns in a typical action of a recognition target stored in advance, and uses a combination of trajectory sets acquired by the trajectory set estimation unit 104 and a reference pattern. Calculate the correlation. When the correlation optimization unit 105 selects a combination of trajectory sets that maximizes the sum of correlations, the estimation of the point trajectory set of the recognition target area based on the motion pattern model specific to the recognition target is optimized. The As a result, at the stage of the trajectory set estimation unit 104, an effect of correctly recognizing the entire same recognition target region that is erroneously estimated as another region due to a difference in trajectory can be expected.

  A specific operation of the trajectory pattern correlation optimization unit 105 will be described.

(Learning phase)
The trajectory pattern correlation optimization unit 105 first creates a reference pattern in a typical operation to be recognized in advance and stores it in, for example, the storage device (DB) 13. For example, referring to FIG. 3, the trajectory pattern correlation optimization unit 105 extracts a point trajectory for a sample motion (Sample Motion 1), and generates a set of point trajectories on the recognition target region based on the similarity of the trajectories. (Segmentation). The point trajectory extraction method and the segmentation based on the trajectory similarity of the point trajectories may be the same method as the trajectory set estimation unit 104. However, the trajectory pattern correlation optimization unit 105 does not necessarily need to use the result of the detection sequence, and may use the result of known correctly recognized recognition target region data.

The trajectory pattern correlation optimization unit 105 selects representative trajectories (eg, center of gravity trajectories) from the set of point trajectories S ₁ , S ₂ , S ₃ ,... Segmented according to the trajectory similarity, and each trajectory. Relative values of each other are stored in the storage device (DB) 13 as a reference pattern of the sample operation 1. For example, as shown in FIG. 4, the relative value that is an element of the reference pattern is the difference between the trajectories as the relative value of (i → j) in the two trajectories (i, j).
(Formula 2)
And the area ratio of the orbital set
(Formula 3)
Using,
(Formula 4)
Can be defined. Here, k is an index of the type of motion sample, [tT, t] is a section in which a trajectory is acquired, and p is a point of an element that generates a trajectory. The section is set so as to include one cycle of the operation, for example. Using this, a reference pattern MP _{k of} operation _k is defined.
(Formula 5)
Here, S is the set of the entire point trajectory, and ind (S) is the entire index of the subset of the point trajectory set. Interchanged relative relationship of the track the (i, j) th component and (j, i) elements one may be stored either because it is equivalent. Both can be converted by the following equation.
(Formula 6)

(Recognition phase)
Next, recognition using the learned norm pattern will be described. First, segmentation of the point trajectory set extracted from the input image is executed by the processing of the trajectory set estimation unit 104. In FIG. 5, the point trajectory is associated with the areas S ₁ , S ₂ , S ₃ , and S ₄ . S ₁ , S ₂ , and S ₃ indicate that the same pedestrian area is divided into three. The trajectory pattern correlation optimization unit 105 can define a test pattern as follows for each divided set, as in the case of the reference pattern.
(Formula 7)
Here, since the (i, j) element and the (j, i) element in which the relative relations of the trajectories are interchanged are equivalent, the trajectory pattern correlation optimization unit 105 acquires only one of them and compares it with the reference pattern. In this case, it is possible to exclusively select only the higher correlation by appropriately converting.

The test pattern has a degree of freedom of scale and period with respect to the normative pattern. Therefore, the trajectory pattern correlation optimization unit 105 sets the scale parameter α and the period parameter β when associating the test pattern with the reference pattern. The scale parameter α is a parameter for enlarging or reducing the value of the trajectory difference. The cycle parameter β is a parameter that determines a test pattern section used for collation. Data in the selected interval is interpolated for normalization to align the length of the reference pattern with the interval in the time direction. For example, the elements of the test pattern including the scale parameter α and the period parameter β are defined as follows.
(Formula 8)
Using this, the correlation indicating the similarity between the (i, j) element of the test pattern and the (m, n) element of the reference pattern can be defined as follows, for example.
(Formula 9)
This formula is an index for evaluating the similarity between an element of the input test pattern and one element of the reference pattern of the motion k between 0 and 1 after a certain scale α and period β are set. Yes.

  In addition to the relative trajectory pattern, the criterion for measuring the similarity takes into account the proximity of the area ratio of the corresponding image area of the pair of point trajectory sets. γ and δ are weighting factors for comparing the relative trajectory pattern and the area ratio, respectively.

  The scale α is a scale ratio with respect to the case where the scale of the reference pattern is 1, and may be an infinite continuous amount. However, in reality, the range of α is limited by defining the minimum size and the maximum size that can be taken as a recognition target by the application, and α becomes a discrete amount by quantizing the section.

  The period β is a ratio when the period of the reference pattern is set to 1, and indicates that an operation that is β times faster (or slower) than the period of the reference pattern is allowed as the same operation.

  k is the number of motion types stored as a reference pattern, and is a finite natural number. For example, rigid body movement (when each part moves in the same orbit, including a stationary state), walking right (walking right with respect to the camera), walking left (walking left with respect to the camera) The value of k is determined by the number of motion patterns that are assumed to be recognized by applications such as right running (running to the right with respect to the camera) and left running (running to the left with respect to the camera). .

Furthermore, the detected trajectory correlation value defined by the sum of weights of the trajectories in the test pattern that have passed through the same detection area weighted by the detection reliability in each detection result is n _Sij , the weight coefficient is λ _S , and the objective function Is defined as follows, for example.
(Formula 10)
However, S _e ′ is a subset arbitrarily selected from the elements of the set of regions formed by the trajectory set {S _p } = {S ₁ , S ₂ ,. {I, j} are test pattern elements determined by S _e ′, and α, β, k, m, and n are determined so as to maximize the objective function under the determined S _e ′. . The trajectory pattern correlation optimization unit 105
(Formula 11)
So that the optimized subset is reselected from the original trajectory set S. However, X represents the combination of selection of {S _e '}. For example, when the region composed of the trajectory set estimated by the trajectory set estimation unit 104 is S ₁ , S ₂ , S ₃ , S _{4 as} shown in the left of FIG. 6, a combination of exclusive selection of the entire subset As one of the combinations shown in the right of FIG.
(Formula 12)
Exists. X is a variable that represents one of the possible combinations of selections, and the possible values of X are all cases of exclusive selection combinations of subsets.

  The definition of the objective function is not limited to (Formula 10). That is, the definition of the objective function may be a definition that can quantitatively evaluate the optimum association between the point trajectory and the recognition target area. For example, the definition of the objective function may be a definition that can quantitatively evaluate the optimum association between the point trajectory and the recognition target region using the similarity between the trajectories in the trajectory set estimation unit 104 and the criterion based on the detection sequence. For example, the objective function is defined as the optimal correspondence between the point trajectory and the recognition target region using the similarity of the pattern learned in advance based on the relative relationship between the trajectory sets in the trajectory pattern correlation optimization unit 105 and the criterion based on the detection sequence. It may be a definition that can be quantitatively evaluated. A specific method of optimization may be various general nonlinear optimizations in addition to exhaustive search.

(Operation)
The operation of the moving object recognition system 1 in the present embodiment will be described with reference to the drawings. Referring to FIG. 7, first, the detection sequence acquisition unit 103 acquires a sequence of results of detecting ROIs that are candidates for the target object region (Step 1).

  Next, the point trajectory acquisition unit 102 acquires a point trajectory that is a trajectory in time and space of the point on the image (Step 2).

Next, as described above, the trajectory set estimation unit 104 performs segmentation of the point trajectory set based on the trajectory similarity and the detected trajectory correlation value based on n _pq or the like (Step 3).

  Next, the trajectory pattern correlation optimization unit 105 executes re-optimization of point trajectory set selection based on the correlation with the pre-stored reference pattern (Step 4).

  FIG. 8 is a flowchart showing an example of the operation in Step 4. Referring to FIG. 8, first, the trajectory pattern correlation optimization unit 105 reselects a combination from the acquired point trajectory set (Step 401). The trajectory pattern correlation optimization unit 105 calculates an objective function (Formula 11) in the selection (Step 402). At this time, in each of the selected combinations, optimization calculation for adjusting the parameters (k, α, β) so that the objective function takes the maximum value is performed. Step 401 and Step 402 are repeatedly calculated, and finally a combination of point trajectory sets that maximizes the objective function is determined (Step 403).

For example, as shown in FIG. 6, when the trajectory set estimation unit 104 divides the point trajectory set into S ₁ , S ₂ , S ₃ , and S ₄ , S ₁ , S ₂ , and S ₃ correspond to the reference pattern in FIG. An example is the case of the operation to be performed. In this case, it can be expected that the objective function is maximized when S ₁ ′ = {S ₁ , S ₂ , S ₃ } is selected as a set of trajectory sets in the objective function for the reference pattern 1. Because other relative orbital and S ₄ are significantly different from the pattern that is included in the Code pattern, S ₄ the objective function can be expected to be maximized when selected as S ₂ '= {S _4} alone. By re-selecting the region association by such optimization, it can be expected that the target region is correctly recognized as shown in the right diagram of FIG.

  In the above, the norm pattern is defined based on the difference between orbits as the relative value of (i → j) in the two orbits (i, j), but the norm pattern must be defined by the difference between the orbits. Instead, it may be defined by a set of point trajectories themselves.

  As described above, the moving object recognition system 1 according to the present embodiment correlates the reference pattern stored in advance with respect to the distance and similarity of the point trajectory and the association of the region with the recognition target using the detection sequence. The re-selection of association is performed by taking into account optimization. Thereby, it becomes possible to correctly associate the regions of the entire recognition target by suppressing an association error due to a relative difference in motion.

<Embodiment 2>
The moving object recognition system 1 according to the present embodiment is configured to realize recognition target area association by optimizing the correlation between a point trajectory and a pre-stored reference pattern. The moving object recognition system 1 in the present embodiment includes a point trajectory acquisition unit 102 and a trajectory pattern correlation optimization unit 105 (see FIG. 10). Unlike the first embodiment, in this embodiment, the step of treating similar trajectories as a set in advance, like the trajectory set estimation unit 104 in FIG. 2, is omitted, and the trajectory pattern is a direct relative of arbitrary trajectories. Described by relationships.

  However, the area of the area is acquired as the area of the representative area at the time of acquisition of the representative trajectory by averaging the point trajectories in the appropriate area when the arbitrary trajectory is acquired. In this case, the same expression as in (Formula 4) can be used. When the area at the time of obtaining the representative trajectory is constant in any trajectory (including the case where the point trajectory itself is used instead of the representative trajectory), the area area ratio value can be omitted.

  The trajectory pattern correlation optimizing unit 105 selects a combination of point trajectories having the highest correlation with a pre-stored reference pattern and performs optimization for associating with a recognition target region.

  As described above, the object recognition system 1 according to the present embodiment optimizes the region division based on the correlation between the reference pattern stored in advance and the point trajectory. Thereby, it becomes possible to make it correspond as an area | region of the whole pedestrian, without recognizing the area | region containing the leg part which produces a big relative motion with respect to a trunk part like the pedestrian crossing in front of a camera. As a result, it is possible to recognize the whole body region robustly against changes in appearance due to the movement of the recognition target.

<Embodiment 3>
The moving object recognition system 1 according to the present embodiment is configured to realize recognition target area association by optimizing correlations between point trajectories, detection sequences, and pre-stored reference patterns. In the moving object recognition system 1 in the present embodiment, a detection sequence acquisition unit 103 is added to the second embodiment. That is, the trajectory pattern correlation optimization unit 105 optimizes the objective function so as to add the correlation between the point trajectories that have passed through the same detection rectangle in the detection sequence in addition to the correlation with the reference pattern stored in advance. The recognition target areas are associated with each other (see FIG. 11).

  The trajectory pattern correlation optimization unit 105 of the present embodiment (see FIG. 11) defines an objective function using three criteria based on the similarity between trajectories, the correlation with the detection sequence, and the similarity with the reference pattern. , The recognition area correspondence is optimized.

  According to the present embodiment, the association by the optimization considering the correlation with the pattern stored in advance is performed on the similarity of the point trajectory and the association of the region using the detection sequence to the recognition target. Thereby, it becomes possible to correctly associate the regions of the entire recognition target by suppressing an association error due to a relative difference in motion.

<Embodiment 4>
The moving object recognition system 1 in the present embodiment includes a point trajectory acquisition unit 102, a trajectory set estimation unit 104, and a trajectory pattern correlation optimization unit 105 (see FIG. 12). The trajectory set estimation unit 104 associates the point trajectory with a specific recognition target. For example, the trajectory set estimation unit 104 performs segmentation of the point trajectory set based on the trajectory similarity based on the spatial distance between the point trajectories and the geometric similarity of the trajectories. This embodiment corresponds to the case where the second term on the right side is 0 in the objective function of (Equation 1) because the detection sequence acquisition unit 103 does not exist.

  As described above, the moving object recognition system 1 according to the present embodiment deals with the optimization by adding the correlation to the pre-stored reference pattern to the correspondence of the region to the recognition target using the similarity of the point trajectory. A reselection is made. Thereby, it becomes possible to correctly associate the regions of the entire recognition target by suppressing an association error due to a relative difference in motion.

  While the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

  Moreover, although embodiment of this invention is described as follows, it is not limited to the following description.

(Appendix)
(Appendix 1)
Storage means for storing a trajectory of a set to which a trajectory pattern of points constituting a recognition target belongs, as a reference pattern;
Point trajectory acquisition means for acquiring a trajectory of a point on the image;
Trajectory pattern correlation optimizing means for selecting a set of point trajectories based on the similarity between the point trajectory acquired by the point trajectory acquiring means and the reference pattern, and extracting the selected point trajectory set as a recognition target on the image When,
An object recognition device including:

(Appendix 2)
The reference pattern includes a region area ratio of a set to which the orbital pattern belongs,
The object recognition apparatus according to appendix 1, wherein the trajectory pattern correlation optimization unit selects the point trajectory set based on the similarity of the area ratio.

(Appendix 3)
The object recognition device according to any one of appendices 1 to 2, wherein the trajectory pattern included in the reference pattern is configured by a difference in spatio-temporal coordinates between trajectories in a combination of point trajectory pairs included in the trajectory pattern.

(Appendix 4)
Further comprising detection sequence acquisition means for acquiring image region candidates in which the recognition target exists over a plurality of times;
The trajectory pattern correlation optimizing unit calculates a correlation between point trajectories based on the number of times the point trajectory passes through the same detection area acquired by the detection sequence acquiring unit, and based on the correlation, the point trajectory 4. The object recognition device according to any one of supplementary notes 1 to 3, wherein a set is selected.

(Appendix 5)
A point trajectory set estimation means for calculating a correlation between a plurality of point trajectories acquired from the point trajectory acquisition means, and selecting the point trajectory set based on the correlation,
The trajectory pattern correlation optimization unit selects a point trajectory set based on the similarity between the point trajectory set selected by the point trajectory set estimation unit and the reference pattern, and the selected point trajectory set is displayed on the image. 5. The object recognition device according to any one of appendices 1 to 4, which is extracted as a recognition object.

(Appendix 6)
In a computer comprising storage means for storing a trajectory of a set to which a trajectory pattern of points constituting a recognition target belongs, as a reference pattern,
A point trajectory acquisition step for acquiring a trajectory of a point on the image;
A trajectory pattern correlation optimization step of selecting a set of point trajectories based on the similarity between the acquired point trajectory and the reference pattern, and extracting the selected point trajectory set as a recognition target on the image;
Object recognition program that executes

(Appendix 7)
The reference pattern includes a region area ratio of a set to which the orbital pattern belongs,
The object recognition program according to appendix 6, wherein in the trajectory pattern correlation optimization step, the point trajectory set is selected based on a similarity of the area ratio.

(Appendix 8)
The object recognition program according to any one of claims 6 to 7, wherein the trajectory pattern included in the reference pattern includes a difference in spatio-temporal coordinates between trajectories in a combination of point trajectory pairs included in the trajectory pattern. .

(Appendix 9)
Further comprising a detection sequence acquisition step of acquiring image region candidates in which the recognition target exists over a plurality of times;
In the trajectory pattern correlation optimization step, the correlation between the point trajectories is calculated based on the number of times the point trajectories have passed through the same detection area acquired by the detection sequence acquisition step, and the point trajectory is calculated based on the correlation. 9. The object recognition program according to any one of appendices 6 to 8 for selecting a set.

(Appendix 10)
Calculating a correlation between a plurality of point trajectories acquired in the point trajectory acquisition step, further including a point trajectory set estimation step of selecting the point trajectory set based on the correlation;
In the trajectory pattern correlation optimization step, a point trajectory set is selected based on the similarity between the point trajectory set selected in the point trajectory set estimation step and the reference pattern, and the selected point trajectory set is displayed on the image. The object recognition program according to any one of appendices 6 to 9, which is extracted as a recognition object.

(Appendix 11)
The trajectory of the set to which the trajectory pattern of the points constituting the recognition target belongs is stored as a reference pattern,
Get the trajectory of the point on the image,
An object recognition method for selecting a set of point trajectories based on similarity between an acquired point trajectory and the reference pattern, and extracting the selected point trajectory set as a recognition target on the image.

(Appendix 12)
The reference pattern includes a region area ratio of a set to which the orbital pattern belongs,
The object recognition method according to appendix 11, wherein the point trajectory set is selected based on a similarity of the area ratio.

(Appendix 13)
The object recognition method according to any one of appendices 11 to 12, wherein the trajectory pattern included in the reference pattern includes a difference in spatio-temporal coordinates between trajectories in a combination of point trajectory pairs included in the trajectory pattern.

(Appendix 14)
Obtaining a candidate image region where the recognition target exists over a plurality of times;
The correlation between point trajectories is calculated based on the number of times the point trajectories have passed through the same acquired detection area, and the point trajectory set is selected based on the correlations. Object recognition method.

(Appendix 15)
Calculate the correlation between the acquired plurality of point trajectories, select the point trajectory set based on the correlation,
The point trajectory set is selected based on the similarity between the selected point trajectory set and the reference pattern, and the selected point trajectory set is extracted as a recognition target on the image. Object recognition method.

DESCRIPTION OF SYMBOLS 1 Moving object recognition system 10 Electronic controller 11 Central processing unit 12 Memory | storage device (Mem)
13 Storage device (DB)
DESCRIPTION OF SYMBOLS 14 Interface 15 Interface 20 Image input device 30 Terminal device 102 Point orbit acquisition part 103 Detection sequence acquisition part 104 Orbit set estimation part 105 Orbit pattern correlation optimization part

Claims (10)

Storage means for storing a trajectory of a set to which a trajectory pattern of points constituting a recognition target belongs, as a reference pattern;
Point trajectory acquisition means for acquiring a trajectory of a point on the image;
Trajectory pattern correlation optimizing means for selecting a set of point trajectories based on the similarity between the point trajectory acquired by the point trajectory acquiring means and the reference pattern, and extracting the selected point trajectory set as a recognition target on the image When,
An object recognition device including: The reference pattern includes a region area ratio of a set to which the orbital pattern belongs,
The object recognition apparatus according to claim 1, wherein the trajectory pattern correlation optimization unit selects the point trajectory set based on a similarity of the area ratio.   The object recognition device according to claim 1, wherein the trajectory pattern included in the reference pattern includes a difference in spatio-temporal coordinates between trajectories in a combination of point trajectory pairs included in the trajectory pattern. . Further comprising detection sequence acquisition means for acquiring image region candidates in which the recognition target exists over a plurality of times;
The trajectory pattern correlation optimizing unit calculates a correlation between point trajectories based on the number of times the point trajectory passes through the same detection area acquired by the detection sequence acquiring unit, and based on the correlation, the point trajectory The object recognition apparatus according to claim 1, wherein a set is selected. A point trajectory set estimation means for calculating a correlation between a plurality of point trajectories acquired from the point trajectory acquisition means, and selecting the point trajectory set based on the correlation,
The trajectory pattern correlation optimization unit selects a point trajectory set based on the similarity between the point trajectory set selected by the point trajectory set estimation unit and the reference pattern, and the selected point trajectory set is displayed on the image. The object recognition device according to claim 1, wherein the object recognition device is extracted as a recognition object. In a computer comprising storage means for storing a trajectory of a set to which a trajectory pattern of points constituting a recognition target belongs, as a reference pattern,
A point trajectory acquisition step for acquiring a trajectory of a point on the image;
A trajectory pattern correlation optimization step of selecting a set of point trajectories based on the similarity between the acquired point trajectory and the reference pattern, and extracting the selected point trajectory set as a recognition target on the image;
Object recognition program that executes The reference pattern includes a region area ratio of a set to which the orbital pattern belongs,
The object recognition program according to claim 6, wherein in the trajectory pattern correlation optimization step, the point trajectory set is selected based on a similarity of the area ratio.   The object recognition program according to any one of claims 6 to 7, wherein the trajectory pattern included in the reference pattern includes a difference in spatio-temporal coordinates between trajectories in a combination of point trajectory pairs included in the trajectory pattern. . The trajectory of the set to which the trajectory pattern of the points constituting the recognition target belongs is stored as a reference pattern,
Get the trajectory of the point on the image,
An object recognition method for selecting a set of point trajectories based on similarity between an acquired point trajectory and the reference pattern, and extracting the selected point trajectory set as a recognition target on the image. The reference pattern includes a region area ratio of a set to which the orbital pattern belongs,
The object recognition method according to claim 9, wherein the point trajectory set is selected based on a similarity of the area ratio.