JP4866793B2 - Object recognition apparatus and object recognition method - Google Patents

Description

  The present invention relates to an object recognition apparatus and an object recognition method for quickly recognizing the position of a monitoring object photographed with a video camera or the like in real time.

  When a three-dimensional object having an arbitrary shape is image-processed from moving images taken by a video camera or the like to recognize a specific monitoring target in real time, depending on the positional relationship between the camera and the object (monitoring target) The problem that the apparent image of the monitored object photographed by the camera changes variously must be dealt with. Such recognition methods can be broadly classified into a method for comparing with a geometric model and a method based on template matching.

  The method based on comparison with the geometric model stores the geometric model of the monitored object in advance, analyzes the structure of the object in the acquired image, and recognizes the monitored object by comparing with the stored geometric model To do. In comparison with a geometric model, it is difficult to accurately model the monitoring target when the shape of the monitoring target in the image is complicated. Further, the complexity of the geometric model reduces the calculation cost in image processing. Has the problem of increasing.

  On the other hand, in the method based on template matching, apparent images that change variously of the monitoring target are stored in advance as template images, and the acquired image is matched with the stored template image. It has the advantage of not relying on it. However, since this method has to store a large amount of various images as template images for matching processing, the required storage capacity increases and the time required for image processing for matching, etc. There is a problem that it is long. For this reason, various approaches have been tried to solve such problems.

  As one of such approaches, there is a method of performing matching processing using principal component analysis. Principal component analysis is a method based on multivariate data analysis, in which the characteristics of a multivariate measurement are explained by a small number of synthetic variables. Many methods such as information compression such as image compression and feature extraction for pattern recognition are used. It is used in the field of When using principal component analysis in object recognition by matching processing, the template image for matching processing is compressed as a low-dimensional eigenvector and stored as dictionary data to reduce the data volume to be stored. be able to.

  In object recognition using principal component analysis, a technique described in Non-Patent Document 1 is known. A method of compressing a template image as a low-dimensional eigenvector using principal component analysis to create dictionary data and recognizing a three-dimensional object from a moving image using the dictionary data is disclosed.

  That is, in Non-Patent Document 1, in order to recognize the position and orientation of a monitoring object that is a specific object, a plurality of apparent images that change due to rotation or the like are captured in advance as templates, and the eigenvectors of the images for each image are obtained. In the object recognition stage, an image acquired in real time is cut out, a characteristic value obtained when the cut out image is projected onto a space spanned by eigenvectors, and a specific value is calculated in the cut out image. It is determined whether or not an object is included, and if a specific object is not included, the obtained image is cut out again. If a specific object is included, which one is cut out A technique related to pattern matching by an eigenspace method of determining whether or not a template image is similar is disclosed.

Murase et al: "3D object recognition by 2D collation-parametric eigenspace method" (Science Theory, Vol.J77-D-II, No.11, pp.2179-2187,1994)

  As described above, according to the parametric eigenspace method, the image information of the template image can be reduced in dimension by using the eigenvector, so that the amount of information necessary for learning can be suppressed.

And in the nonpatent literature 2, the technique regarding a particle filter is disclosed as a method of estimating the state vector in each time sequentially.
The particle filter generates a large number of particles (= state vectors) that can be regarded as independent realization values, estimates the next state for each particle by approximation (step a), and further determines the particles that should remain as the next state. The state vector at each time is sequentially estimated by resampling the particles according to the probability distribution (procedure b) and repeating steps a and b.

Higuchi: "Explanation Particle Filter" (Science Theory, Vol.88, No.12, pp.989-994,2005)

  By comparing both the state vector of the particle estimated by the particle filter and the characteristic vector of the actual captured object, the number of objects to be compared can be reduced, and the amount of calculation is less than that of the parametric eigenspace method. Thus, the position of the monitoring object can be estimated.

However, the above-described technique has the following problems.
When a specific object is to be recognized by the eigenspace method of Non-Patent Document 1, the captured image must be compared with the template image, and in which range of the captured image the monitoring target exists. An operation to confirm (for example, an operation to obtain the center position of the monitoring object from the captured image) is required. If the center position of the monitoring object is not known, a plurality of captured images must be cut out, and each of the cut out images must be compared with a template image, which increases the amount of calculation in image processing. .

  In addition, even in the particle filter of Non-Patent Document 2, when the center position of the monitoring target is not known, the position at the next timing of the monitoring target cannot be estimated with particles, and the monitoring target It will be difficult to track. In particular, there is a problem that it is difficult to track a moving object to be monitored in real time.

  Therefore, the present invention provides an object recognition apparatus and an object recognition method that can rapidly recognize the position of a monitoring object photographed by a video camera or the like and can follow a moving monitoring object in real time.

  The present invention obtains a template image eigenvector from image information of a plurality of template images obtained by rotating a monitoring object or a three-dimensional model thereof, and stores the template image eigenvector in advance in the template image eigenvector storage unit 11 so that actual image information is stored. An image information storage unit 12 that stores at a predetermined time interval, an eigenspace method state estimation unit 13 that compares the real image information and the template image eigenvector to identify the position of the monitoring object, and a monitoring object by the particle filter method The position of the monitoring object is recognized in real time by the particle filter method state estimation unit 14 that identifies the position of the object and the estimation method switching unit 15 that switches between the eigenspace method state estimation unit 13 and the particle filter method state estimation unit 14 To do.

  That is, the object recognition apparatus according to the first aspect of the present invention captures or displays a monitored object or a three-dimensional model thereof from a plurality of angles, replaces the image information displayed or captured with a matrix display, and converts an eigenvector for the matrix. A template image eigenvector storage unit 11 that stores the eigenvector in advance, stores an image taken at a predetermined time interval as real image information, and stores the real image information in a predetermined format. A plurality of sizes are cut out, the image information of the cut out image is replaced with a vector display as a cut-out image vector, and the cut-out image vector is compared with the eigenvector, respectively. A set of an output image vector and the eigenvector is obtained, and the actual image information corresponding to the extracted image vector is extracted. Based on the position, the eigenspace method state estimation unit 13 that identifies the position of the monitoring object and a particle filter method generate a plurality of particles that are position candidates of the monitoring object, and the real image information is based on the position candidates. Each of the image information is replaced with a vector display as a cut-out image vector, the cut-out image vector and the eigenvector are respectively compared, and the cut-out image vector and the eigenvector having the highest similarity are compared. The particle filter method state estimating unit 14 for specifying the position of the monitoring object from the position candidates corresponding to the cut-out image vector and the eigenspace method state estimating unit 13 determine the position of the monitoring object. When the position can be specified, the position of the monitoring object photographed at the next timing is specified by the particle filter. The state estimation unit 14 is configured to perform the identification, and when the particle filter method state estimation unit 14 cannot identify the position of the monitoring object, the position of the monitoring object to be photographed at the next timing is specified. And an estimation method switching unit 15 to be performed by the eigenspace method state estimation unit 13.

  In addition to the configuration of the first invention, the object recognizing device according to the second invention generates a plurality of particles that are candidate positions of the monitoring object by the particle filter method, using the particle filter method state estimation unit 14 Later, each particle is expressed as (T (t + 1) -T (t)) · V (t) (where V (t) is a velocity vector calculated from the position of the specified monitoring object, T (t +1) -T (t) is characterized by being translated by an imaging time interval until imaging is performed at the next timing.

  In addition to the configuration of the first invention or the second invention, the object recognition device according to the third invention is photographed or displayed by photographing or displaying the monitoring object or its three-dimensional model from a plurality of angles. In addition to calculating the eigenvector for the matrix in advance, the enlarged image information or reduced image information obtained by enlarging or reducing the image information is replaced with the matrix display, and the enlarged image eigenvector or reduced image for the matrix is replaced with the matrix display. An eigenvector is calculated in advance, and the template image eigenvector storage unit 11 stores the enlarged image eigenvector or the reduced image eigenvector together with the eigenvector, and the eigenspace method state estimation unit 13 stores the cutout eigenvector. According to the cutout position of the image vector, (1) the cutout image vector Each of the extracted image vector and the eigenvector is compared to obtain a set of the extracted image vector and the eigenvector having the highest similarity, or (2) the extracted image vector and the enlarged image eigenvector are respectively compared. Determining a set of the cutout image vector and the enlarged image eigenvector having the highest similarity, or (3) comparing the cutout image vector and the reduced image eigenvector, respectively, Select either a high set of the cutout image vector and the reduced image eigenvector, and specify the position of the monitoring object from the cutout position of the real image information corresponding to the cutout image vector. In the particle filter method state estimation unit 14, (1) each of the cut-out image vector and the eigenvector is set in accordance with the cut-out position of the cut-out image vector. In comparison, the combination of the cutout image vector and the eigenvector having the highest similarity is obtained, or (2) the cutout image vector and the enlarged image eigenvector are respectively compared, and the similarity is highest. Obtaining a set of the cut-out image vector and the enlarged image eigenvector, or (3) comparing the cut-out image vector and the reduced image eigenvector, respectively, One of whether to obtain a set of reduced image eigenvectors is selected, and the position of the monitoring object is specified from the position candidates corresponding to the cut-out image vector.

  Further, in the object recognition device according to a fourth aspect of the invention, in addition to the configurations of the first to third aspects, the eigenspace method state estimation unit 13 includes the cut-out image vector having the highest similarity and the A set of eigenvectors is obtained, the position of the monitoring object is specified from the cut-out position of the real image information corresponding to the cut-out image vector, and the monitoring object is determined from the shooting angle or display angle of the image information corresponding to the eigenvector The angle of the object is specified, and the particle filter state estimation unit 14 obtains a set of the extracted image vector and the eigenvector having the highest similarity, and performs the monitoring from the position candidates corresponding to the extracted image vector. The position of the object is specified, and the angle of the monitoring object is specified from the shooting angle or display angle of the image information corresponding to the eigenvector. It is.

  In addition to the configuration of the fourth invention, in addition to the configuration of the fourth invention, the object recognizing device according to the fifth invention is such that the particle filter method state estimation unit 14 generates a plurality of particles that are position candidates for the monitoring object by the particle filter method. Predicting the traveling direction of the monitored object according to the angle of the identified monitored object, and the particles within an ellipse range centering on the position of the identified monitored object and having the traveling direction as a major axis It is characterized by moving.

  In addition, the object recognition method according to the sixth aspect of the present invention is to capture or display a monitoring object or a three-dimensional model thereof from a plurality of angles, replace the image information captured or displayed with a matrix display, and change an eigenvector for the matrix. A template image eigenvector storage step S11 to be stored, a real image information storage step S12 to store images taken at predetermined time intervals as real image information, a step S13a for cutting out a plurality of the real image information in a predetermined size, and cut out In step S13b, the image information of the extracted image is replaced with a vector display as a cut-out image vector, and the cut-out image vector and the eigenvector are compared with each other by comparing the cut-out image vector with the eigenvector. Finding a set and cutting out the actual image information corresponding to the cut-out image vector A step S13c including an eigenspace method state estimation step S13c for identifying the position of the monitoring object from the position, a step S14a for generating a plurality of particles as position candidates for the monitoring object by the particle filter method, Step S14b in which image information is cut out based on the position candidates, and the image information is replaced with a vector display as a cut-out image vector. The cut-out image vector is compared with the eigenvector, and the similarity is the highest. Particle filter method state estimation step including step S14c for obtaining a set of the high cut-out image vector and the eigenvector, and step S14d for specifying the position of the monitoring object from the position candidates corresponding to the cut-out image vector. S14 and eigenspace method state estimation step S13 If the position of the monitoring object can be identified more, step S15a for switching the position of the monitoring object to be imaged at the next timing to be performed in the particle filter method state estimation step S14, the particle filter method If the position of the monitoring object cannot be specified in the state estimation step S14, the switching is performed so that the position of the monitoring object captured at the next timing is specified in the eigenspace method state estimation step S13. The position of the monitoring object is specified by the estimation method switching step S15 including S15b.

  Further, in the object recognition method according to the seventh invention, in addition to the procedure of the sixth invention, in step S14a of the particle filter method state estimation step S14, particles serving as position candidates for the monitoring object are detected by the particle filter method. After generating a plurality of particles, each particle is represented by (T (t + 1) -T (t)) / V (t) (where V (t) is a velocity vector calculated from the position of the identified monitoring object, T (t + 1) −T (t) is characterized in that it is moved in parallel by an imaging time interval until imaging is performed at the next timing.

  Further, in the object recognition method according to the eighth invention, in addition to the procedure of the sixth invention or the seventh invention, in the template image eigenvector storage step S11, the monitoring object or its three-dimensional model is photographed from a plurality of angles. Or, display and replace the image information captured or displayed with a matrix display, calculate the eigenvector for the matrix in advance, expand or reduce the image information, replace the enlarged image information or reduced image information with a matrix display, An enlarged image eigenvector or reduced image eigenvector for the matrix is calculated in advance, and the enlarged image eigenvector or the reduced image eigenvector is stored together with the eigenvector. In step S13c of the eigenspace method state estimation step S13, At the cutout position of the cutout image vector Then, (1) each of the cutout image vector and the eigenvector is compared to obtain a set of the cutout image vector and the eigenvector having the highest similarity, or (2) the cutout image vector and Each of the enlarged image eigenvectors is compared to obtain a set of the extracted image vector and the enlarged image eigenvector having the highest similarity, or (3) the extracted image vector and the reduced image eigenvector Each of the comparisons is to select a set of the cutout image vector and the reduced image eigenvector having the highest similarity, and from the cutout position of the real image information corresponding to the cutout image vector The position of the monitoring object is specified, and in step S14c of the particle filter method state estimation step S14, the position of the cut-out image vector is determined according to the cut-out position of the cut-out image vector. (1) The clipped image vector and the eigenvector are respectively compared to obtain a set of the clipped image vector and the eigenvector having the highest similarity, or (2) the clipped image vector and the enlarged image Each image eigenvector is compared with each other to obtain a set of the extracted image vector and the enlarged image eigenvector having the highest similarity, or (3) each of the extracted image vector and the reduced image eigenvector is compared. Then, it is selected whether to obtain a set of the cutout image vector and the reduced image eigenvector having the highest similarity. In step S14d of the particle filter method state estimation step S14, the cutout image vector is selected. The position of the monitoring object is specified from the position candidates corresponding to the above.

  Further, the object recognition method according to the ninth invention is the cut-out image having the highest similarity in step S13c of the eigenspace method state estimation step S13 in addition to the procedures of the sixth invention to the eighth invention. A set of a vector and the eigenvector is obtained, the position of the monitoring object is specified from the cut-out position of the real image information corresponding to the cut-out image vector, and from the shooting angle or display angle of the image information corresponding to the eigenvector The angle of the monitoring object is specified, and in step S14d of the particle filter method state estimation step S14, a set of the cutout image vector and the eigenvector having the highest similarity is obtained, and the cutout image vector corresponds to the cutout image vector. The position of the monitoring object is specified from the position candidates, and the shooting angle or table of image information corresponding to the eigenvector is displayed. By identifying the angle of the monitoring target object from an angle, and is characterized in that to identify the position and angle of the monitored object.

  Further, in the object recognition method according to the tenth invention, in addition to the procedure of the ninth invention, in step S14a of the particle filter method state estimation step S14, a plurality of particles that are position candidates of the monitoring object are detected by the particle filter method. In the generation, the traveling direction of the monitoring object is predicted according to the angle of the identified monitoring object, and is within the range of an ellipse having the traveling direction as a major axis centering on the position of the identified monitoring object. The position and angle of the monitoring object are specified by moving the particles.

  According to the object recognition device according to the first configuration of the present invention (or the object recognition method according to the sixth procedure of the present invention), the eigenspace method state estimation unit 13 (or eigenspace method state estimation step S13) and the particle filter method state estimation. Two parts 14 (or particle filter method state estimation step S14) and an estimation method switching unit 15 (or estimation method switching step S15) for switching between them are provided, so that the first stage or the particle filter method state estimation unit 14 (or particle filter method state estimation step S14), if the position estimation fails, the eigenspace method state estimation unit 13 (or eigenspace method state estimation step S13) estimates the position of the captured image from a wide range. After the position is estimated, the particle filter By the state estimation unit 14 (or particle filter method state estimating step S14), (Real-time recognition of the monitored object) Fast location estimate can be continued.

  Further, according to the object recognition apparatus (or the object recognition method according to the seventh procedure of the present invention) of the second configuration of the present invention, the velocity is calculated from the position of the specified monitoring target, and each particle is determined according to the velocity. Can be moved with high accuracy by moving the measurement object in motion.

  In addition, according to the object recognition device according to the third configuration of the present invention (or the object recognition method according to the eighth procedure of the present invention), the size of the monitoring object whose apparent size varies depending on the captured position. Template images with different eigenvectors are prepared and these eigenvectors are stored respectively, and the eigenspace method state estimation unit 13 (or eigenspace method state estimation step S13) or the particle filter method state estimation unit 14 (particle filter method state estimation step). In comparing the similarities in S14), the position recognition of the monitoring object for a wide range is realized by using the eigenvector of the template image having a size corresponding to the position where the actual image information is cut out.

  Further, according to the object recognition apparatus (or the object recognition method according to the ninth procedure of the present invention) according to the fourth configuration of the present invention, the eigenspace method state estimation unit 13 (eigenspace method state estimation step S13) and the particle filter method state In the estimation unit 14 (particle filter method state estimation step S14), a pair of a cutout image vector and an eigenvector having the highest similarity is obtained, and a shooting angle or a display angle of image information corresponding to the eigenvector is obtained. The angle of the monitoring object can be specified.

  In addition, according to the object recognition device (or the object recognition method according to the tenth procedure of the present invention) of the fifth configuration of the present invention, the movement direction of the monitoring target is predicted by specifying the position and angle of the monitoring target. In the particle filter method, the number of particles by the particle filter method is reduced (or the particles are densified) by moving particles at the next timing within an ellipse range whose major axis is the moving direction of the monitored object. Thus, the speed (or accuracy) of the position estimation of the monitoring object is realized.

  Embodiments for carrying out the present invention will be described in Embodiment 1 and Embodiment 2.

A first embodiment of the present invention will be described with reference to FIGS.
FIG. 1 is an explanatory diagram illustrating the configuration of the object recognition apparatus according to the first embodiment. The object recognition device 1 is connected to a photographing device (video camera) 10 and specifies the position of a monitoring object photographed by the photographing device 10 in real time. A template image eigenvector storage unit 11, a real image information storage unit 12, an eigenspace method state estimation unit 13, a particle filter method state estimation unit 14, and an estimation method switching unit 15 are provided inside.

  Although description by illustration is abbreviate | omitted, as an example of the hardware constitutions which implement | achieve the object recognition apparatus 1, the personal computer provided with memory, central processing unit (CPU), and the capture board can be considered. An image photographed by the photographing apparatus 10 is captured on a personal computer by a capture board. The memory has storage areas corresponding to the template image eigenvector storage unit 11 and the real image information storage unit 12, and further, operations of the eigenspace method state estimation unit 13, the particle filter method state estimation unit 14, and the estimation method switching unit 15. Memory programs and so on are stored. The arithmetic processing of the eigenspace method state estimation unit 13, the particle filter method state estimation unit 14, and the estimation method switching unit 15 is performed by the CPU, and is performed based on a program stored in the memory.

  In order to specify the position of the monitoring object according to the first embodiment, the monitoring object is captured from a plurality of angles, the captured image information is replaced with a matrix display, and an eigenvector for the matrix is calculated in advance. The template image eigenvector storage unit 11 stores the calculated eigenvector. Note that the image information may be created by displaying a three-dimensional model of the monitoring object from a plurality of angles and using the displayed image information.

The method for obtaining the eigenvector of the template image is as follows. That is, a monitoring object (or a three-dimensional model of the monitoring object) is photographed (or displayed) from various angles in advance, and the image data is defined as P _i (1 ≦ i ≦ d). Here, the number of template image d is taken, P _i is an image data of m × n _{pixels, x ij (1 ≦ j ≦} J = m × n) a j-th pixel value of the image data P _i X _i = (x _i1 , x _i2 ,..., X _iJ ) ^T , x _i becomes a J-dimensional vector. Note that x _i is subjected to normalization processing so that the magnitude is | x _i | = 1, and the average value of each component x _ij (1 ≦ i ≦ d, 1 ≦ j ≦ J) Let m be.

Then, let M be a vector whose components are m, and S = (x ₁ −M, x ₂ −M,..., X _d −M) (x ₁ −M, x ₂ −M,..., X _d − M) For the covariance matrix S defined by ^T , calculate a pair (λ _j , u _j ) of eigenvalues and eigenvectors with Su _j = λ _j u _j (where λ _j ≧ λ _{j + 1} ), Choose eigenvalue and eigenvector pairs (λ _j , u _j ) in the descending order of eigenvalues (where k is less than or equal to d), and matrix E = (u ₁ , u ₂ ,..., U _k from k eigenvectors. ) And the vector g _i = E ^T (x _i −M) (1 ≦ i ≦ d) and the matrix E are stored in the template image eigenvector storage unit 11 as template image eigenvectors.

FIG. 2 shows an example of a template image. The monitored object is rotated by 10 degrees to take 36 images, of which 16 images are displayed in FIG. 3 obtains a covariance matrix S for the same template image as FIG. 2, selects _three eigenvectors as u ₁ , u ₂ , u ₃ , and obtains a vector g _i (1 ≦ i ≦ 36) obtained from the eigenvectors. Three-dimensional display. A template image that continuously changes due to rotation or the like generally has a strong correlation between two consecutive images. Therefore, when a template image is projected onto an eigenspace, a smooth closed curve is drawn as shown in FIG. . This is because two points on the eigenspace corresponding to two images having strong correlation are close in distance.

  The real image information storage unit 12 stores images taken at predetermined time intervals as real image information. That is, the image information captured from the imaging device 10 is recorded at predetermined time intervals.

The eigenspace method state estimation unit 13 cuts out a plurality of pieces of real image information with a predetermined size, replaces the image information of the cut-out images as cut-out image vectors with vector displays, and extracts the cut-out image vector and the eigenvector. Are compared to each other to obtain a pair of cutout image vectors and eigenvectors having the highest similarity, and the position of the monitoring object is specified from the cutout positions of the real image information corresponding to the cutout image vectors. is there.
The eigenspace method state estimation unit 13 obtains a pair of a cutout image vector and an eigenvector having the highest similarity, and determines the position of the monitoring object from the cutout position of the real image information corresponding to the cutout image vector. In addition to specifying, the angle of the monitoring object may be specified from the imaging angle or display angle of the image information corresponding to the eigenvector.

The particle filter method state estimation unit 14 generates a plurality of particles as position candidates for the monitoring target by the particle filter method, cuts out the actual image information based on the position candidates, and uses the image information as a cut image vector. Each is replaced with a vector display, and the clipped image vector and the eigenvector are respectively compared to obtain a pair of the clipped image vector and the eigenvector having the highest similarity, and from the position candidates corresponding to the clipped image vector, The position of the monitoring object is specified.
The particle filter method state estimation unit 14 obtains a pair of a cutout image vector and an eigenvector having the highest similarity, specifies the position of the monitoring object from position candidates corresponding to the cutout image vector, You may make it specify the angle of this monitoring target object from the imaging | photography angle or display angle of the image information corresponding to the eigenvector.

  When the eigenspace method state estimation unit 13 can identify the position of the monitoring object, the estimation method switching unit 15 determines the position of the monitoring object to be imaged at the next timing using the particle filter method state estimation. If the position of the monitoring object cannot be specified by the particle filter method state estimation unit 14, the position of the monitoring object to be photographed at the next timing is specified in the eigenspace method state. This is performed by the estimation unit 13.

  Next, the procedure of the object recognition method according to the first embodiment will be described with reference to the flowchart of FIG.

In the template image eigenvector storage step S11, the object to be monitored or its three-dimensional model is photographed or displayed from a plurality of angles by the method described above, the photographed or displayed image information is replaced with a matrix display, and an eigenvector for the matrix is obtained. Remember.
Image information photographed by the photographing apparatus 10 is sent to the real image information storage unit 12, and in the real image information storage step S12, images photographed at predetermined time intervals are stored as real image information.

  The eigenspace method state estimation step S13 includes a step S13a for cutting out a plurality of real image information with a predetermined size, a step S13b for replacing the image information of the cut out image as a cut image vector, and a cut image vector and an eigenvector. And determining the position of the monitoring target object from the cut-out position of the real image information corresponding to the cut-out image vector. Consists of S13c. In FIG. 4, the process in step S13c is described by dividing it into step S13c-1, step S13c-2, and step S13c-3.

More specifically, the eigenspace method state estimation unit 13 reads the real image information from the real image information storage unit 12, cuts out a plurality of images having the same size as the template image (step S13a), and extracts the cut image information. Each is replaced with a vector display (step S13b). Here, as an example, when the size of the template image is 30 × 50 pixels and the size of the image photographed by the photographing device is 60 × 100 pixels, the template image is shifted pixel by pixel from the actual image information photographed by the photographing device 10. When an image is cut out, the number of cut-out images is 31 × 51. Therefore, a number r (1 ≦ r ≦ 31 × 51) is assigned to each cut out image, and a correlation between the image r and the template image is obtained as follows. That is, for the clipped image r, y _j (1 ≦ j ≦ J = m × n) is the j-th pixel value of the image r, and the clipped image vector is y (r) = (y ₁ , y ₂ ,. , y _J ) ^T , y (r) is a J-dimensional vector. Here, normalization processing is performed so that y (r) has a magnitude of | y (r) | = 1.

In step S13c-1, the eigenvector is referenced from the template image eigenvector storage unit 11 (step S13c-2), and the similarity between the cut-out image vector and the eigenvector is evaluated. That is, referring to the vector g _i (1 ≦ i ≦ 36) and the matrix E, specifically, | g _i −E ^T (y (r) −M) | is replaced by each r (1 ≦ r ≦ 31 × 51 ) And each i (1 ≦ i ≦ 36).

In step S13c-3, a combination of (r ′, i ′) that minimizes the value of | g _i −E ^T (y (r) −M) | is obtained and corresponds to the cut-out image vector y (r ′). The position of the monitoring object is specified from the cutout position of the real image information to be performed. Here, if the value of | g _{i ′} −E ^T (y (r ′) − M) | exceeds a certain value, the position of the monitoring object in the eigenspace method state estimation step S13 can be specified. Returning to step S13a as having not been made (NO in step S13c-3), the position of the monitoring object at the next imaging timing is specified again in eigenspace method state estimation step S13.
In the present embodiment, the case where there is one monitoring object is described as an example. However, when there are a plurality of monitoring objects, | g _i −E ^T (y (r) −M) in step S13c-3. ) |, After obtaining the combination of (r ′, i ′) that minimizes the value, the same processing as step S13c-3 is performed on the clipped image r ′ and the clipped image other than its periphery. Thereafter, the same processing may be repeated as many times as the number of monitoring objects (or as long as the minimum value of | g _i −E ^T (y (r) −M) | is equal to or smaller than a certain value).

In step S13c-3, a set of a cutout image vector and an eigenvector having the highest similarity is obtained, and the position of the monitoring target is specified from the cutout position of the real image information corresponding to the cutout image vector. The angle of the monitoring object may be specified from the shooting angle or display angle of the image information corresponding to the eigenvector. That is, a combination of (r ′, i ′) that minimizes the value of | g _i −E ^T (y (r) −M) | is obtained, and the shooting angle or display angle of the template image i corresponding to the vector g _i Should be specified.
In this way, the eigenspace method state estimation unit 13 (eigenspace method state estimation step S13) obtains a pair of a cutout image vector and an eigenvector having the highest similarity, and obtains a shooting angle of image information corresponding to the eigenvector or By obtaining the display angle, the angle of the monitoring object can be specified.

If the value of | g _{i ′} −E ^T (y (r ′) − M) | does not exceed a certain value in step S13c-3, the result is output to an output device (not shown) such as a display. At the same time, the process proceeds to the next step S15a (YES in step S13c-3).

  In the particle filter method state estimation step S14, a plurality of particles to be monitored object position candidates are generated by the particle filter method (step S14a), the actual image information is cut out based on the position candidates, and the image information is cut. Each vector is replaced with a vector display as an output image vector (step S14b), and the cut image vector and eigenvector are respectively compared to obtain a set of the cutout image vector and eigenvector having the highest similarity (step S14c). The position of the monitoring object is specified from the position candidates corresponding to the cut-out image vector (step S14d). In FIG. 4, the process in step S14a will be described in steps S14a-1, S14a-2, and step S14a-3, and the process in step S14b will be described in steps S14b-1 and S14b-2. The process in step S14c is divided into step S14c-1 and step S14c-2.

  The particle filter generates a large number (100 in this embodiment) of particles, sequentially moves the particles randomly by a predetermined method, and there is a monitoring target somewhere in the moved particles. Assuming that the position of the monitoring object is estimated.

  Specifically, the particle filter method state estimation step S14 is activated after the switching from the eigenspace method state estimation step S13 to the particle filter method state estimation step S14 in step S15a, and generates initial particles (step S14a). -1). Here, since the position of the monitoring target is known in step S13c-3, all of the initial particles are generated at the position of the monitoring target.

In step S14a-2, the generated particles are randomly moved in order to predict the position of the monitoring target at the next timing. That is, when the position of the particle n (1 ≦ n ≦ 100) at the timing t is P ⁽ⁿ⁾ (t) = (P _x ⁽ⁿ⁾ (t), P _y ⁽ⁿ⁾ (t)), P the ^{(n) (t + 1)} = (P x (n) (t) + Rand x, P y (n) (t) + Rand y), determines the position of the particle at the next timing t + 1. Here, Rand _x and Rand _y are random numbers generated within a certain range with 0 as the central value, and are | Rand _x |, | Rand _y | ≦ maxdistance (positive constant value).

In addition to these methods, in Step S14a, after generating a plurality of particles as position candidates for the monitoring object by the particle filter method, each particle is (T (t + 1) -T (t)).・ V (t) (However, V (t) is the velocity vector calculated from the specified position of the monitored object, and T (t + 1) -T (t) is taken until the next timing. You may make it move parallel only (time interval).
That is, in step ^{S14a-2, P (n)} (t + 1) = (P x (n) (t) + (T (t + 1) -T (t)) × V x (t) + Rand x , P _y ⁽ⁿ⁾ (t) + (T (t + 1) −T (t)) × V _y (t) + Rand _y ), the particle position at the next timing t + 1 is determined.
In this way, by calculating the speed from the position of the specified monitoring object and translating each particle in accordance with the speed, the position of the moving measurement object can be tracked with high accuracy. .

In step S <b> 14 b-1, the actual image information is cut out based on the position of each particle with reference to the actual image information captured at the next timing and stored in the actual image information storage unit 12. Specifically, since the position of the particle n is P ⁽ⁿ⁾ (t + 1) = (P _x ⁽ⁿ⁾ (t + 1), P _y ⁽ⁿ⁾ (t + 1)), the coordinate P ^{( n) A} clipped image centered on (t + 1) and having the same size as the template image is created for each n (1 ≦ n ≦ 100), and each clipped image is assigned a number n (1 ≦ n ≦ 100). 100).

In step S14b-2, the image information of the clipped image is replaced with a vector display. That is, for the cut image n, y _j (1 ≦ j ≦ J = m × n) is the j-th pixel value of the image n, and the cut image vector is y (n) = (y ₁ , y ₂ ,. , y _J ) ^T , y (n) is a J-dimensional vector. Here, normalization processing is performed so that y (n) has a magnitude of | y (n) | = 1.

In step S14c-1, the eigenvector is referred from the template image eigenvector storage unit 11 (step S14c-2), and the similarity between the cut-out image vector and the eigenvector is evaluated. That is, referring to the vector g _i (1 ≦ i ≦ 36) and the matrix E, specifically, | g _i −E ^T (y (n) −M) | is each n (1 ≦ n ≦ 100) and It is calculated for each i (1 ≦ i ≦ 36).

In step S14d, a combination of (n ′, i ′) that minimizes the value of | g _i −E ^T (y (n) −M) | is obtained, and the particle corresponding to the cut-out image vector y (n ′) is obtained. The position of the monitoring object is specified from the position of n. Here, if the value of | g _{i ′} −E ^T (y (n ′) − M) | exceeds a certain value, the position of the monitoring object in the particle filter method state estimation step S14 can be specified. If not, the process proceeds to step S15b (NO in step S14d), and the position of the monitoring object at the next imaging timing is specified in the eigenspace method state estimation step S13.
In this embodiment, the case where there is one monitoring object has been described as an example. However, when there are a plurality of monitoring objects, | g _i −E ^T (y (n) −M) | After obtaining the combination of (n ′, i ′) that minimizes the value of, the same processing as step S14d is performed on the clipped image n ′ and the clipped image other than the periphery thereof. The same process may be repeated for the number of objects (or as long as the minimum value of | g _i −E ^T (y (n) −M) | is equal to or smaller than a certain value).

In step S14d, a pair of the cutout image vector and the eigenvector having the highest similarity is obtained, the position of the monitoring target is specified from the position candidates corresponding to the cutout image vector, and the eigenvector is supported. You may make it specify the angle of this monitoring target object from the imaging | photography angle or display angle of image information. That is, a combination of (n ′, i ′) that minimizes the value of | g _i −E ^T (y (n) −M) | is obtained, and the shooting angle or display angle of the template image i corresponding to the vector g _i Should be specified.
As described above, the particle filter method state estimation unit 14 (particle filter method state estimation step S14) obtains a pair of the cutout image vector and the eigenvector having the highest similarity, and the imaging angle of the image information corresponding to the eigenvector or By obtaining the display angle, the angle of the monitoring object can be specified.

If the value of | g _{i ′} −E ^T (y (n ′) − M) | does not exceed a certain value in step S14d, the result is displayed on an output device (not shown) such as a display. At the same time, the process proceeds to the next step S14a-3 (YES in step S14d).

In step S14a-3, particles at the next timing t + 1 are regenerated in order to estimate the position of the monitored object photographed at the next timing t + 1. As the generation method, for each particle n at the timing ^{_{t | g i -E T (y}} (n) -M) | minimum value of ^{_{| g i '-E T (y}} (n) -M) | asking, Likelihood is used as an evaluation index for each particle n. When the value of | g _{i ′} −E ^T (y (n) −M) | is small, the likelihood is high, and conversely, | g _{i ′} − When the value of E ^T (y (n) −M) | is large, the likelihood is assumed to be low, and the particles having high likelihood are regenerated by a method that is likely to be adopted as a candidate at the next timing.

  FIG. 5 shows a particle generation method based on the particle filter method, and particles 51 having a high likelihood and particles 52 having a normal likelihood are left as a plurality of particles or one particle at the timing t. The particles 53 having a small likelihood are disappeared / discolored at timing t. By doing so, at the next timing t + 1, the same number of particles are generated at the timing t, and position estimation with high accuracy is expected at the next timing t + 1.

  After the particles are regenerated at the next timing t + 1, the process proceeds to step S14a-2, and the position estimation of images taken after the next timing is continuously performed by the particle filter state estimation step S14.

  In the estimation method switching step S15, when the position of the monitoring object can be specified by the eigenspace method state estimation step S13, the position of the monitoring object photographed at the next timing is specified by the particle filter method state estimation. If it is not possible to specify the position of the monitoring object in step S15a, which is switched to be performed in step S14, and the particle filter method state estimation step S14, the position of the monitoring object captured at the next timing is determined. It consists of step S15b which switches so that identification may be performed in eigenspace method state estimation step S13.

  Specifically, in step S15a, when YES is determined in step S13c-3, switching is performed so that the position of the monitoring target imaged at the next timing is specified in the particle filter method state estimation step S14. Then, the process proceeds to step S14a-1. Further, step S15b performs switching so that the position of the monitoring object to be photographed at the next timing is specified in eigenspace method state estimation step S13 when NO in step S14d. Proceed to S13a.

  As described above, the eigenspace method state estimation unit 13 (or eigenspace method state estimation step S13) and the particle filter method state estimation unit 14 (or particle filter method state estimation step S14) are provided, and estimation is performed for switching between them. By providing the method switching unit 15 (or estimation method switching step S15), when the position estimation at the first stage or the particle filter method state estimation unit 14 (or particle filter method state estimation step S14) fails, the eigenspace method is used. The state estimation unit 13 (or eigenspace method state estimation step S13) performs position estimation from a wide range of the photographed image, and after the position estimation is completed, the particle filter method state estimation unit 14 (or particle filter) By legal state estimation step S14) , It is possible to continue the high-speed position estimate (real-time recognition of the monitored object).

  A second embodiment of the present invention will be described with reference to FIGS. In addition, the same code | symbol is used about the part used as the same structure and method as Example 1, and detailed description is abbreviate | omitted.

In the second embodiment, the model of the automobile shown in FIG. 6 is set as a monitoring target, and the object recognition apparatus 1 recognizes the position of the model of the automobile.
FIG. 7 is a template image obtained by photographing a car model, which is an object to be monitored, at different angles. Twelve out of 36 pictures taken by rotating the car angle by 10 degrees are extracted. It is displayed. In addition, it is preferable to process a template image after removing backgrounds other than the monitoring target object from image information actually taken as shown in FIG.
FIG. 8 shows the eigenvector of the template image when the rotation angle of the model of the automobile is 30 degrees, and displays it as an image. As described above, when the eigenvector is displayed as an image, the outline of the car model as the monitoring object appears.

  Here, the object to be monitored or its three-dimensional model is photographed or displayed from a plurality of angles, the captured or displayed image information is replaced with a matrix display, eigenvectors for the matrix are calculated in advance, and the image information is expanded or The reduced enlarged image information or reduced image information is replaced with a matrix display, and an enlarged image eigenvector or reduced image eigenvector for the matrix is calculated in advance. In the template image eigenvector storage unit 11, in addition to the eigenvector, an enlarged image eigenvector or reduced image It is preferable to store the image eigenvector together.

  FIG. 9 is an explanatory diagram showing the relationship between real image information and enlarged image information or reduced image information. That is, in addition to the template image 92, enlarged image information 93 in which a car model as a monitoring target is enlarged and reduced image information 91 in reduced size are created according to each rotation angle, and the enlarged image information 93 and the reduced image information 91 are obtained. Also, the eigenvector of the enlarged image and the eigenvector of the reduced image are created, and eigenvectors obtained with different sizes are prepared according to the degree of change in the apparent image of the monitoring target.

The eigenspace method state estimation unit 13 (step S13c of eigenspace method state estimation step S13) and the particle filter method state estimation unit 14 (step S14c of particle filter method state estimation step S14) extract the extracted image vector. In response to the,
(1) Each of the cutout image vector and the eigenvector is compared to obtain a pair of the cutout image vector and the eigenvector having the highest similarity.
(2) By comparing each of the cutout image vector and the enlarged image eigenvector to obtain a set of the cutout image vector and the enlarged image eigenvector having the highest similarity,
Or
(3) each of the cut image vector and the reduced image eigenvector is compared to obtain a set of the cut image vector and the reduced image eigenvector having the highest similarity;
Is selected, and the position of the monitoring object is specified from the cut position of the real image information corresponding to the cut image vector or the position candidate corresponding to the cut image vector.
That is, in the real image information 94, the monitored object looks small in the portion 94a photographed as a distant position, and the monitored object appears large in the portion 94b photographed as a close position. The cutout image corresponding to the far position is compared with the reduced image eigenvector, and the cutout image corresponding to the near position is compared with the enlarged image eigenvector.

  In this way, template images having different sizes are prepared for the monitoring objects having different apparent sizes depending on the captured positions, and these eigenvectors are stored respectively, and the eigenspace method state estimating unit 13 (Or eigenspace method state estimation step S13) and the particle filter method state estimation unit 14 (particle filter method state estimation step S14) compare the similarity of the template image having a size corresponding to the position where the actual image information is cut out. By using the eigenvector, the position recognition of the monitoring target for a wide range is realized.

  Subsequently, processing of the particle filter method will be described. In the second embodiment, the particle filter method state estimation unit 14 (step S14a of the particle filter method state estimation step S14) is identified when generating a plurality of particles that are candidates for the position of the monitoring target by the particle filter method. The traveling direction of the monitored object is predicted in accordance with the angle of the monitored object, and the particles are moved within an ellipse range whose major axis is the traveling direction with the position of the identified monitored object as the center.

  For example, for a monitoring object that can predict the direction of movement from the apparent rotation angle to some extent, such as an automobile, when generating multiple particles that are candidates for the position of the monitoring object by the particle filter method, Rather than randomly moving, it is more effective to make each particle unevenly distributed according to the moving direction of the monitoring object.

That is, in predicting the position of the monitoring object at the next timing, as shown in FIG. 10, the position of the monitoring object 101 is the center and the moving direction 102 (θ) is within the ellipse range 103 as the major axis. , Move each generated particle randomly. P ^{(n) where} the major radius of the ellipse is a, the minor radius is b, r _Rand is a random number generated in the range [0,1], and θ _Rand is a random number generated in the range [0,2π]. (t + 1) = (P _x ⁽ⁿ⁾ (t) + Δx, P _y ⁽ⁿ⁾ (t) + Δy) The position of the particle at the next timing t + 1 is determined. However, Δx = c × (cosθcosθ _Rand −sinθsinθ _Rand ), Δy = c × (sinθcosθ _Rand + cosθsinθ _Rand ), c = r _Rand × {(ab) ² ÷ (b ² cos ² θ _Rand + a ² sin ² θ _Rand )} ^1/2 .

  In this way, the position and angle of the monitoring object are specified, the movement direction of the monitoring object is predicted, particles in the next timing in the particle filter method, and the elliptical range with the movement direction of the monitoring object as the major axis The number of particles by the particle filter method can be suppressed (or the density of the particles can be increased), and the speed of the position estimation of the monitoring target (or higher accuracy) can be realized.

  The present invention may be used in applications such as monitoring the position and posture of a workpiece attached to an industrial robot, and monitoring the movement of a person or an object.

Explanatory drawing which shows the structure of the object recognition apparatus by Example 1. FIG. An explanatory diagram showing an example of a template image Three-dimensional projection diagram showing template image in eigenspace Flowchart showing the procedure of the object recognition method according to the first embodiment. Explanatory diagram showing how particles are generated by the particle filter method Photograph of monitoring object (model of car) in Example 2 Template image of the monitoring object (car model) in Example 2 Image diagram of eigenvectors obtained from template images and displayed. Explanatory drawing which shows the relationship between real image information and enlarged image information or reduced image information Explanatory drawing which shows the range which estimates the position of the monitoring target object at the next timing

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Object recognition apparatus 10 Imaging device 11 Template image eigenvector storage part 12 Real image information storage part 13 Eigenspace method state estimation part 14 Particle filter method state estimation part 15 Estimation method switching part 51,52,53 Particle 91 Automobile which is a monitoring object Reduced image information obtained by reducing a model 92 Template image 93 Enlarged image information obtained by enlarging an automobile model as a monitoring target 94 Real image information 94a A portion of the real image information that is photographed as a distant position 94b Near the real image information A portion 101 to be photographed as the position of the monitoring object 102 Advancing direction 103 Particle generation range

Claims (10)

Capture or display the monitoring object or its three-dimensional model from a plurality of angles, replace the captured or displayed image information with a matrix display, calculate the eigenvector for that matrix in advance,
A template image eigenvector storage unit 11 for storing the eigenvector;
A real image information storage unit 12 that stores images taken at predetermined time intervals as real image information;
A plurality of the actual image information is cut out at a predetermined size, the image information of the cut-out image is replaced with a vector display as a cut-out image vector, and the cut-out image vector and the eigenvector are respectively compared and similar An eigenspace method state estimation unit for obtaining a set of the cutout image vector and the eigenvector having the highest performance and specifying the position of the monitoring object from the cutout position of the real image information corresponding to the cutout image vector 13 and
By generating a plurality of particles that are candidate positions of the monitoring object by the particle filter method, each of the actual image information is cut out based on the position candidates, and the image information is replaced with a vector display as a cut-out image vector, The clipped image vector and the eigenvector are respectively compared to obtain a set of the clipped image vector and the eigenvector having the highest similarity, and the monitoring target is determined from the position candidates corresponding to the clipped image vector. A particle filter method state estimation unit 14 for specifying the position of an object;
If the position of the monitoring object can be specified by the eigenspace method state estimation unit 13, the particle filter method state estimation unit 14 specifies the position of the monitoring object captured at the next timing. Thus, if the position of the monitoring object cannot be specified by the particle filter method state estimation unit 14, the eigenspace method state estimation unit 13 specifies the position of the monitoring object to be photographed at the next timing. An estimation method switching unit 15 to be performed in
An object recognition apparatus comprising:   The particle filter method state estimation unit 14 generates a plurality of particles that are candidates for the position of the object to be monitored by the particle filter method, and then converts each particle to (T (t + 1) −T (t)) · V ( t) (where V (t) is the velocity vector calculated from the specified position of the monitored object, and T (t + 1) -T (t) is the shooting time interval until shooting at the next timing) The object recognition apparatus according to claim 1, wherein the object recognition apparatus is translated only by the parallel movement. The object to be monitored or its three-dimensional model is photographed or displayed from a plurality of angles, the photographed or displayed image information is replaced with a matrix display, eigenvectors for the matrix are calculated in advance, and the image information is enlarged or reduced. The enlarged image information or the reduced image information is replaced with a matrix display, and an enlarged image eigenvector or reduced image eigenvector for the matrix is calculated in advance,
In the template image eigenvector storage unit 11, in addition to the eigenvector, the enlarged image eigenvector or the reduced image eigenvector is stored together,
In the eigenspace method state estimation unit 13, according to the cutout position of the cutout image vector,
(1) Each of the cutout image vector and the eigenvector is compared to determine a set of the cutout image vector and the eigenvector having the highest similarity,
(2) Each of the extracted image vector and the enlarged image eigenvector is compared to obtain a set of the extracted image vector and the enlarged image eigenvector having the highest similarity,
Or
(3) Each of the cut-out image vector and the reduced image eigenvector is compared to obtain a set of the cut-out image vector and the reduced image eigenvector having the highest similarity.
Is selected, and the position of the monitoring object is specified from the cut-out position of the real image information corresponding to the cut-out image vector,
In the particle filter method state estimation unit 14, according to the cutout position of the cutout image vector,
(1) Each of the cutout image vector and the eigenvector is compared to determine a set of the cutout image vector and the eigenvector having the highest similarity,
(2) Each of the extracted image vector and the enlarged image eigenvector is compared to obtain a set of the extracted image vector and the enlarged image eigenvector having the highest similarity,
Or
(3) Each of the cut-out image vector and the reduced image eigenvector is compared to obtain a set of the cut-out image vector and the reduced image eigenvector having the highest similarity.
The object recognition apparatus according to claim 1, wherein the position of the monitoring target is specified from the position candidates corresponding to the cut-out image vector. The eigenspace method state estimation unit 13 obtains a set of the cutout image vector and the eigenvector having the highest similarity, and determines the monitoring target object from the cutout position of the real image information corresponding to the cutout image vector. Specify the position, and specify the angle of the monitoring object from the shooting angle or display angle of the image information corresponding to the eigenvector,
The particle filter method state estimation unit 14 obtains a set of the extracted image vector and the eigenvector having the highest similarity, and specifies the position of the monitoring object from the position candidates corresponding to the extracted image vector. The object recognition apparatus according to any one of claims 1 to 3, wherein an angle of the monitoring object is specified from a shooting angle or a display angle of image information corresponding to the eigenvector.   The particle filter method state estimation unit 14 predicts the traveling direction of the monitoring target object according to the angle of the specified monitoring target object when generating a plurality of particles that are candidates for the position of the monitoring target object by the particle filter method. The object recognition apparatus according to claim 4, wherein the particle is moved within an ellipse range with the traveling direction as a major axis around the position of the identified monitoring target. A template image eigenvector storage step S11 for capturing or displaying the monitoring object or its three-dimensional model from a plurality of angles, replacing the image information captured or displayed with a matrix display, and storing eigenvectors for the matrix;
Real image information storage step S12 for storing images taken at predetermined time intervals as real image information;
Step S13a for cutting out a plurality of the actual image information with a predetermined size, Step S13b for replacing the image information of the cut-out image with a vector display as a cut-out image vector, and comparing the cut-out image vector with the eigenvector, respectively. Step S13c for obtaining a set of the cutout image vector and the eigenvector having the highest similarity and specifying the position of the monitoring object from the cutout position of the real image information corresponding to the cutout image vector. Including the eigenspace method state estimation step S13;
Step S14a for generating a plurality of particles as position candidates for the monitoring object by the particle filter method, cutting out the actual image information based on the position candidates, and replacing the image information with a vector display as a cut-out image vector, respectively. Step S14b, comparing the cut-out image vector and the eigenvector, respectively, to obtain a set of the cut-out image vector and the eigenvector having the highest similarity, and the position candidate corresponding to the cut-out image vector From the step S14d including the step S14d for specifying the position of the monitoring object,
When the position of the monitoring object can be specified by the eigenspace method state estimation step S13, the position of the monitoring object captured at the next timing is specified in the particle filter method state estimation step S14. If the position of the monitoring object cannot be specified by the switching step S15a and the particle filter method state estimation step S14, the eigenspace method state estimation step S13 specifies the position of the monitoring object to be photographed at the next timing. An estimation method switching step S15 including a step S15b of switching so as to be performed at
An object recognition method for identifying the position of a monitoring object by   In step S14a of the particle filter method state estimation step S14, a plurality of particles that are candidates for the position of the monitoring object are generated by the particle filter method, and then each particle is (T (t + 1) -T (t)).・ V (t) (However, V (t) is the velocity vector calculated from the specified position of the monitored object, and T (t + 1) -T (t) is taken until the next timing. The object recognition method according to claim 6, wherein the object is translated by a time interval. In the template image eigenvector storage step S11, the monitored object or its three-dimensional model is photographed or displayed from a plurality of angles, the photographed or displayed image information is replaced with a matrix display, and eigenvectors for the matrix are calculated in advance. The enlarged image information or reduced image information obtained by enlarging or reducing the image information is replaced with a matrix display, and an enlarged image eigenvector or reduced image eigenvector for the matrix is calculated in advance, in addition to the eigenvector, the enlarged image eigenvector or the Store the reduced image eigenvector together,
In step S13c of the eigenspace method state estimation step S13, according to the cutout position of the cutout image vector,
(1) Each of the cutout image vector and the eigenvector is compared to determine a set of the cutout image vector and the eigenvector having the highest similarity,
(2) Each of the extracted image vector and the enlarged image eigenvector is compared to obtain a set of the extracted image vector and the enlarged image eigenvector having the highest similarity,
Or
(3) Each of the cut-out image vector and the reduced image eigenvector is compared to obtain a set of the cut-out image vector and the reduced image eigenvector having the highest similarity.
Is selected, and the position of the monitoring object is specified from the cut-out position of the real image information corresponding to the cut-out image vector,
In step S14c of the particle filter method state estimation step S14, according to the cutout position of the cutout image vector,
(1) Each of the cutout image vector and the eigenvector is compared to determine a set of the cutout image vector and the eigenvector having the highest similarity,
(2) Each of the extracted image vector and the enlarged image eigenvector is compared to obtain a set of the extracted image vector and the enlarged image eigenvector having the highest similarity,
Or
(3) Each of the cut-out image vector and the reduced image eigenvector is compared to obtain a set of the cut-out image vector and the reduced image eigenvector having the highest similarity.
Select one of the
The position of the monitoring target is specified from the position candidates corresponding to the cut-out image vector in step S14d of the particle filter method state estimation step S14. Object recognition method. In step S13c of the eigenspace method state estimation step S13, a set of the cutout image vector and the eigenvector having the highest similarity is obtained, and the monitoring is performed from the cutout position of the real image information corresponding to the cutout image vector. While specifying the position of the object, specify the angle of the monitoring object from the imaging angle or display angle of the image information corresponding to the eigenvector,
In step S14d of the particle filter method state estimation step S14, a set of the cutout image vector and the eigenvector having the highest similarity is obtained, and the position of the monitoring object is determined from the position candidates corresponding to the cutout image vector. The position and angle of the monitoring object are specified by specifying the angle of the monitoring object from the imaging angle or display angle of the image information corresponding to the eigenvector. The object recognition method according to claim 8.   In step S14a of the particle filter method state estimation step S14, when generating a plurality of particles that are candidates for the position of the monitoring object by the particle filtering method, the traveling direction of the monitoring object according to the angle of the specified monitoring object And the position and angle of the monitoring object are specified by moving the particles within an ellipse whose major axis is the traveling direction around the position of the specified monitoring object. The object recognition method according to claim 9.