JP4553915B2 - Motion detection device, motion detection method, and motion detection program - Google Patents

Description

  The present invention relates to a technique for accurately detecting movement of a moving object from an image of bad weather.

  Conventionally, in order to grasp changes in the natural environment and movements of moving objects such as vehicles, ships, and people, a surveillance camera is usually installed at a predetermined outdoor location, and the images captured by the camera are monitored live. A method and a method of checking an image obtained by performing image processing on the video are used. In particular, in the latter case, in order to accurately grasp the movement of the moving object, it greatly depends on the performance of the technology for processing the image.

  Regarding image processing technology, processing using the cross-correlation method (also referred to as “CC method” or “pattern matching method”) used in the field of coding represented by MPEG (Moving Picture Experts Group) The technology to do is famous. This CC method is a method in which similarity between two consecutive images is evaluated by a cross-correlation function or the like, and a similar region is regarded as a movement distance (vector) between images.

  However, the cross-correlation method is a processing method on the premise that there is no luminance fluctuation between the images, and therefore has low applicability when detecting the movement of a moving object in a natural environment where the luminance fluctuation is severe. it is conceivable that.

Therefore, in Non-Patent Document 1, an attempt is made to estimate a movement of a moving object based on an optical flow method using luminance fluctuations of video and images taken with a camera. Non-Patent Document 2 discloses a technique for detecting a moving object such as a vehicle existing in the foreground from a background such as tree fluctuation. Further, Non-Patent Document 3 discloses an image processing technique when the weather is foggy.
JL Barron, 2 others, “Systems and experiment performance of optical flow techniques”, Int'l J. Computer Vision, 1994, vol. 12, p. 43-77 J. Kato, 4 others, “An HMM / MRF-based stochastic framework for robust vehicle tracking”, IEEE PAMI, 2004, vol.5, no.3, p.142-154 SG Narasimhan, 1 other, "Vision and the atmosphere", Int'l J. Computer Vision, 2002, no.48, no.3, p.233-254 HW Haussecker, 1 other, "Computing optical flow with physical models of brightness variation", IEEE Trans. PAMI, vol.23, no.6, p.661-673 H. Sakino, “Nonlinear robust velocity estimation of vehicles from a snowfall traffic scene”, IEEE ICPR, 2002, vol. 4, p. 60-63

  However, the optical flow method described in Non-Patent Document 1 is a method that detects motion using only luminance fluctuations of video and images, and does not consider a motion model related to weather phenomena, and therefore considers changes in weather. There was a problem that the movement of the moving object could not be detected accurately.

  In addition, the estimation method described in Non-Patent Document 2 is a method of detecting a moving object under conditions where the weather is clear, so in the case of weather covered with rain or snow that covers the background and the foreground. However, there is a problem that the movement of the moving object cannot be accurately detected.

  Furthermore, since Non-Patent Document 3 discloses only a processing method that makes it easy to see a building or the like, there has been a problem that the movement of the moving object cannot be estimated.

  In Non-Patent Document 4, in order to solve these problems, a luminance variation model based on a physical phenomenon such as a light source movement model is applied. However, it does not consider complex physical models when multiple phenomena occur at the same time, such as snowfall phenomena such as wind and snow. There was a problem that could not be detected.

  The present invention has been made in view of the above, and an object of the present invention is to stably and accurately detect the movement of a moving object even in bad weather.

The motion detection apparatus according to the first aspect of the present invention includes an image input means for inputting an image including weather, an image storage means for storing the input image in time series, and a luminance at a position on the image at time t. Λ ₁ I _XX + 2λ ₂ I _XY + λ ₃ I _YY (where I _XX , I _XY , and I _YY are modeled on the anisotropic diffusion of luminance with respect to time variation of I (= ∂I / ∂t)) The second order differential of the luminance in the horizontal direction, the oblique direction, and the vertical direction, respectively, and λ ₁ , λ ₂ , and λ ₃ are the horizontal diffusion, the oblique direction, and the vertical diffusion coefficients), and the luminance is diffusely reflected. Of -κI (where κ is an exponential coefficient) and Isinθ · cosθ + I ² sinθ ² · cosθ ² (where θ is the angle of the wave) The anisotropic diffusion With the number and the exponential and the sine function _{| I t + I X u +} I Y v- (λ 1 I XX + 2λ 2 I XY + λ 3 I YY) + κI + Isinθ · cosθ + I 2 sinθ 2 · cosθ 2 | ( however, ( u, v) is a velocity variation equation e ₀ of the velocity vector to be moved), and is an expression with the constraint that the velocity vector at the position on the image is parallel to the velocity vector at a position near the position. derive the e _1, the equation e ₀ in the formula and the objective function deriving means and e ₁ to derive the objective function obtained by substituting a robust function based on robust statistics, the image storage unit two successive images is read out from, by substituting the luminance values of the two images in the objective function, and the velocity vector estimating means for estimating the velocity vector, display means for displaying the speed vector estimated , And summarized in that with.

In the present invention, λ ₁ I _XX + 2λ ₂ I that models the time-dependent change in luminance I at the position on the image at time t (= ∂I / ∂t) that the luminance diffuses anisotropically. _XY + λ ₃ I _YY (where I _XX , I _XY , and I _YY are second-order derivatives of luminance in the horizontal direction, diagonal direction, and vertical direction, respectively , and λ ₁ , λ ₂ , and λ ₃ are horizontal direction, diagonal direction, and vertical direction, respectively. Direction diffusion coefficient), an exponential function of -κI (where κ is an exponential coefficient) that models diffused luminance, and Isinθ · defined as a sine function of cos θ + I ² sin θ ² · cos θ ² (where θ is the angle of the wave), and using the anisotropic diffusion function, the exponential function, and the sine function, | I _t + I _X u + I _Y v− (Λ ₁ I _XX + 2λ ₂ I _XY + λ ₃ I _YY ) + κI + I sin θ · cos θ + I ² sin θ ² · cos θ ² | (where (u, v) are velocity vectors to be moved) to derive a luminance variation equation e _0, and the velocity vector of the position on the image and the vicinity of the position Using an objective function obtained by substituting the equation e ₀ and the equation e ₁ into a robust function based on robust statistics, deriving the equation e ₁ with a constraint that the velocity vector of the position is parallel . Since the velocity vector of the moving object is estimated from the luminance values of two consecutive images, the movement of the moving object can be detected stably and accurately even in bad weather.

  A motion detection apparatus according to a second aspect of the present invention is the motion detection apparatus according to claim 1, wherein the velocity vector estimation unit divides the two images into the same number of sub-blocks, and The speed vector is estimated every time, and the two images are divided into a number of sub-blocks different from the number of sub-blocks, and the speed vector is estimated repeatedly. Based on the velocity vector, energy distribution deriving means for obtaining the energy distribution of the moving object, and determination for sequentially determining the similarity between the energy distribution and the predetermined energy distribution from the energy distribution of the image having a small number of sub-blocks And when the similarity reaches a predetermined level, the display means determines the number of sub-blocks having the energy distribution. And summarized in that displaying a velocity vector of the split image.

  In the present invention, the two images are divided into the same number of sub-blocks, the velocity vector is estimated for each sub-block, and the two images are different from the number of sub-blocks. It is repeatedly performed by dividing into sub-blocks and estimating the velocity vector. Based on the estimated velocity vector, the energy distribution of the moving object is obtained, and the energy distribution is similar to the predetermined energy distribution. In order to display the velocity vector of the image divided into the number of sub-blocks having this energy distribution when the degree of similarity is reached in order from the energy distribution of the image with a small number of sub-blocks and the similarity reaches a predetermined level, It is possible to detect a more appropriate movement of the moving object. In general image processing, the number of sub-blocks is determined empirically, but in the present invention, it is also possible to determine an image having the number of sub-blocks approximated to a predetermined energy distribution by the motion detection device.

  A motion detection apparatus according to a third aspect of the present invention is the motion detection apparatus according to claim 1 or 2, wherein the velocity vector estimation means uses a Lorentz function as the robust function used when calculating the objective function. This is the gist.

  In the present invention, since the speed vector estimation means uses the Lorentz function as the robust function used when calculating the objective function, the calculation speed of the objective function can be improved.

  A motion detection apparatus according to a fourth aspect of the present invention is the invention according to claim 2 or 3, wherein the predetermined energy distribution is a Kolmogorov energy distribution.

A motion detection method according to a fifth aspect of the present invention includes a first step of inputting an image including weather, a second step of storing the input image in the image storage means in time series, and a time t Λ ₁ I _XX + 2λ ₂ I _XY + λ ₃ I _YY (where I _XX , I _XY , I _YY are the second-order derivatives of luminance in the horizontal direction, the diagonal direction, and the vertical direction, respectively , and λ ₁ , λ ₂ , and λ ₃ are the diffusion coefficients in the horizontal direction, the diagonal direction, and the vertical direction, respectively. , -KappaI modeling that luminance is diffuse reflection (where, kappa is exponential coefficient) and exponential functions, · Isinθ modeled that luminance changes in a wave shape ^{cosθ + I 2 sinθ 2 · cosθ} 2 ( however, theta Is the sine function of the wave angle) And, by using the the anisotropic diffusion function and the exponential function and the sine function _{| I t + I X u +} I Y v- (λ 1 I XX + 2λ 2 I XY + λ 3 I YY) + κI + Isinθ · cosθ + I 2 sinθ 2 · cosθ ² | (where (u, v) is the velocity vector of the moving object), the luminance fluctuation equation e ₀ is derived, and the velocity vector at the position on the image and the velocity vector at the position near the position are parallel. the derive the equation e ₁ which is a constraint condition, a third step of deriving the objective function obtained by substituting a robust function based an said formula e ₀ and the equation e ₁ robust statistics, two consecutive It reads the image from said image storage means, substituting the luminance values of the two images in the objective function, a fourth step of estimating the velocity vector to display the velocity vector estimated A fifth step, is summarized in that with.

  A motion detection method according to a sixth aspect of the present invention is the motion detection method according to claim 5, wherein the fourth step divides the two images into the same number of sub-blocks, The speed vector is estimated every time, and the two images are divided into a number of sub-blocks different from the number of sub-blocks, and the speed vector is estimated repeatedly. Obtaining an energy distribution of the moving object based on the velocity vector; and determining a similarity between the energy distribution and a predetermined energy distribution in order from an energy distribution of an image having a small number of sub-blocks. The fifth step further includes the step of dividing the image into the number of sub-blocks having the energy distribution when the similarity reaches a predetermined level. And summarized in that displaying a velocity vector.

  The motion detection method according to a seventh aspect of the present invention is the invention according to claim 5 or 6, wherein the fourth step uses a Lorentz function as the robust function used when calculating the objective function. This is the gist.

  A motion detection method according to an eighth aspect of the present invention is the invention according to claim 6 or 7, wherein the predetermined energy distribution is a Kolmogorov energy distribution.

  A gist of a motion detection program according to a ninth aspect of the present invention is to cause a computer to execute each step in the motion detection method according to any one of claims 5 to 8.

  According to the present invention, it is possible to detect the movement of a moving object stably and accurately even in bad weather.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Configuration of motion detection device]
FIG. 1 is a configuration diagram illustrating a motion detection apparatus 10 according to an embodiment of the present invention. The motion detection apparatus 10 includes an image input unit 100 that inputs an image including weather, an image storage unit 110 that stores the input image in time series, an objective function derivation unit 120 that derives an objective function, and a velocity vector. An estimated speed vector estimating unit 130, an energy distribution deriving unit 140 for determining the energy distribution of the weather, a determining unit 150 for determining the similarity of the obtained energy distribution, and a speed vector estimated based on the determination result are displayed. It is the structure provided with the display part 160 to perform.

  The motion detection device 10 is realized by a CPU, a memory, a hard disk, and the like that constitute a computer main body, and an operation process of each step described below is executed by a program.

[Processing of motion detection device]
Next, processing of each unit 100 to 160 configuring the motion detection device 10 will be described. Note that all the meanings of the luminance value, intensity value, and gray value of an image are equivalent, and image data used for calculation will be described below as the gray value of an image.

  First, a plurality of continuous images including weather are input to the image input unit 100 (step S1). The image input here is an image taken under bad weather such as rain, fog, and snow. In this embodiment, the fog scene shown in FIG. 2A and the image shown in FIG. The snow scene image shown is used.

  Next, the image accumulating unit 110 accumulates a plurality of images input by the image input unit 100 in time series (step S2). The time series means a sequence in which observed values of phenomena that change with time are recorded with time.

  Subsequently, the objective function deriving unit 120 sets the velocity vector to be moved as a variable based on the anisotropic diffusion function indicating the luminance diffusion phenomenon, the exponential function indicating the irregular reflection of luminance, and the sine function indicating the wave of luminance. Then, an objective function is derived by substituting the luminance variation equation and the constraint condition formula based on the parallel mobility of the moving object into a robust function based on robust statistics (step S3). A specific method for deriving the objective function will be described later.

  The velocity vector estimation unit 130 reads two consecutive images from the image storage unit 110, substitutes the gray values of the two images into the objective function derived in step 3, and calculates the luminance variation equation as an optical flow component. And the anisotropic diffusion coefficient of the anisotropic diffusion function as an unknown related to meteorological physics, the exponential coefficient of the exponential function, and the direction (angle) of the sine function are estimated (step S4).

  Specifically, the speed vector estimation unit 130 divides the two consecutive read images into the same number of sub-blocks, and uses the gray values of the sub-blocks located at the same location for each sub-block. The above velocity vector is estimated by performing the minimization calculation in the steepest descent method. Further, the two images are divided into a number of sub-blocks different from the number of the previous sub-blocks, and the velocity is calculated in the same manner. It repeatedly performs estimation of vectors and the like.

  For example, first, two continuous images are divided into a total of nine sub-blocks of 3 × 3, and a velocity vector or the like is estimated for each sub-block. Next, the two continuous images are divided into a total of 16 sub-blocks of 4 × 4, and the velocity vector and the like are similarly estimated. Next, a repetitive speed vector or the like is estimated for a total of 25 images of 5 × 5 and a total of 36 images of 6 × 6 (hereinafter omitted). Note that the image dividing method is not limited to the above. For example, even 3 × 5 or 5 × 3 does not affect the obtained effect.

  The energy distribution deriving unit 140 obtains the energy distribution of the movement target based on the velocity vector estimated in step S4 (step S5). A method for obtaining this energy distribution will be described later.

  The determination unit 150 determines the similarity between the energy distribution obtained in step S5 and the predetermined energy distribution in order from the energy distribution of the image with the smaller number of sub-blocks (step S6).

  The predetermined energy distribution used in the present embodiment is a Kolmogorov energy distribution, which is one of the wind statistical distributions described later. Further, as a method for determining the degree of similarity, for example, a cross-correlation function is used.

  Finally, when the similarity reaches a predetermined level, the display unit 160 displays the motion vector (velocity vector) of the image divided into the number of sub-blocks having the energy distribution reaching the level as an arrow symbol on the screen. Display (step S7).

[Derivation method of objective function]
Next, a specific method for deriving the objective function described in step S3 will be described based on the reason.

  In general, the luminance fluctuation between two consecutive time-series images in a weather scene is very complicated. In the fog scene shown in FIG. 2A, the average luminance is high, and the luminance variation with respect to time is moderate. In addition, it is also characterized by low visibility in the depth direction and locally low contrast. Accordingly, since the luminance diffusion phenomenon and the wavy shade value change are remarkable, it is necessary to consider the anisotropic diffusion model indicating the diffusion phenomenon and the wave model indicating the wave as factors of the luminance fluctuation.

  Further, in the snowfall scene and the snowfall scene shown in FIG. 2B, since there is irregular reflection of light from the road and snow particles where snow has accumulated, it is necessary to consider an exponential function model of luminance. Needless to say, even in the case of a clear sky scene, there is irregular reflection of luminance from a dry road.

  Therefore, an objective function considering an anisotropic diffusion model, a wave model, and an exponential function model is derived as a luminance variation model related to meteorological physics.

First, the anisotropic diffusion model will be described. The anisotropic diffusion model indicates a luminance diffusion phenomenon with respect to a change in time, and can be expressed by Expression (1). This is because the movement of airflow exists in various directions in the space. For example, the density of fog is strongly influenced by the wind, so it is impossible to assume the isotropic diffusion, so it is modeled as a diffusion phenomenon of luminance. Is.

Note that I indicates a two-dimensional image luminance, and I _xx , I _xy , and I _yy are second-order derivatives of the space in the horizontal direction, the vertical direction, and the oblique direction, respectively. Further, λ ₁ , λ ₂ , and λ ₃ are diffusion coefficients in the horizontal direction, the vertical direction, and the oblique direction, respectively. The left side represents a change in luminance over time.

Next, the exponential function model will be described. The exponential function model indicates diffused reflection of luminance with respect to time change, and can be expressed by Expression (2).

  Note that κ is an exponential coefficient. Further, δt is a sampling time.

Next, the wave model will be described. The wave model represents a luminance wave with respect to a change in time, and can be approximated as a time change of a trigonometric function as shown in Expression (3). Note that the wave direction (angle) θ is defined as a power, considering only two components of low frequency and high frequency. Also, the right side of the I, I ² are those corresponding to the amplitude.

  Here, the movement of a moving object with a rigid body, such as a car or train, included in the photographed image clearly includes more parallel components than the movement of fog or snow, and has a certain area or more. is doing. Accordingly, a constraint model as a constraint condition regarding the size of the rigid body and the parallel movement of the motion is further given to the objective function derived by the model group.

The size of the rigid body can be expressed by the number of connections between the position (i, j) of the image and the position (i + 1, j + 1) in the vicinity thereof. Further, for example, the condition that two neighboring velocity vectors a = (a ₁ , a ₂ ) and b = (b ₁ , b ₂ ) are parallel can be expressed by Expression (4).

Therefore position (i, j) constraints e ₁ of vertical and horizontal, 8 neighboring points connected under oblique on oblique relative can be represented by the formula (5).

  In addition to the eight-point vicinity connection, it is also possible to use constraint conditions such as four-point vicinity connection on the top, bottom, left and right, and two-point vicinity connection on the left and right.

Based on the equations (1) to (3) described above, a luminance variation equation e ₀ with (u, v) as the velocity vector of the moving object is derived, and this luminance variation equation e ₀ and equation (5) deriving an objective function shown in equation (6) obtained by substituting the robust function based and constraint e ₁ shown in robust statistics.

Note that γ means strength control with respect to the constraint condition e ₁ and is 1.0 in the present embodiment. The Lorentz function ρ is used as the robust function.

From this objective function, a velocity vector (u, v) as an optical flow component, an anisotropic diffusion coefficient λ ₁ , λ ₂ , λ ₃ , an exponential coefficient κ, and a direction (angle) θ of a sine function as unknowns related to meteorological physics And calculate. When calculating this objective function, as described above, minimization calculation is performed using the steepest descent method represented by Equation (7).

  Note that p indicates the number of iterations.

In the iterative process in the nonlinear calculation represented by Expression (7), the anisotropic diffusion coefficient and the exponential coefficient may be unnaturally calculated as negative values. Therefore, in the iterative process, as shown in the equation (8), a constraint condition for setting only a positive value is further given.

  By applying the robust estimation method described above, other factors that cause luminance fluctuations such as noise, discontinuous components, and occlusion are regarded as outliers, so that a velocity vector having a certain level of accuracy can be estimated. .

[Method of obtaining energy distribution of moving object]
In a fog scene and a snowfall scene, fog and snow dance occur due to the influence of the wind. Therefore, the accuracy of optical flow estimation can be further improved by incorporating the following wind model.

Here, in order to incorporate the wind model, the energy distribution of the moving object is calculated using the velocity vector estimated from the objective function. The energy to be moved assumes that fog or snow is affected by wind and its kinetic energy is determined. When one particle has mass m and velocity u, the equation (9 ).

  Here, since m is a fine particle such as fog or snow, it is set to 0.001 in the present embodiment. Thus, the number of pixels at each speed is regarded as the number of particles, and the result of integration at each speed is calculated as energy.

[About judgment method]
The Kolmogorov energy distribution already known as the wind statistical distribution is used. According to this distribution, when the relationship between wind fluctuation (velocity) and energy is displayed in log-log form, the energy can be approximated to the power of −2.0 with respect to the wave number k as shown in Equation (10). It has been confirmed statistically. However, this approximation is a characteristic obtained only in the low frequency band, that is, in the low speed region. In this embodiment, an evaluation is made as to how close the estimated energy distribution of the weather is to this energy distribution.

[About estimation results]
First, the estimation result of the movement target movement will be described. FIG. 3 is a diagram in which speed vectors obtained by estimating the two types of weather images shown in FIG. FIG. 4 is a diagram in which speed vectors obtained by estimating the two types of weather images shown in FIG. 2 in the present embodiment are displayed by arrows. 3 and 4 show an enlarged view of only the region surrounded by circles shown in FIG. 2 (hereinafter the same applies to all the drawings). In addition, the conventional method in the present embodiment is a method using the HS (Horn & Schunkck) method or the LK (Lucus & Kanade) method (see Non-Patent Document 5).

  In the case of the conventional method, a motion vector that is considerably disturbed with respect to the traveling direction is detected in an area where a traveling vehicle that is a moving object exists, and the traveling direction of the traveling vehicle cannot be accurately grasped. However, all the motion vectors estimated in the present embodiment are displayed in a direction substantially parallel to the traveling direction of the traveling vehicle. Even in bad weather, the influence of the movement of fog or snow Therefore, it can be confirmed that the movement of the moving object can be detected more accurately than in the past.

  Next, the estimation result of the moving direction (angle) of the moving object will be described. FIG. 5 is a diagram showing the direction (angle) estimated from the snowfall scene image in the present embodiment. FIG. 5A is a three-dimensional display of the result of calculating the average of the angles formed by the velocity vectors in the vicinity of four points in the vertical and horizontal directions and evaluating with the SIN function. The vertical axis indicating the upward direction indicates the magnitude of the angle. The larger the value, the more non-directional, that is, the variation in the moving direction. The area where the traveling vehicle is present is expressed in white and the size in the upward direction is substantially zero, so that it can be understood that the vehicle is moving in parallel. FIG. 5B is a two-dimensional representation of the three-dimensional display of FIG.

  On the other hand, FIG. 6 is a diagram showing the direction (angle) estimated from the image of the snowfall scene by the conventional method. FIG. 6A shows the result estimated by the HS method, and FIG. 6B shows the result estimated by the LK method. In the case of the conventional method, it can be confirmed that there is no parallel vector in the region where the traveling vehicle corresponding to the white outline in FIG. That is, in the conventional method, the difference between a car and snow is not known, but in the present embodiment, the difference can be clearly grasped. This is because the conventional method does not consider the constraint condition regarding the parallelism of the movement, and the effectiveness of the constraint condition of the parallel mobility used in the present embodiment can be confirmed from the above result.

  Next, the evaluation result for the constraint condition will be described. FIG. 7 is a diagram showing the pixel occurrence probability with respect to the average value of the angles formed by the velocity vectors in the vicinity of four points in the velocity vector estimated in the present embodiment for the snowfall scene image. Further, FIG. 7B evaluates the case where the number of neighboring connections in the constraint condition is changed to 2, 6, and 8.

  According to FIG. 7 (a), it can be clearly understood that the estimation result in the present embodiment has higher parallelism than the conventional method, and according to FIG. 7 (b), by increasing the number of connections. It can be confirmed that the result more reflects the area of the traveling vehicle.

  Next, the evaluation results of the anisotropic diffusion coefficient and the exponential coefficient will be described. FIG. 8 is a diagram displaying the averaged diffusion coefficient distribution obtained by estimating the snowfall scene image according to the present embodiment. FIG. 9 is a diagram showing an averaged index coefficient distribution obtained by estimating the snowfall scene image in the present embodiment. According to FIGS. 8 and 9, it can be confirmed that the diffusion coefficient and the exponential coefficient are relatively high in the region of the traveling vehicle where the luminance variation is locally significant.

  Then, the evaluation result of energy distribution is demonstrated. FIG. 10 is a diagram showing the probability of energy occurrence with respect to the speed estimated in the present embodiment. FIG. 10A is a diagram showing the energy occurrence probability obtained by calculating the energy using the equation (9) based on the speed of the fog scene image estimated in the present embodiment, and FIG. It is a distribution map in the case of a snowfall scene. FIG. 10 also clearly shows the Kolmogorov energy distribution (statistical model) approximated to the power of −2.0.

  Note that the estimation results shown in FIG. 10 are obtained by varying the number of sub-blocks from a total of 9 × 3 × 3 to a total of 400 × 20 × 20, estimating the velocity vector for each image, The results of estimation using the total number of 100 subblocks of 10 × 10, which is the highest, are described.

  The energy of the estimation result in the present embodiment is approximated by about -1.9 power when the power is calculated from the low speed region (1.E-03 to 1.E-01) in both scenes. It can be confirmed that the similarity with the energy distribution of Kolmogorov is high. Since these energy distributions obtained based on fog scenes and snowfall scenes are similar to the statistical model, the estimated motion components include wind fluctuation components for fog and snowfall. Suggests that

  On the other hand, in the case of the LK method, since the graph is distributed on the high speed side as a whole, it can be determined that it is different from the wind model. In the case of the HS method, it is approximated to a wind model in a fog scene, but is distributed in a slightly high speed region in a snowfall scene, and it may be determined that the motion cannot be detected accurately depending on the difference in weather. it can. The deviation between the estimation result obtained in the present embodiment and the conventional method is not only capable of detecting a traveling vehicle with high accuracy, but also enabling estimation of a large estimation error for the movement of fog and snow around. is there.

  Finally, the motion detection device described in the present embodiment is applicable not only in the case of fog or snow but also in the case of rain or fine weather, and has a difference in the obtained effect. is not. The applicable range is applicable not only to the fields of fluid dynamics and computer vision, but also to general monitoring such as human flow and traffic flow, and disaster monitoring.

  According to the present embodiment, based on the anisotropic diffusion function indicating the luminance diffusion phenomenon, the exponential function indicating the luminance irregular reflection, and the sine function indicating the luminance wave, the velocity vector to be moved is used as a variable. Using the objective function that derives the brightness variation equation and substitutes the brightness variation equation and the constraint formula based on the parallel mobility of the moving object into the robust function based on the robust statistics, from the gray value of two consecutive images Since the speed vector of the moving object is estimated, the movement of the moving object can be detected stably and accurately even in bad weather.

  According to the present embodiment, two images are divided into the same number of sub-blocks, the velocity vector is estimated for each sub-block, and the two images are different from the number of sub-blocks. It is repeatedly performed to divide into a number of sub-blocks and estimate the velocity vector. Based on the estimated velocity vector, the energy distribution of the movement target is obtained, and this energy distribution and a predetermined energy distribution Since the similarity is determined in order from the energy distribution of the image with the smaller number of sub-blocks, and the similarity reaches a predetermined level, the velocity vector of the image divided into the number of sub-blocks having this energy distribution is displayed. Therefore, it is possible to detect the movement of the moving object more appropriately. In general image processing, the number of sub-blocks is determined empirically, but in the present embodiment, the motion detection device 10 can also determine an image with the number of sub-blocks approximated to a predetermined energy distribution. It becomes.

  According to the present embodiment, the speed vector estimation unit 130 uses the Lorentz function as the robust function used when calculating the objective function, so the calculation speed of the objective function can be improved.

It is the block diagram which showed the motion detection apparatus which concerns on embodiment of this invention. It is a figure which shows the image of a foggy scene and a snowfall scene. It is the figure which displayed the velocity vector which estimated the image of two types of weather shown in FIG. 2 with the conventional method with the arrow. It is the figure which displayed the speed vector which estimated the image of two types of weather shown in FIG. 2 in this Embodiment with the arrow. It is the figure which displayed the direction (angle) which estimated the image of the snowfall scene in this Embodiment. It is the figure which displayed the direction (angle) which estimated the image of the snowfall scene by the conventional method. It is a figure showing the occurrence probability of the pixel with respect to the average value of the angle which the velocity vector of 4 points | pieces mutually makes in the velocity vector which estimated the image of the snowfall scene in this Embodiment. It is the figure which displayed the averaged diffusion coefficient distribution which estimated the image of the snowfall scene in this Embodiment. It is the figure which displayed the averaged index coefficient distribution which estimated the image of the snowfall scene in this Embodiment. It is a figure which shows the occurrence probability of the energy with respect to the speed estimated in this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Motion vector detection apparatus 100 ... Image input part 110 ... Image storage part 120 ... Objective function derivation part 130 ... Speed vector estimation part 140 ... Energy distribution derivation part 150 ... Determination part 160 ... Display part S1-S7 ... Step

Claims (9)

An image input means for inputting an image including the weather;
Image storage means for storing the input image in time series;
Time variation of the luminance I of position on the image at time t _{(= ∂I / ∂t), 1} λ was modeled that luminance is diffused anisotropically _{I XX + 2λ 2 I XY +} λ 3 I YY ( where , I _XX , I _XY , and I _YY are the second-order derivatives of luminance in the horizontal direction, the oblique direction, and the vertical direction, respectively , and λ ₁ , λ ₂ , and λ ₃ are the diffusion coefficients in the horizontal direction, the oblique direction, and the vertical direction, respectively. square spread function and, -κI (although, kappa exponential coefficient) that models that luminance is irregularly reflected and exponential ^{functions, Isinθ · cosθ + I 2 sinθ} 2 · cosθ 2 where the brightness is modeled to change the wave shape (Where θ is the angle of the wave) and a sine function, and using the anisotropic diffusion function, the exponential function, and the sine function, | I _t + I _X u + I _Y v− (λ ₁ I _XX + 2λ _{2 I XY + λ 3 I YY)} + κI + Is inθ · cos θ + I ² sin θ ² · cos θ ² | (where (u, v) is the velocity vector to be moved), and derives the luminance variation equation e _0, and calculates the velocity vector of the position on the image and the position near the position. Deriving an objective function by deriving an expression e ₁ with a constraint that the velocity vector is parallel and substituting the expression e ₀ and the expression e ₁ into a robust function based on robust statistics Means,
Speed vector estimation means for reading two consecutive images from the image storage means, substituting the luminance values of the two images into the objective function, and estimating the speed vector;
Display means for displaying the estimated velocity vector;
A motion detection apparatus comprising: The velocity vector estimation means divides the two images into the same number of sub-blocks, estimates the velocity vector for each sub-block, and further determines the two images as the number of sub-blocks. Dividing the number of sub-blocks into different numbers and estimating the velocity vector repeatedly,
Energy distribution deriving means for obtaining an energy distribution of the moving object based on the estimated velocity vector;
Determination means for sequentially determining the degree of similarity between the energy distribution and the predetermined energy distribution from the energy distribution of an image having a small number of sub-blocks;
The motion detection according to claim 1, wherein the display unit displays a velocity vector of an image divided into the number of sub-blocks having the energy distribution when the similarity reaches a predetermined level. apparatus.   The motion detection apparatus according to claim 1, wherein the velocity vector estimation unit uses a Lorentz function as the robust function used when calculating the objective function.   The motion detection apparatus according to claim 2, wherein the predetermined energy distribution is a Kolmogorov energy distribution. A first step of inputting an image including weather;
A second step of storing the input image in the image storage means in time series;
Time variation of the luminance I of position on the image at time t _{(= ∂I / ∂t), 1} λ was modeled that luminance is diffused anisotropically _{I XX + 2λ 2 I XY +} λ 3 I YY ( where , I _XX , I _XY , and I _YY are the second-order derivatives of luminance in the horizontal direction, the oblique direction, and the vertical direction, respectively , and λ ₁ , λ ₂ , and λ ₃ are the diffusion coefficients in the horizontal direction, the oblique direction, and the vertical direction, respectively. square spread function and, -κI (although, kappa exponential coefficient) that models that luminance is irregularly reflected and exponential ^{functions, Isinθ · cosθ + I 2 sinθ} 2 · cosθ 2 where the brightness is modeled to change the wave shape (Where θ is the angle of the wave) and a sine function, and using the anisotropic diffusion function, the exponential function, and the sine function, | I _t + I _X u + I _Y v− (λ ₁ I _XX + 2λ _{2 I XY + λ 3 I YY)} + κI + Is inθ · cos θ + I ² sin θ ² · cos θ ² | (where (u, v) is the velocity vector to be moved), and derives the luminance variation equation e _0, and calculates the velocity vector of the position on the image and the position near the position. A third equation for deriving an objective function obtained by substituting the equation e ₀ and the equation e ₁ into a robust function based on robust statistics is derived by deriving an equation e ₁ with a constraint that the velocity vector is parallel . Steps,
A fourth step of reading two consecutive images from the image storage means, substituting the luminance values of the two images into the objective function, and estimating the velocity vector;
A fifth step of displaying the estimated velocity vector;
A motion detection method comprising: The fourth step divides the two images into the same number of sub-blocks, estimates the velocity vector for each sub-block, and further determines the two images as the number of sub-blocks. Dividing the number of sub-blocks into different numbers and estimating the velocity vector repeatedly,
Obtaining an energy distribution of the moving object based on the estimated velocity vector;
Further determining the degree of similarity between the energy distribution and the predetermined energy distribution in order from the energy distribution of an image with a small number of sub-blocks,
The said 5th step displays the velocity vector of the image divided | segmented into the number of subblocks which have the said energy distribution, when the said similarity reaches a predetermined level. Motion detection method.   The motion detection method according to claim 5, wherein the fourth step uses a Lorentz function as the robust function used when calculating the objective function.   The motion detection method according to claim 6, wherein the predetermined energy distribution is a Kolmogorov energy distribution.   9. A motion detection program for causing a computer to execute each step in the motion detection method according to claim 5.