JP6351917B2 - Moving object detection device - Google Patents

Description

  The present invention relates to a moving object detection apparatus that estimates the number and position of moving objects, a moving speed including a moving direction, and the like.

Conventionally, a technique for estimating the number, position, or moving direction of a moving object such as a pedestrian within a sensor observation range using various sensors is known.
In order to estimate the number, position, or moving direction of moving objects, for example, by observing an object passing through a passage by a plurality of cameras, and based on the displacement of the moving object image on the image captured by each camera, There are methods for estimating the number, position, etc.
However, in this way, in the method of estimating the number and positions of moving objects using a plurality of cameras, the estimation accuracy is significantly degraded when a low-resolution camera is used. In addition, for example, when a visible light camera is used at night and the reflection intensity of light by a moving object is low, the shape of the moving object cannot be obtained correctly from the image, and the estimation accuracy is significantly deteriorated. In addition, for example, when the shape of the moving object changes with time, such as when a pedestrian changes the face direction, the estimation accuracy deteriorates.

As another method for estimating the number, position, or moving direction of moving objects, for example, by observing an object with a plurality of laser sensors and integrating observation information of each laser sensor on the same coordinate, There are methods for estimating the number, position, etc.
As described above, in the method of detecting a moving object using a plurality of laser sensors, in order to estimate the moving direction of the moving object, a sufficiently wide range compared to the moving range of the moving object per time frame is observed by the laser sensor. It is assumed that. However, since a laser sensor that projects laser light has a narrower observable range per unit time than a camera, the smaller the number of laser sensors, the narrower the observation range, and as a result, the estimation accuracy of the moving direction is significantly degraded.

Regarding the above-described problem, Patent Document 1 discloses that when a person passes through a slit light irradiation position using a laser that irradiates slit light on a passage and an imaging device that observes reflected light of the slit light. Discloses a technique for estimating the number, position, or traveling direction of a person from a change in luminance.
In the technique disclosed in Patent Literature 1, when one imaging device and one laser arranged in the vicinity of the imaging device are used, the number of passages of a person in the passage can be counted. In addition, when one imaging device and lasers arranged at a plurality of locations in the vicinity of the imaging device are used, it is possible to count the number of passes for each direction of person movement.

JP 2005-25593 A

  Since the range in which the flow of a moving object such as a pedestrian needs to be observed may be wide, it is desirable that the moving direction detection device be low-cost.

  According to the technique disclosed in Patent Document 1, in order to estimate the number of moving objects in each moving direction, there is a problem that it is necessary to arrange lasers in at least two places.

  The present invention has been made to solve the above problems, and accurately estimates the number, position, and moving direction of a moving object by using one camera and one laser sensor installed at one place. It is an object of the present invention to provide a moving object detection device that can do this.

  The moving object detection device according to the present invention is based on an object position data creation unit that calculates a position coordinate of a moving object based on distance measurement information measured by a distance sensor, and on video information captured by a video acquisition sensor. A pixel movement direction data creation unit that calculates the estimated movement direction of each area on the video, and a position movement direction correlation unit that estimates the number of moving objects and the movement direction based on the position coordinates and the estimated movement direction. is there.

  According to the present invention, it is possible to accurately estimate the moving speed including the number, position, and moving direction of moving objects by using one camera and one laser sensor installed at one place.

It is a block diagram of the moving object detection system provided with the moving object detection apparatus which concerns on Embodiment 1 of this invention. It is a figure for demonstrating an example of the installation condition of a ranging sensor and a video acquisition sensor in this Embodiment 1, and the observation conditions of a ranging sensor and a video acquisition sensor. It is a figure for demonstrating an example of the installation condition of a ranging sensor and a video acquisition sensor in this Embodiment 1, and an example of the observation conditions of a ranging sensor and a video acquisition sensor, and FIG. It is the bird's-eye view seen from. It is a block diagram of the moving object detection apparatus which concerns on Embodiment 1 of this invention. 5A and 5B are diagrams showing an example of the hardware configuration of the moving object detection device according to Embodiment 1 of the present invention. It is a flowchart explaining operation | movement of the moving object detection apparatus which concerns on Embodiment 1 of this invention. It is a conceptual diagram explaining the speed estimation operation | movement of the moving object in step ST603 of FIG. It is a block diagram of the moving object detection apparatus which concerns on Embodiment 2 of this invention. It is a flowchart explaining operation | movement of the moving object detection apparatus which concerns on Embodiment 2 of this invention. FIG. 10 is a conceptual diagram illustrating an operation of setting a speed hypothesis by a moving direction hypothesis setting unit in step ST904 of FIG. 9, taking the case of i = 1 as an example. It is a flowchart explaining the operation | movement of step ST904 of FIG. 9 in detail. FIG. 10 is a conceptual diagram showing an operation of updating the likelihood of speed hypotheses v ₀ , v ₁ , v ₂ by the moving direction determination unit in step ST 909 of FIG. 9, taking the case of j = 1 and j = 2 as an example. It is a flowchart explaining the detailed operation | movement of step ST909 of FIG. It is a block diagram of the moving object detection apparatus which concerns on Embodiment 3 of this invention. It is a flowchart explaining operation | movement of the moving object detection apparatus which concerns on Embodiment 3 of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1 FIG.
In the following description, as an example, it is assumed that the moving object detection device 100 according to Embodiment 1 of the present invention is applied to a pedestrian moving in a path extending in one direction as a detection target. In other words, the following description will be made assuming that a pedestrian is detected as a moving object.

FIG. 1 is a configuration diagram of a moving object detection system including a moving object detection device 100 according to Embodiment 1 of the present invention.
As shown in FIG. 1, in the moving object detection system, the moving object detection device 100 is connected to a distance measuring sensor 200 and a video acquisition sensor 300 via a network.
The distance measuring sensor 200 is, for example, a laser distance meter, a radar, an ultrasonic sensor, an active stereo sensor, or the like, and scans within one time frame of an image captured by the image acquisition sensor 300 to acquire distance measurement data.
The image acquisition sensor 300 is a visible light camera, an infrared camera, or the like, and is installed so as to look down the passage for detecting a pedestrian, and images the passage.
The moving object detection device 100 is based on the distance measurement information in the distance measurement sensor observation range in the time frame acquired from the distance measurement sensor 200 and the image in the image acquisition sensor observation range in the time frame acquired from the image acquisition sensor 300. The position of the pedestrian moving along the passage, the moving speed including the moving direction, etc. are estimated.

Note that one distance measuring sensor 200 and one image acquisition sensor 300 form one set, and the one set of distance measurement sensor 200 and image acquisition sensor 300 is installed for one moving object detection target region. Is done.
In addition, a plurality of sets including the one distance measuring sensor 200 and one image acquisition sensor 300 are connected to one moving object detecting device 100 via a network, and the moving object detecting device 100 has one distance measuring method. For each set of the sensor 200 and one image acquisition sensor 300, a process for estimating the position, moving speed, etc. of a pedestrian moving through the passage can be performed. Specific processing contents will be described later.

2 and 3 are diagrams for explaining an example of the installation status of the distance measurement sensor 200 and the image acquisition sensor 300 and the observation conditions of the distance measurement sensor 200 and the image acquisition sensor 300 in the first embodiment. is there. FIG. 3 is an overhead view of FIG. 2 viewed from the positive direction of the z-axis.
2 and 3, the x-axis direction is the direction in which the passage extends, the z-axis direction is the direction perpendicular to the ground, the y-axis direction is perpendicular to the direction in which the passage extends and is parallel to the ground, and the x-axis, y-axis, The positive direction of the z-axis is a right-handed orthogonal coordinate system.

As shown in FIGS. 2 and 3, the image acquisition sensor 300 and the distance measuring sensor 200 are installed so as to look down the passage.
The image acquisition sensor 300 acquires the image of the passage at a preset viewing angle, and the range obtained by projecting the installation position of the image acquisition sensor 300 onto the xy plane on the passage is the observation range 53 of the image acquisition sensor 300. And
The distance measuring sensor 200 scans the y-axis direction at least once in one time frame to acquire distance measurement data, and is one-dimensional on the xy plane parallel to the y-axis from the position where the distance measuring sensor 200 is installed. A range projected onto the line is set as an observation range 54 of the distance measuring sensor 200.
The observation range 53 of the image acquisition sensor 300 and the observation range 54 of the distance measurement sensor 200 partially overlap, and the width of the observation range 54 of the distance measurement sensor 200 in the x-axis direction is the observation of the image acquisition sensor 300. It is sufficiently narrower than the width of the range 53 in the x-axis direction.
The one-dimensional line that is the observation range of the distance measuring sensor 200 may be selectable according to the observation environment. For example, the distance measuring sensor 200 may observe a plurality of one-dimensional lines and select a one-dimensional line that is less likely to be blocked by an obstacle or the like as an observation range from each one-dimensional line.

FIG. 4 is a configuration diagram of the moving object detection device 100 according to Embodiment 1 of the present invention.
As shown in FIG. 4, the moving object detection apparatus 100 includes a distance measurement data acquisition unit 11, an object position data creation unit 12, a video acquisition unit 13, a pixel movement direction data creation unit 14, and a position movement direction correlation unit. 15, a display unit 16, and a recording unit 17.
The distance measurement data acquisition unit 11 and the object position data creation unit 12 constitute a distance measurement sensor processing unit 101, and the video acquisition unit 13 and the pixel movement direction data creation unit 14 constitute a video acquisition sensor processing unit 102. The position movement direction correlation unit 15 constitutes a fusion processing unit 103.

A command is sent to the distance measurement data acquisition unit 11 and the distance measurement sensor 200, and distance measurement information of the distance measurement sensor observation range in the latest time frame is acquired from the distance measurement sensor 200. The distance measurement data acquisition unit 11 outputs the acquired distance measurement information to the object position data creation unit 12.
The object position data creation unit 12 identifies a time frame in which the center of gravity position of the moving object has passed through the observation range 54 of the distance measurement sensor 200 based on the distance measurement information acquired from the distance measurement data acquisition unit 11 in a plurality of time frames. . Further, the object position data creation unit 12 calculates the gravity center position of the moving object in the x-axis and y-axis directions from the plurality of time frames acquired from the distance measurement data acquisition unit 11, and the object position data representing the gravity center position. Create
The object position data creation unit 12 outputs the time frame when the center of gravity of the moving object passes, that is, the passing time of the moving object in the observation range 54 of the distance measuring sensor 200 and the object position data to the position movement direction correlation unit 15. .
In this example, the apparatus calculates the center of gravity position of the entire moving object as an example of the physical quantity representing the position coordinates of the moving object. However, the position coordinates of a part of the moving object may be used instead of the center of gravity position. For example, it is good also as a position of the both shoulders with respect to a pedestrian, and the position of the wheel with respect to a motor vehicle.

The video acquisition unit 13 sends a command to the video acquisition sensor 300 and acquires the video in the observation range 53 of the video acquisition sensor 300 in the latest time frame from the video acquisition sensor 300. The video acquisition unit 13 outputs the acquired video to the pixel movement direction data creation unit 14.
The pixel movement direction data creation unit 14 calculates the estimated speeds in the x-axis and y-axis directions for each pixel based on the videos in the plurality of time frames acquired from the video acquisition unit 13, and generates pixel speed data representing the estimated speeds. Create for each time frame. The pixel movement direction data creation unit 14 outputs the pixel velocity data to the position movement direction correlation unit 15.
Here, as an example of a device that calculates the estimated speed for each region on the video, the pixel movement direction data creation unit 14 that is a device that calculates the estimated speed for each pixel is used, but the estimated speed on the video is calculated. It is not always necessary to select the region for each pixel. For example, the processing load may be reduced by calculating the estimated speed for each area of 10 pixels vertically and 10 pixels horizontally.
Here, the pixel movement direction data creation unit 14 calculates the velocity in the x-axis and y-axis directions as an example of the physical quantity representing the movement direction of the object, but the purpose is to estimate only the movement direction of each moving object. In, it is not always necessary to calculate the speed. For example, the pixel movement direction data creation unit 14 may calculate a vector having a constant size that represents only the movement direction or an azimuth angle that represents the movement direction.

The position movement direction correlation unit 15 receives the object position data output from the object position data creation unit 12 and the pixel velocity data output from the pixel movement direction data creation unit 14, and receives the moving object specified by the object position data. Estimate the moving speed.
The position movement direction correlator 15 is data representing the time at which the center of gravity position passes through the observation range 54 of the distance measuring sensor 200, the center of gravity position on the x axis and the y axis, and the estimated speed in the x axis and y axis directions for the moving object Is displayed on the display unit 16. The position movement direction correlator 15 also determines, for each moving object, the time when the centroid position passes the observation range 54 of the distance measuring sensor 200, the centroid position on the x and y axes, and the estimated speed in the x and y axis directions. Is stored in the recording unit 17.
Here, for each moving object, the time when the centroid position passes the observation range 54 of the distance measuring sensor 200, the centroid position on the x-axis and the y-axis, and data representing the estimated speed in the x-axis and y-axis directions are combined, This is called fusion data.

The display unit 16 is a display device such as a display, for example, and displays the fusion data output from the position movement direction correlation unit 15.
The recording unit 17 stores the fusion data output from the position movement direction correlation unit 15.
Here, as shown in FIG. 4, the display unit 16 and the recording unit 17 are included in the moving object detection device 100, but not limited thereto, the display unit 16 and the recording unit 17 are moved. It may be provided outside the object detection device 100, and the moving object detection device 100, the display unit 16, and the recording unit 17 may be connected via a network.

5A and 5B are diagrams showing an example of a hardware configuration of the moving object detection device 100 according to Embodiment 1 of the present invention.
In the first embodiment of the present invention, each function of the distance measuring sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103 is realized by the processing circuit 501. That is, the moving object detection device 100 includes a processing circuit 501 for performing control for displaying or storing information on the position, speed, and the like of the moving object based on the acquired distance measurement information and video information.
The processing circuit 501 may be dedicated hardware as illustrated in FIG. 5A or a CPU (Central Processing Unit) 507 that executes a program stored in the memory 506 as illustrated in FIG. 5B.

  When the processing circuit 501 is dedicated hardware, the processing circuit 501 includes, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), and an FPGA (Field-Programmable). Gate Array) or a combination thereof.

  When the processing circuit 501 is the CPU 507, the functions of the distance measurement sensor processing unit 101, the video acquisition sensor processing unit 102, and the fusion processing unit 103 are realized by software, firmware, or a combination of software and firmware. That is, the distance measurement sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103 are a CPU 507 that executes programs stored in an HDD (Hard Disk Drive) 502, a memory 506, and the like, a system LSI (Large- This is realized by a processing circuit such as Scale Integration. It can also be said that the programs stored in the HDD 502, the memory 506, and the like cause the computer to execute the procedures and methods of the distance measurement sensor processing unit 101, the video acquisition sensor processing unit 102, and the fusion processing unit 103. Here, the memory 506 is, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Memory), or the like. Or a volatile semiconductor memory, a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), or the like.

It should be noted that part of the functions of the ranging sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103 are realized by dedicated hardware, and part of them are realized by software or firmware. Also good. For example, the distance sensor processing unit 101 is realized by a processing circuit 501 as dedicated hardware, and the processing circuit 501 is stored in the memory 506 for the image acquisition sensor processing unit 102 and the fusion processing unit 103. The function can be realized by reading out and executing.
The display unit 16 is a display 503, for example.
For example, the recording unit 17 uses the HDD 502. This is only an example, and the recording unit 17 may be configured by a DVD, a memory 506, or the like.
In addition, the moving object detection device 100 includes an input interface device 504 and an output interface device 505 that communicate with external devices such as the distance measurement sensor 200 and the image acquisition sensor 300. For example, the distance sensor processing unit 101 uses the input interface device 504 such as USB (Universal Serial Bus) or Ethernet (registered trademark; description is omitted below) as the distance measurement information acquired by the distance sensor 200. get. In addition, for example, the video acquisition sensor processing unit 102 inputs video captured by the video acquisition sensor 300 using DVI (Digital Visual Interface (registered trademark)), HDMI (High-Definition Multimedia Interface (registered trademark)), USB, Ethernet, or the like. Obtained using the interface device 504. Further, for example, the fusion processing unit 103 connects and outputs information such as the number, position, and speed of moving objects to the display 503 using an output interface device 505 such as USB or Ethernet.

The operation of the moving object detection device 100 according to the first embodiment will be described.
FIG. 6 is a flowchart for explaining the operation of the moving object detection device 100 according to the first embodiment of the present invention.
Using FIG. 6 as an example, in the case where the moving speed is estimated for a moving object whose center of gravity has passed the observation range 54 of the distance measuring sensor 200 in the kth (k is a natural number) time frame from the start of observation. The operation of the moving object detection device 100 will be described.
Hereinafter, the kth time frame from the start of observation is denoted as time k. Further, it is assumed that the installation positions and observation ranges of the distance measurement sensor 200 and the video acquisition sensor 300 are preset and known, and a clock for measuring the observation times of the distance measurement sensor 200 and the video acquisition sensor 300 is a distance measurement. Assume that the sensor 200 and the image acquisition sensor 300 are synchronized.

The ranging sensor processing unit 101 extracts a moving object whose center of gravity has passed through the observation range 54 of the ranging sensor 200 at time k from ranging information of a plurality of time frames acquired by the ranging sensor 200. Object position data is created (step ST601). Specifically, the object position data creation unit 12 extracts a moving object that has passed through the observation range 54 of the distance measurement sensor 200 based on the distance measurement information of a plurality of time frames output from the distance measurement data acquisition unit 11. The time frame when the gravity center position of the moving object passes through the observation range of the distance measuring sensor 200 is specified, and the object position data is created. When a plurality of moving objects are extracted, the time frame is specified and the object position data is created for each moving object. The object position data creation unit 12 accumulates the distance measurement information of the time frame output by the distance measurement data acquisition unit 11 in a distance measurement information storage unit (not shown), and is stored in the distance measurement information storage unit. Object position data may be created based on a plurality of distance measurement information. The object position data creation unit 12 outputs the created object position data to the position movement direction correlation unit 15.
As a method for extracting the center of gravity position of the moving object from the distance measurement information of each time frame, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2011-108223 is used. The method disclosed in this document is a method for extracting the number and position of moving objects at a specific height. For example, when extracting the position of a pedestrian, the height of the pedestrian's head is extracted. The number and position of existing objects can be determined.
In addition, not only the said method but the object position data preparation part 12 may acquire the object position data of the time k produced beforehand. Specifically, for example, the object position data creation unit 12 does not create the object position data at time k in each case in step ST601, but stores the already created object position data. The object position data may be acquired.

The video acquisition sensor processing unit 102 creates pixel velocity data at time k from the video of each time frame acquired by the video acquisition sensor 300 (step ST602). Specifically, the pixel movement direction data creation unit 14 creates pixel velocity data based on the videos in a plurality of time frames output from the video acquisition unit 13. Note that the pixel movement direction data creation unit 14 causes the video information storage unit (not shown) to store the video of the time frame output from the video acquisition unit 13, and based on the video stored in the video information storage unit. In addition, pixel speed data may be created. The pixel movement direction data creation unit 14 outputs the created pixel velocity data to the position movement direction correlation unit 15.
As a method for creating pixel velocity data from the video of each time frame, for example, B.I. Lucas, T .; Kanade, “An Iterative Image Registration Technology with an Application to Stereo Vision,” Proceedings DARPA Image Understanding Working Shop. 121-130, April 1981. Is used. The method disclosed in this document is based on the assumption that the luminance intensity of each pixel is spatially continuous and that the motion of the object is linear. The velocity vector that minimizes is estimated. With this method, pixel velocity data is obtained in which the velocity vector of a pixel representing a stationary object has a value close to zero, and the velocity vector of a pixel representing a moving object has a value close to the moving velocity of the object.
In addition, not only the said method but the pixel moving direction data preparation part 14 may acquire the pixel velocity data of the time k produced beforehand. Specifically, for example, the pixel movement direction data creation unit 14 does not create pixel speed data at time k in each step ST602, but stores already created pixel speed data and stores the data. The acquired pixel speed data may be acquired.
Note that the pixel movement direction data creation unit 14 may create pixel velocity data only for a partial region on the video according to the observation environment. For example, if an area where a moving object cannot exist, such as an area where pedestrians cannot pass, is known from the data of the previous time frame or before data acquisition, the pixel speed of the area where the moving object cannot exist Creation of data may be omitted.

  The position movement direction correlation unit 15 uses the object position data at time k output from the object position data generation unit 12 in step ST601 and the pixel velocity data at time k output from the pixel movement direction data generation unit 14 in step ST602. Thus, the moving speed is estimated for the moving object specified by the object position data at time k, and fusion data at time k relating to the moving object is created (step ST603).

Here, FIG. 7 is a conceptual diagram illustrating the speed estimation operation of the moving object in step ST603 of FIG. In FIG. 7, it is assumed that two moving objects whose center of gravity has passed through the observation range 54 of the distance measuring sensor 200 at time k are extracted.
The position movement direction correlation unit 15 extracts a pixel velocity 62 within the object size 63 around the center of gravity 61 for the center of gravity 61 of each moving object at time k. Then, the position movement direction correlation unit 15 calculates the estimated speeds v ₁ and v ₂ of each moving object from the pixel speed within the object size 63.

Here, it is assumed that the x and y coordinates in the real space coincide with the x and y coordinates on the frame of the image obtained from the video acquisition sensor 300.
In the following, a method of calculating the estimated speed v ₁ = (v _{1, x} , v _{1, y} ) is taken as an example of a moving object in which the center of gravity position at time k is (x _d , y _d ) in the object position data. Will be explained. Here, x _d is the x-axis position of the center of gravity of the moving object, y _d is the y-axis position of the center of gravity of the moving object. Further, v _{1, x} and v _{1, y} are the x-axis direction component and the y-axis direction component of the estimated speed. Since the observation range 54 of the distance measuring sensor 200 is narrow in the x-axis direction, _xd may be the x-axis position at the center of the observation range of the distance measuring sensor 200.

Hereinafter, the pixel velocity data at the time k is expressed as V _x (x _i , y _i ) and V _y (x _i , y _i ). Here, x _i and y _i represent the x-axis center position and the y-axis center position of the i-th pixel, respectively, and V _x (x _i , y _i ) represents the x-axis direction component of the pixel velocity, V _y (x _i , Y _i ) represents the y-axis direction component of the pixel velocity. Incidentally, the total number of pixels is set to _{N p.}

ここで、Ｗ（ｘ_ｉ，ｙ_ｉ；ｘ_ｄ，ｙ_ｄ）は、物体サイズ６３を表す関数であり、任意の（ｘ_ｄ、ｙ_ｄ）に対して以下の式（３）を満たす関数である。

例えば、（ｘ_ｄ、ｙ_ｄ）を中心に円形の物体サイズ６３を定義する場合、以下の式（４）のように定義する。

ここで、物体サイズ６３の半径を表すパラメタをＲとし、以下の式（５）を満たす画素（ｘ_ｉ’、ｙ_ｉ’）の個数をｎ（Ｒ）とする。

なお、想定する移動物体の種類や、映像取得センサ３００における画素速度データの誤差および測距センサ２００の物体位置データの誤差に応じて、物体サイズ６３の大きさを変えたり、形状を楕円や長方形等にしてもよい。本実施の形態のように、歩行者を検出対象としている場合、物体サイズ６３は、上方から見下ろした歩行者の平均的な大きさを表すものとすればよい。
また、上記のように物体サイズ６３内部の画像速度データを一様に加算平均する他に、物体サイズ６３内で重み付け平均するように関数Ｗ（ｘ_ｉ、ｙ_ｉ；ｘ_ｄ、ｙ_ｄ）の値に傾斜を持たせてもよい。
また、物体位置データには静止物体や実在する物体以外の不要なデータも誤って含まれていることを前提に、推定速度を基に不要なデータを棄却してもよい。例えば、推定速度が閾値未満であれば静止物体として移動物体の個数から除いてもよい。すなわち、物体位置データ作成部１２で算出した移動物体の個数を、仮の移動物体個数とし、位置移動方向相関部１５において最終的な移動物体個数の推定をおこなうとしてもよい。The estimated velocity v ₁ of the moving object at the center of gravity position (x _d , y _d ) is calculated from the following equations (1) and (2).

Here, W (x _i, y _i ; x _d, y _d ) is a function representing the object size 63, and is a function that satisfies the following expression (3) for any (x _d , y _d ). is there.

For example, when the circular object size 63 is defined around (x _d , y _d ), it is defined as the following equation (4).

Here, R represents a parameter representing the radius of the object size 63, and n (R) represents the number of pixels (x _{i ′} , y _{i ′} ) satisfying the following expression (5).

Note that the size of the object size 63 is changed or the shape is changed to an ellipse or a rectangle according to the type of the moving object to be assumed, the error in the pixel velocity data in the image acquisition sensor 300, and the error in the object position data in the distance measuring sensor 200. You may make it. As in the present embodiment, when a pedestrian is the detection target, the object size 63 may represent the average size of the pedestrian looking down from above.
In addition to uniformly adding and averaging the image speed data inside the object size 63 as described above, the function W (x _i , y _i ; x _d , y _d ) is weighted averaged within the object size 63. The value may be inclined.
Further, it is possible to reject unnecessary data based on the estimated speed on the assumption that unnecessary data other than a stationary object or an actual object is erroneously included in the object position data. For example, if the estimated speed is less than the threshold value, it may be excluded from the number of moving objects as stationary objects. That is, the number of moving objects calculated by the object position data creation unit 12 may be used as a temporary moving object number, and the position moving direction correlation unit 15 may estimate the final number of moving objects.

Returning to the flow of FIG.
The position movement direction correlation unit 15 determines whether or not the estimated speed has been estimated for all the moving objects specified by the object position data at time k (step ST604).
If it is determined in step ST604 that the estimated speed has not been estimated for all moving objects specified by the object position data at time k (in the case of “NO” in step ST604), the process returns to step ST603, and thereafter Repeat the process.

  When it is determined in step ST604 that the estimated speed is estimated for all moving objects specified by the object position data at time k (in the case of “YES” in step ST604), the position movement direction correlation unit 15 For all the moving objects specified by the object position data at time k, for each moving object, the passing time, the center of gravity position, and the estimated speed estimated in step ST603 are combined, and the display unit 16 as fusion data, and It outputs to the recording part 17 (step ST605).

The display unit 16 displays information based on the fusion data output from the position movement direction correlation unit 15. The information displayed on the display unit 16 is, for example, the number of moving objects, the position of the center of gravity of each moving object, and the estimated speed. In addition, since the estimated speed includes information on the moving direction, the display unit 16 passes the observation range 54 of the distance measuring sensor 200 per unit time in the x positive direction or the x negative direction of the passage, for example. It is also possible to display the number of moved objects.
The information displayed on the display unit 16 may be determined based on preset conditions. Not limited to this, information to be displayed on the display unit 16 by the user from an input device (not shown) may be set in advance, or information to be displayed by the user during display. It can also be made switchable.

The recording unit 17 records the fusion data output from the position movement direction correlation unit 15. The information recorded by the recording unit 17 is, for example, the number of moving objects, the position of the center of gravity of each moving object, and the estimated speed. Further, for example, the number of moving objects that have passed through the observation range 54 of the distance measuring sensor 200 in the x positive direction or the x negative direction of the passage per unit time may be recorded in the recording unit 17.
The information to be recorded in the recording unit 17 may be determined based on preset conditions. However, the present invention is not limited to this, and information to be recorded in the recording unit 17 by an input device (not shown) may be set in advance, or the information to be recorded can be changed by the user. You can also

As described above, according to the first embodiment, the number, position, and movement of moving objects are achieved by one image acquisition sensor 300 and one distance measuring sensor 200 arranged at one detection location. The moving speed including the direction can be estimated.
In general, a distance measurement sensor equipped with both a light projecting element and a light receiving element has more components and more frequently corrects aging deterioration than an image acquisition sensor that requires only a light receiving element. , Operating costs are high. In the moving object detection device 100 according to the first embodiment, only one distance measuring sensor 200 such as a laser sensor is required for one detection point, so that the number is smaller than a method using a plurality of distance measuring sensors. The same function can be realized at a cost.

  Further, according to the first embodiment, since the moving speed including the moving direction of the moving object can be estimated by the light projecting element of one distance measuring sensor 200 installed at one place, slit light is projected at a plurality of places. There are fewer restrictions on the installation conditions and the cost of installation work can be reduced compared to the method of lighting.

  Further, according to the first embodiment, the number, position, and moving direction of a moving object can be estimated by the image acquisition sensor 300 and the distance measuring sensor 200 that scans only in a one-dimensional direction. In general, the distance measuring sensor 200 that scans in a one-dimensional direction requires less costs for manufacturing, installation work, and operation than the distance measuring sensor that scans in a two-dimensional direction. . Therefore, the same function can be realized at a lower cost compared to a method using a distance measuring sensor such as a laser sensor that scans in a two-dimensional direction.

  Further, since the distance measuring sensor 200 can also observe the shape of the moving object in the irradiation direction of the laser or the like irradiated by the distance measuring sensor 200, the position estimation error of the moving object is smaller than the image acquisition sensor 300, and It is easy to distinguish each moving object from a plurality of adjacent moving objects. Therefore, according to the first embodiment, the estimation accuracy of the number and positions of moving objects is higher than the method using only the video acquisition sensor 300 such as a visible light camera.

  Further, according to the first embodiment, the estimated speed for each pixel is calculated based on the luminance and intensity of the pixel for the image of the image acquisition sensor 300, and the moving direction of the moving object is estimated. Therefore, when the resolution of the image acquisition sensor 300 is low and the shape at the time of moving object observation is unclear, for example, when the pixel of the image is rough with respect to the size of the moving object or when a part of the outline of the moving object is missing, Compared to the method using only the video acquisition sensor 300 such as a visible light camera, the movement direction estimation accuracy is less deteriorated.

  Further, according to the first embodiment, since the number of moving objects is estimated based on the observation result of the distance measuring sensor 200, when the shadow of the moving object is reflected in the image or on the background screen When a moving image is being reproduced, the number of moving objects can be estimated without erroneously determining a change in luminance intensity with no substance in the background as a moving object.

Embodiment 2. FIG.
In the first embodiment, on the assumption that the luminance and intensity of the moving object are continuous on the video acquired by the video acquisition sensor 300, the center of gravity position of the moving object based on the distance measurement information from the distance measurement sensor 200, The pixel speed based on the video from the video acquisition sensor 300 is associated in the same time frame.
However, in reality, the luminance intensity of the moving object changes discontinuously due to, for example, a deviation in illuminance of the light source, a change in background color, rotation and deformation of the moving object, sensitization noise during night photographing, and the like. In such a case, the pixel speed of the moving object temporarily becomes a value greatly different from the speed of the true moving object, and the moving object may disappear on the pixel speed of a certain time frame.
Therefore, in this second embodiment, an embodiment will be described in which a plurality of candidates for the estimated speed of each moving object are set from the pixel velocities of a plurality of time frames, and a likely estimated speed is determined from each candidate. .

Since the configuration of the moving object detection system according to the second embodiment is the same as that described with reference to FIG. 1 in the first embodiment, a duplicate description is omitted.
In addition, the installation status of the distance measurement sensor 200 and the image acquisition sensor 300 and the observation conditions of the distance measurement sensor 200 and the image acquisition sensor 300 have also been described with reference to FIGS. Therefore, a duplicate description is omitted.

FIG. 8 is a block diagram of a moving object detection device 100a according to Embodiment 2 of the present invention.
In FIG. 8, the same components as those in the moving object detection device 100 according to Embodiment 1 described with reference to FIG.
The moving object detection device 100a according to the second embodiment of the present invention is different from the moving object detection device 100 according to the first embodiment in that the fusion processing unit 103 sets a moving direction hypothesis instead of the position movement direction correlation unit 15. The only difference is that the unit 18 and the movement direction determining unit 19 are configured.
In the second embodiment, the object position data creation unit 12 outputs the object position data to the movement direction hypothesis setting unit 18. Further, the pixel movement direction data creation unit 14 outputs the pixel velocity data to the movement direction hypothesis setting unit 18 and the movement direction determination unit 19.

The movement direction hypothesis setting unit 18 acquires the object position data output from the object position data generation unit 12 and the pixel velocity data of the time frame output from the pixel movement direction data generation unit 14, and is observed by the distance measuring sensor 200. A plurality of candidate values of the moving speed of the moving object thus set are set. Here, the candidate value of the moving speed of the moving object set by the moving direction hypothesis setting unit 18 is referred to as a speed hypothesis.
The movement direction hypothesis setting unit 18 outputs the speed hypothesis to the movement direction determination unit 19.

  The movement direction determination unit 19 acquires the pixel velocity data of the time frame output from the pixel movement direction data creation unit 14 and the velocity hypothesis of the moving object output from the movement direction hypothesis setting unit 18, and from among the velocity hypotheses. The moving speed of the moving object is determined. The moving direction determination unit 19 obtains fusion data representing the time at which the center of gravity of the moving object passes through the observation range 54 of the distance measuring sensor, the center of gravity on the x-axis and the y-axis, and the estimated speed in the x-axis and y-axis directions The data is output to the display unit 16 and the recording unit 17.

  Since the hardware configuration of the moving object detection device 100a is the same as that described with reference to FIGS. 5A and 5B in the first embodiment, a duplicate description is omitted.

The operation of the moving object detection device 100a according to the second embodiment will be described.
FIG. 9 is a flowchart for explaining the operation of the moving object detection device 100a according to the second embodiment of the present invention.
In the following description, k is a natural number equal to or greater than N + 1. The parameter N will be described later.

  The ranging sensor processing unit 101 extracts a moving object whose center of gravity has passed through the observation range 54 of the ranging sensor 200 at time k from ranging information of a plurality of time frames acquired by the ranging sensor 200. Object position data is created (step ST901). The specific operation is the same as step ST601 in FIG. 6 described in the first embodiment. The object position data creation unit 12 of the distance measurement sensor processing unit 101 outputs the created object position data to the movement direction hypothesis setting unit 18.

The control unit (not shown) of moving object detection apparatus 100a initializes variable i (step ST902). Here, the initial value of the variable i is 0.
Video acquisition sensor processing section 102 creates pixel velocity data at time ki from the video of a plurality of time frames acquired by video acquisition sensor 300 (step ST903). A specific method for creating pixel velocity data is the same as that in step ST602 of FIG. 6 described in the first embodiment. The pixel movement direction data creation unit 14 of the image acquisition sensor processing unit 102 outputs the pixel velocity data at time k−i to the movement direction hypothesis setting unit 18 and the movement direction determination unit 19.

  The movement direction hypothesis setting unit 18 includes the object position data at time k output from the object position data generation unit 12 in step ST901, and the pixel velocity data at time k-i output from the pixel movement direction data generation unit 14 in step ST903. Is used to set a speed hypothesis for the moving object included in the object position data at time k (step ST904).

Here, FIG. 10 is a conceptual diagram showing an operation of setting a speed hypothesis by the movement direction hypothesis setting unit 18 in step ST904 of FIG. 9 taking the case of i = 1 as an example.
The movement direction hypothesis setting unit 18 has an estimated value of the center of gravity position of the moving object at time k−1, that is, an object position estimated value 64 at time k−1, A pixel velocity 65 at time k−1 within the object size 63 centered on the object position estimated value 64 of k−1 is extracted. Then, the moving direction hypothesis setting unit 18 calculates a moving object speed hypothesis v ₁ from the extracted pixel speed 65 at time k−1.

FIG. 11 is a flowchart for explaining in detail the operation of step ST904 of FIG.
Here, the center of gravity position of a certain moving object as a target is (x _d , y _d ), and the velocity hypothesis is v _u = (v _{u, x} , v _{u, y} ). x _d is the x-axis position of the center of gravity of the moving object, y _d is the y-axis position of the center of gravity of the moving object, and v _{u, x} , v _{u, y} are the x-axis direction component of the estimated speed and the y-axis, respectively. The direction component. Since the observation range 54 of the distance measuring sensor 200 is narrow in the x-axis direction, xd may be the x-axis position at the center of the observation range 54 of the distance measuring sensor 200.

Also, the pixel velocity data at time k-i is _{represented as Vk-i} , x ( _xq , _yq ) and _Vk-i , y ( _xq , _yq ). Here, x _q and y _q represent the x-axis center position and the y-axis center position of the q-th pixel, respectively, and V _k−i , x (x _q , y _q ) are x-axis direction components of the pixel velocity. , V _k−i , y (x _q , y _q ) represent the y-axis direction component of the pixel velocity. Incidentally, the total number of pixels is set to _{N p.}

First, the movement direction hypothesis setting unit 18 selects one unselected pixel from the video acquired from the pixel movement direction data creation unit 14, that is, the video acquired by the video acquisition sensor 300 (step ST1101). Hereinafter, the pixel selected by the movement direction hypothesis setting unit 18 in step ST1101 is referred to as a “selected pixel”, and the x-axis center position and the y-axis center position of the selected pixel are (x _s , y _s ).

ここで、Ｗ（ｘ_ｑ，ｙ_ｑ；ｘ_ｓ，ｙ_ｓ）は、物体サイズ６３を表す関数であり、任意の（ｘ_ｓ，ｙ_ｓ）に対して以下の式（８）を満たす関数である。

例えば、（ｘ_ｓ，ｙ_ｓ）を中心に円形の物体サイズ６３を定義する場合、以下の式（９）のように定義する。

ここで、物体サイズ６３の半径を表すパラメタをＲとし、以下の式（１０）を満たす画素（ｘ_ｑ’，ｙ_ｑ’）の個数をｎ（Ｒ）とする。

また、想定する移動物体の種類や、映像取得センサ３００における画素速度データの誤差および測距センサ２００の物体位置データの誤差に応じて、物体サイズ６３の大きさを変えたり、形状を楕円や長方形等にしてもよい。本実施の形態のように、歩行者を検出対象としている場合、物体サイズ６３は、上方から見下ろした歩行者の平均的な大きさを表すものとすればよい。
また、上記のように物体サイズ６３内部の画像速度データを一様に加算平均する他に、物体サイズ６３内で重み付け平均するように関数Ｗ（ｘ_ｑ，ｙ_ｑ；ｘ_ｓ，ｙ_ｓ）の値に傾斜を持たせてもよい。Movement direction hypothesis setting unit 18, the pixel rate data at time k-i in the vicinity of the selected pixel, and calculates the average velocity _v m ₌ the selected pixel _{(v m, x, v m} , y) to (step ST1102). Here, v _{m, x} and v _{m, y} are the x-axis direction component and the y-axis direction component of the average speed.
The average velocity _{v m = (v m, x} , v m, y) is calculated from the following equation (6) and (7).

Here, W (x _q , y _q ; x _s , y _s ) is a function representing the object size 63, and is a function that satisfies the following expression (8) for any (x _s , y _s ). is there.

For example, when the circular object size 63 is defined around (x _s , y _s ), it is defined as the following formula (9).

Here, a parameter representing the radius of the object size 63 is R, and the number of pixels (x _{q ′} , y _{q ′} ) satisfying the following expression (10) is n (R).

Further, the size of the object size 63 is changed or the shape is changed to an ellipse or a rectangle according to the type of the moving object to be assumed, the error in the pixel velocity data in the image acquisition sensor 300, and the error in the object position data in the distance measuring sensor 200. You may make it. As in the present embodiment, when a pedestrian is the detection target, the object size 63 may represent the average size of the pedestrian looking down from above.
In addition to uniformly adding and averaging the image speed data inside the object size 63 as described above, the function W (x _q , y _q ; x _s , y _s ) is weighted and averaged within the object size 63. The value may be inclined.

  Based on the average speed calculated in step ST1102, the movement direction hypothesis setting unit 18 calculates a position where the selected pixel is extrapolated to the time k at the average speed (step ST1103). Here, the position where the selected pixel is extrapolated at time k to the average speed is referred to as an extrapolation position.

ここで、Ｔ_{ｋ−ｉ，ｋ}は時刻ｋ−ｉから時刻ｋのフレーム間経過時間を表す。特に、ｉ＝０の場合はＴ_ｋ，ｋ＝０とする。
また、式（１１）および式（１２）では、移動物体の運動モデルとして等速直進運動を仮定したが、移動物体の種類に応じて、例えば等加速度直進運動など別の運動モデルを仮定し外挿位置を算出してもよい。The moving direction hypothesis setting unit 18 assumes the extrapolated position (x _e , y _e ) and the moving object travels straight at a constant speed between time k-i and time k, and the following equations (11) and (12 ).

Here, T _{k-i, k} represents the elapsed time between frames from time k-i to time k. In particular, when i = 0, T _{k, k} = 0.
Also, in equations (11) and (12), constant-velocity linear motion is assumed as the motion model of the moving object. However, depending on the type of the moving object, another motion model such as constant-acceleration linear motion is assumed. The insertion position may be calculated.

ここで、行列右上のｔは行列の転置を表し、行列右上の「−１」は逆行列を表す。
また、σ_ｘ ^２、σ_ｙ ^２、σ_ｘｙ ^２は、外挿位置と移動物体の重心位置との残差に関する揺らぎの大きさを表すパラメタとし、σ_ｘ ^２はｘ軸成分の分散、σ_ｙ ^２はｙ軸成分の分散、σ_ｘｙ ^２はｘ成分とｙ成分の共分散を表す。例えば、画素速度データのｙ軸方向成分が、ｘ軸方向成分に比べ誤差が大きい場合、σ_ｙ ^２をσ_ｘ ^２よりも大きな値に設定することで、成分ごとの揺らぎの違いを考慮した残差を算出できる。
または、外挿位置と移動物体の重心位置との残差に関して揺らぎの数値化が困難な場合等では、各成分の揺らぎは独立かつ一様と仮定して、より簡単な、以下の式（１４）によって残差Δｒを求めてもよい。

The moving direction hypothesis setting unit 18 calculates the residual between the extrapolated position calculated in step ST1103 and the gravity center position of the moving object (step ST1104).
The moving direction hypothesis setting unit 18 extrapolates the position (x _e , y _e ) calculated from the pixel velocity data at time k−i and the center of gravity position (x _d , y _d ) of the moving object in the object position data at time k. Is calculated by the following equation (13).

Here, t in the upper right of the matrix represents transposition of the matrix, and “−1” in the upper right of the matrix represents an inverse matrix.
Also, σ _x ² , σ _y ² , and σ _xy ² are parameters representing the magnitude of fluctuation related to the residual between the extrapolated position and the center of gravity of the moving object, and σ _x ² is the variance of the x-axis component, σ _y ² represents the variance of the y-axis component, and σ _xy ² represents the covariance of the x component and the y component. For example, when the y-axis direction component of the pixel velocity data has a larger error than the x-axis direction component, by setting σ _y ² to a value larger than σ _x ^2, it is possible to consider the difference in fluctuation for each component. The difference can be calculated.
Or, in the case where it is difficult to quantify the fluctuation of the residual between the extrapolated position and the center of gravity position of the moving object, the fluctuation of each component is assumed to be independent and uniform, and a simpler equation (14 ) To obtain the residual Δr.

  The moving direction hypothesis setting unit 18 determines whether or not the residual Δr calculated in step ST1104 is minimum compared to other residuals already calculated in the loop processing of steps ST1101 to ST1107 ( Step ST1105).

  When it is determined in step ST1105 that it is not minimum compared to other residuals, that is, when the residual calculated in the immediately preceding step ST1104 is not temporarily minimum (in the case of “NO” in step ST1105), The process of step ST1106 is skipped and the process proceeds to step ST1107.

When it is determined in step ST1105 that it is minimum compared to other residuals, that is, when the residual calculated in the immediately preceding step ST1104 is provisionally minimum (in the case of “YES” in step ST1105), The moving direction hypothesis setting unit 18 sets the average velocity v _m calculated in step ST1102 as the velocity hypothesis v _u of the moving object whose center of gravity position at time k is (x _d , y _d ) (step ST1106). Note that if another value already been set as a speed hypothesis in the loop process of steps ST1101~ step ST 1107, the moving direction hypothesis setting unit 18 replaces the average velocity _{v m} calculated in step ST1102 as the new average speed.

  The moving direction hypothesis setting unit 18 determines whether or not all the pixels in the pixel velocity data at the time k-i have been selected (step ST1107).

  If it is determined in step ST1107 that all the pixels are not selected (in the case of “NO” in step ST1107), the process returns to step ST1101, and the subsequent processing is repeated.

  If it is determined in step ST1107 that all the pixels have been selected (in the case of “YES” in step ST1107), the processing in FIG. 11 ends.

As described above, the speed hypothesis of the moving object is set based on the pixel velocity data at time k-i by executing the processing of FIG.
In the process of FIG. 11 described above, the average hypothesis, the extrapolation position, and the residual position of the center of gravity of the moving object are calculated for all the pixels in the pixel velocity data at time k-i, and the velocity hypothesis is set. However, for example, the processing load may be reduced by limiting the pixels to be selected from the assumed maximum speed of the moving object.

  Further, in the process of FIG. 11 described above, only one speed hypothesis is set based on the pixel speed data at the time k-i, but the speed is calculated from the pixel speed data at the time k-i according to the size of the residual. A plurality of hypotheses may be set.

Further, in the processing of FIG. 11 described above, the case where the pixel movement direction data creation unit 14 calculates the velocity in the x-axis and y-axis directions has been described. However, the pixel movement direction data creation unit 14 calculates only the movement direction. For example, based on a parameter representing a typical moving speed of a pedestrian set in advance, the estimated moving direction may be converted into speeds in the x-axis and y-axis directions and processed in the same manner.
Here, the pixel movement direction data creation unit 14 calculates the velocity in the x-axis and y-axis directions as an example of the physical quantity representing the movement direction of the object, but the purpose is to estimate only the movement direction of each moving object. In, it is not always necessary to calculate the speed. For example, a vector having a constant size representing only the moving direction or an azimuth angle representing the moving direction may be calculated. In this case, the object speed hypothesis set by the movement direction hypothesis setting unit 18 is also referred to as a movement direction hypothesis.

Returning to the flowchart of FIG.
The control unit (not shown) of moving object detection device 100a determines whether variable i is greater than or equal to parameter N (step ST905).
The parameter N represents the number of past time frames to be referred to when setting the speed hypothesis. The parameter N sets, for example, the maximum value of the number of time frames for which the assumption that the target motion is going straight ahead at a constant speed from time k-N to time k is appropriate.

  If it is determined in step ST905 that the variable i is less than the parameter N (in the case of “NO” in step ST905), the movement direction hypothesis setting unit 18 adds 1 to the variable i (step ST906), and the process returns to step ST903.

  If it is determined in step ST905 that the variable i is equal to or greater than the parameter N (“YES” in step ST905), the control unit of the moving object detection device 100a initializes the variable j (step ST907). The initial value of the variable j is 1. That is, the control unit substitutes 1 for the variable j. The control unit outputs information indicating that the variable j has been initialized to the video acquisition sensor processing unit 102.

  Video acquisition sensor processing section 102 creates pixel velocity data at time k + j from videos of a plurality of time frames acquired by video acquisition sensor 300 (step ST908). A specific method for creating pixel velocity data is the same as that in step ST602 of FIG. 6 described in the first embodiment. The pixel movement direction data creation unit 14 of the video acquisition sensor processing unit 102 outputs the pixel velocity data at time k + j to the movement direction hypothesis setting unit 18 and the movement direction determination unit 19.

The moving direction determination unit 19 uses the pixel velocity data at time k + j acquired in step ST908, and the likelihood of each velocity hypothesis for the plurality of moving velocity hypotheses set by the moving direction hypothesis setting unit 18 in step ST904. Is updated (step ST909). Here, the value representing the likelihood of the moving object's speed hypothesis is called the speed hypothesis likelihood.
Note that the likelihood of each speed hypothesis is sequentially updated in the loop processing relating to the variable j in steps ST908 to ST911. For example, for a certain speed hypothesis v _i , the likelihood L _{i, k + j} updated by the pixel speed data at time k + j is also the likelihood L _{i, k + j−1} calculated by the loop processing of the previous variable j. And calculated.

FIG. 12 shows an operation of updating the likelihoods of the speed hypotheses v ₀ , v ₁ , v ₂ by the moving direction determination unit 19 in step ST909 of FIG. 9, taking the case of j = 1 and j = 2 as an example. It is a conceptual diagram.
The moving direction determination unit 19 calculates object position estimated values 66a to 66c at time k + 1 and object position estimated values 68a to 68c at time k + 2 based on each speed hypothesis with respect to the center of gravity position 61 of the moving object at time k. To do. Further, the moving direction determination unit 19 extracts the pixel velocities 67a to 67c and 69a to 69c at time k + 1 and time k + 2 within the object size 63 centered on each estimated object position value from the pixel velocity data at time k + 1 and time k + 2. To do.
Then, the moving direction determination unit 19 calculates the likelihood of the speed hypothesis from the difference between the speed hypothesis and the pixel speed within the object size 63 in each time frame. The likelihood represents the degree of matching between the speed hypothesis set from the pixel speed before time k and the pixel speed after time k + 1, and the pixels in the speed hypothesis and the object size 63 are passed through each time frame after time k + 1. The smaller the difference from the speed, the higher the value.
In the example of FIG. 12, the velocity hypothesis _{v 1,} and the pixel rate 67a near the object position estimate 66a, the difference between the pixel rate 69a of the peripheral object position estimates 68a, compared to the rest of the velocity hypothesis _v 0, _{v 2} Therefore, the likelihood of the speed hypothesis v ₁ is larger than the likelihoods of the speed hypotheses v ₀ and v ₂ .

Hereinafter, the operation of step ST909 in FIG. 9 will be described in detail.
FIG. 13 is a flowchart for describing detailed operation of step ST909 of FIG.
Here, the center of gravity position of a certain moving object as a target is (x _d , y _d ), and the velocity hypothesis is v _u = (v _{u, x} , v _{u, y} ). x _d is the x-axis position of the center of gravity of the moving object, y _d is the y-axis position of the center of gravity of the moving object, and v _{u, x} and v _{u, y} are the x-axis direction component, y-axis direction component of the estimated speed, To do. Note that since the observation range 54 of the distance measurement sensor 200 is narrow in the x-axis direction, _xd may be the center of the observation range 54 of the distance measurement sensor 200.
In addition, the pixel velocity data at time k + j is _represented as V _{k + j, x} (x _q , y _q ) and V _{k + j, y} (x _q , y _q ). Here, x _q and y _q represent the x-axis center position and the y-axis center position of the q-th pixel, V _{k + j, x} (x _q , y _q ) represents the x-axis direction component of the pixel velocity, V _{k + j, y} Let (x _q , y _q ) represent the y-axis direction component of the pixel velocity. Note that the total number of pixels is N _p .

ここで、Ｔ_{ｋ，ｋ＋ｊ}は時刻ｋから時刻ｋ＋ｊまでのフレーム間経過時刻を表す。
また、式（１５）および（１６）では、移動物体の運動モデルとして等速直進運動を仮定したが、移動方向決定部１９は、移動物体の種類に応じて、例えば、等加速度直進運動など別の運動モデルを仮定し外挿位置を算出してもよい。Moving direction determination unit 19, the center of gravity of the moving object at time k _{(x d, y} d) and the velocity hypothesis _{v u} from, the object position estimate at time k + j on the velocity hypothesis _{v u (x e, k +} j, y e, _{k + j} ) is calculated (step ST1301).
The moving direction determination unit 19 assumes that the object position estimated value (x _{e, k + j} , y _{e, k + j} ) is traveling at a constant speed straight from time k to time k + j, and the following equations (15) and (15) Calculate from (16).

Here, T _{k, k + j} represents the elapsed time between frames from time k to time k + j.
Further, in equations (15) and (16), a constant velocity linear motion is assumed as the motion model of the moving object. However, the moving direction determination unit 19 is different depending on the type of the moving object, such as a constant acceleration linear motion, for example. The extrapolation position may be calculated assuming the following motion model.

ここで、Ｗ（ｘ_ｑ，ｙ_ｑ；ｘ_{ｅ、ｋ＋ｊ}，ｙ_{ｅ，ｋ＋ｊ}）は、物体サイズ６３を表す関数であり、任意の（ｘ_{ｅ，ｋ＋ｊ}，ｙ_{ｅ，ｋ＋ｊ}）に対して、以下の式（１９）を満たす関数である。

例えば、（ｘ_{ｅ、ｋ＋ｊ}，ｙ_{ｅ、ｋ＋ｊ}）を中心に円形の物体サイズ６３を定義する場合、以下の式（２０）のように定義する。

ここで、Ｒは物体サイズ６３の半径を表すパラメタとし、ｎ（Ｒ）は、以下の式（２１）を満たす画素（ｘ_ｑ’，ｙ_ｑ’）の個数とする。

また、想定する移動物体の種類や、映像取得センサ３００における画素速度データの誤差及び測距センサ２００の物体位置データの誤差に応じて、物体サイズ６３の大きさを変えたり、形状を楕円や長方形等にしてもよい。本実施の形態のように、歩行者を検出対象としている場合、物体サイズ６３は、上方から見下ろした歩行者の平均的な大きさを表すものとすればよい。
また、上記のように物体サイズ６３内部の画像速度データを一様に加算平均する他に、物体サイズ６３内で重み付け平均するように関数Ｗ（ｘ_ｑ，ｙ_ｑ；ｘ_{ｅ，ｋ＋ｊ}，ｙ_{ｅ，ｋ＋ｊ}）の値に傾斜を持たせてもよい。The moving direction determination unit 19 calculates the average velocity v _{m, k + j} = (vm _{, k + j, x} , v) near the moving object at time k + j from the estimated object position calculated at step ST1301 and the pixel velocity data at time k + j. _{m, k + j, y} ) is calculated (step ST1302). v _{m, k + j, x} , v _{m, k + j, y} are the x-axis direction component and y-axis direction component of the average speed.
The moving direction determination unit 19 calculates the average speed v _{m, k + j} = (vm _{, k + j, x} , vm _{, k + j, y} ), for example, as in step ST1102 of FIG. ).

Here, W (x _q , y _q ; x _{e, k + j} , y _{e, k + j} ) is a function representing the object size 63, and for an arbitrary (x _{e, k + j} , y _{e, k + j} ), This is a function that satisfies the equation (19).

For example, when the circular object size 63 is defined around (x _{e, k + j} , y _{e, k + j} ), it is defined as the following equation (20).

Here, R is a parameter representing the radius of the object size 63, and n (R) is the number of pixels (x _{q ′} , y _{q ′} ) that satisfy the following expression (21).

Further, the size of the object size 63 is changed or the shape is changed to an ellipse or a rectangle according to the type of the moving object to be assumed, the error of the pixel velocity data in the image acquisition sensor 300, and the error of the object position data of the distance measuring sensor 200. You may make it. As in the present embodiment, when a pedestrian is the detection target, the object size 63 may represent the average size of the pedestrian looking down from above.
In addition to uniformly adding and averaging the image speed data inside the object size 63 as described above, the function W (x _q , y _q ; x _{e, k + j} , y _{e is used} so as to perform weighted averaging within the object size 63. _{, K + j} ) may have a slope.

ここで、速度仮説ｖ_ｕ，ｖ_{ｍ、ｋ＋ｊ}は以下の式（２３）（２４）で表される縦ベクトルとする。

S_ｋ＋ｊはｖ_ｕ−ｖ_{ｍ，ｋ＋ｊ}の残差共分散行列を表す２行２列のパラメタである。また、時刻ｋ＋ｊ−１の尤度Ｌ_{ｕ，ｋ＋ｊ−１}に関しては、特にｊ＝１の場合

とする。
なお、以上の尤度Ｌ_{ｕ，ｋ＋ｊ}は、平均速度ｖ_{ｍ，ｋ＋ｊ}が移動物体の真の速度に白色ガウス雑音が加算されたとするモデルに基づく値であり、平均速度の誤差分布が有色非ガウス雑音とするモデルを用いてもよい。The moving direction determination unit 19 uses the speed hypothesis v _u = (v _{u, x} , v _{u, y} ) and the average speed v _{m, k + j} calculated in step ST1302 (v _{m, k + j, x} , v _{m, k + j, From y} ), the likelihood L _{u, k + j} of the speed hypothesis is calculated (step ST1303).
The likelihood L _{u, k + j} of the speed hypothesis v _u at time k + j is calculated from the following equation (22).

Here, the velocity hypotheses v _u , v _{m and k + j} are vertical vectors represented by the following equations (23) and (24).

S _{k + j} is a 2-by-2 parameter representing the residual covariance matrix of v _u −v _{m, k + j} . In addition, regarding the likelihood L _{u, k + j−1 at} time k + j−1, particularly when j = 1.

And
The above likelihood L _{u, k + j} is a value based on a model in which the average velocity v _{m, k + j} is _obtained by adding white Gaussian noise to the true velocity of the moving object, and the error distribution of the average velocity is colored non-Gaussian. A model for noise may be used.

Movement direction determination unit 19 determines whether or not the hypothesis likelihood at time k + j has been updated for all speed hypotheses (step ST1304).
If it is determined in step ST1304 that the hypothesis likelihood at time k + 1 is not updated for all speed hypotheses (in the case of “NO” in step ST1304), the process returns to step ST1301, and the subsequent processing is repeated.

  If it is determined in step ST1304 that the hypothesis likelihood at time k + 1 has been updated for all speed hypotheses (in the case of “YES” in step ST1304), the processing in FIG. 13 ends.

  By executing the processing of FIG. 13 described above, the moving direction determination unit 19 updates the value representing the likelihood for each speed hypothesis of the moving object based on the pixel velocity data at time k + j.

Returning to the flowchart of FIG.
The control unit determines whether variable j is greater than or equal to parameter M (step ST910). The parameter M represents the number of time frames used for calculating the likelihood of the speed hypothesis. In the parameter M, for example, a maximum value of the number of time frames in which it is appropriate to assume that the target motion goes straight at a constant speed between time k and time k + M is set in advance.

  In step ST910, when the variable j is less than the parameter M (in the case of “NO” in step ST910), the moving direction determination unit 19 adds 1 to the variable j (step ST911), and returns to step ST908.

ここで、Ｐ（Ｈ_ｕ）は速度仮説ｖ_ｕの事前確率を表し、速度仮説ｖ_ｕごとに設定するパラメタである。
Ｐ（Ｈ_ｕ）の設定方法としては、例えば速度仮説ｖ_ｕを設定する際の式（１４）に関する残差Δｒや、ｖ_１の値の大きさ、画素速度データを取得した映像における光源や背景色等によって設定する。または、すべての速度仮説は設定時点では等しく尤もらしいと見なし、Ｐ（Ｈ_ｕ）を定数としてもよい。In step ST910, when the variable j is greater than or equal to the parameter M (in the case of “YES” in step ST910), the moving direction determination unit 19 calculates each speed hypothesis from the likelihood of each speed hypothesis calculated and updated in step ST909. A posteriori probability is calculated (step ST912).
Specifically, the moving direction determination unit 19 uses the likelihoods L _{u, k + M} (u = 1, 2,..., N _H ) of each speed hypothesis updated with the pixel speed data at time k + 1 to time k + M. The posterior probability P (H _u | V _{k: k + M} ) is calculated from the following equation (26) with respect to N _H is the number of velocity hypotheses.

Here, P _{(H u)} denotes the prior probability of rate hypotheses _{v u,} a parameter to be set for each speed hypothesis _{v u.}
As a setting method of P (H _u ), for example, the residual Δr related to the equation (14) when setting the speed hypothesis v _u , the magnitude of the value of v ₁ , and the light source and background in the video obtained from the pixel speed data Set by color. Alternatively, all speed hypotheses may be considered equally likely at the set time, and P (H _u ) may be a constant.

The moving direction determination unit 19 determines the speed hypothesis v _u having the maximum posterior probability as the estimated speed of the moving object from the posterior probabilities P (H ₁ | V _{k: k + M} ) of the speed hypothesis calculated in Step ST912 ( Step ST913).

The moving direction determination unit 19 determines whether or not the estimated speed has been determined for all moving objects among one or more moving objects specified by the object position data at time k (step ST914).
In step ST914, when the estimated speed has not been determined for all moving objects (in the case of “NO” in step ST914), the process returns to step ST902.
When the estimated speed is determined for all moving objects in step ST914 (in the case of “YES” in step ST914), the moving direction determining unit 19 determines the moving speed of the moving object determined in step ST913 and the gravity center position. Is output to the display unit 16 and the recording unit 17 (step ST915).

  In the above description, the object position data creation unit 12 estimates the number of moving objects. However, it is assumed that the object position data includes erroneous data including unnecessary data other than stationary objects and real objects. The direction determining unit 19 may reject unnecessary data based on the estimated speed. For example, if the estimated speed is less than the threshold value, it may be excluded from the number of moving objects as stationary objects. That is, the number of moving objects calculated by the object position data creation unit 12 may be used as a temporary moving object number, and the final moving object number may be estimated by the moving direction determination unit 19.

  In the above description, the fusion processing unit 103 sets the moving object speed hypothesis from the pixel speed data before time k and determines the speed of the moving object based on the pixel speed data after time k + 1. This is not the only way to choose a frame. For example, the fusion processing unit 103 may set a speed hypothesis based on pixel speed data after time k + 1 and may determine a moving speed of the moving object from the speed hypothesis based on pixel speed data before time k.

As described above, according to the second embodiment, since the moving speed of the moving object is estimated from the pixel speeds of a plurality of time frames, in the pixel speed data calculated from the video acquisition sensor 300, the moving object periphery Even if the pixel speed of the image can temporarily differ greatly from the true object moving speed, the number, position, and number of moving objects by one distance measuring sensor and one image acquisition sensor installed at one detection location. And the moving speed including a moving direction can be estimated.
This effect is effective, for example, when estimating the moving speed including the number, position, and moving direction of moving objects at night. The image acquisition sensor 300 such as a camera has a characteristic that the luminance of each pixel is more likely to fluctuate due to noise in the sensor in an environment where the illuminance of the light source is low such as at night or in a dark place. In such a case, since the luminance of the moving object fluctuates due to noise, the pixel velocity data tends to have a value deviating from the true velocity of the moving object. However, according to the configuration of the second embodiment, a candidate moving speed candidate value is set based on pixel speed data of a plurality of time frames, and each candidate value is compared to estimate a likely moving speed. Therefore, the probability that an erroneous moving speed is estimated can be reduced.

  In addition, the image acquisition sensor 300 that has less fluctuation in luminance intensity due to noise even in a low illumination environment generally has a more complicated manufacturing process, and thus the cost of the image acquisition sensor 300 is higher. Therefore, according to the second embodiment, it is epoch-making in that the same accuracy can be realized at a lower cost as compared with the method using the image acquisition sensor 300 which is not easily affected by noise in a low illumination environment.

  The above effect is also effective when, for example, a pedestrian is targeted as a type of moving object. Pedestrians tend to irregularly change the brightness of the object surface on the image as compared to inorganic materials such as vehicles due to movements of limbs and body orientation. Therefore, for example, when the pedestrian changes the orientation of the face, the pixel speed data tends to be a value that greatly differs from the moving speed of the object. However, according to the configuration of the second embodiment, the pixel speed at the time when the pixel speed data temporarily becomes unstable is rejected as being inconsistent with the pixel speed data of a plurality of time frames. Even when the luminance distribution of the image changes irregularly, the probability that an erroneous moving speed is estimated can be reduced.

  In addition, since there are garments worn by pedestrians, the luminance distribution is diversified compared to inorganic materials such as vehicles. Therefore, it is difficult in principle to select an environment in which the background and the pedestrian are not likely to be confused, and as a result, the pixel velocity data is the pedestrian movement speed in the time frame when the background luminance is the same as the pedestrian luminance. And can be very different values. However, according to the second embodiment, among the plurality of speed hypotheses, the speed hypothesis set from the pixel speed data of the time frame that is likely to be confused with the background is rejected as a hypothesis having low consistency with other time frames. Therefore, even when the pedestrian and the background are easily confused temporarily, the probability that an erroneous moving speed is estimated can be reduced.

  The above effect is also effective when there is a large bias in the luminance distribution on the video, for example, when the outdoor environment during clear weather or when the background includes a light source. If there is a large deviation in the luminance distribution on the image, such as the sun and shade in the outdoors, the continuity of the luminance distribution, which is the premise for calculating the pixel velocity, does not hold, so the pixel velocity data around the moving object Can take values that differ greatly from speed. However, according to the second embodiment, the velocity hypothesis estimated from the pixel velocity data of the time frame when the moving object passes through the region where the luminance distribution is greatly biased is a hypothesis that the consistency with the pixel velocity of other time frames is low. Therefore, even when the luminance distribution is largely biased, it is possible to reduce the probability that an erroneous moving speed is estimated.

Embodiment 3 FIG.
In the first and second embodiments, the number, position, and moving direction of the moving object are included from the barycentric position of the moving object based on the distance measurement information of the distance measuring sensor 200 and the pixel velocity based on the image of the image acquisition sensor 300. The movement speed was estimated.
In the third embodiment, based on the distance measurement information of the distance measuring sensor 200, data relating to the height of the moving object from the ground surface and the width of the moving object is created as attribute data of the moving object. A form is demonstrated.

Since the configuration of the moving object detection system according to the third embodiment is the same as that described with reference to FIG. 1 in the first embodiment, a duplicate description is omitted.
In addition, the installation status of the distance measurement sensor 200 and the image acquisition sensor 300 and the observation conditions of the distance measurement sensor 200 and the image acquisition sensor 300 have also been described with reference to FIGS. Therefore, a duplicate description is omitted.

FIG. 14 is a configuration diagram of a moving object detection device 100b according to Embodiment 3 of the present invention.
In FIG. 14, the same components as those of the moving object detection apparatus 100 according to Embodiment 1 described with reference to FIG.
The moving object detection device 100b according to the third embodiment of the present invention is different from the moving object detection device 100 according to the first embodiment in that the distance measuring sensor processing unit 101 includes the object height data generation unit 20 and the object width data generation. The only difference is that the unit 21 is further provided, and the fusion processing unit 103 is further provided with the object attribute determination unit 22.

The object height data creation unit 20 acquires distance measurement information in a plurality of time frames from the distance measurement data acquisition unit 11, and creates data in which the height of the moving object from the ground surface is estimated. Here, data obtained by estimating the height of the moving object from the ground surface is referred to as object height data.
The object height data creation unit 20 outputs the created object height data to the object attribute determination unit 22.
Note that the moving object in the object height data is associated with the moving object of the object position data created by the object position data creating unit 12, and the correspondence can be referred to from the object height data.

The object width data creation unit 21 obtains distance measurement information in a plurality of time frames from the distance measurement data obtaining unit 11, and creates data in which the widths of the moving object in the x-axis direction and the y-axis direction are estimated. Here, data in which the width of the moving object in the x-axis direction and the y-axis direction is estimated is referred to as object width data.
The object width data creation unit 21 outputs the created object width data to the object attribute determination unit 22.
Note that the moving object in the object width data is associated with the moving object in the object position data created by the object position data creating unit 12, and the correspondence can be referred to from the object width data.

The object attribute determination unit 22 acquires the position and moving speed data of the moving object from the position movement direction correlation unit 15, the object height data from the object height data creation unit 20, and the object width data from the object width data creation unit 21. The attribute of the moving object is determined based on the acquired data.
The object attribute determination unit 22 outputs data representing the position, moving speed, and attribute of the moving object to the display unit 16 and the recording unit 17. Here, data representing the position, moving speed, and attribute of the moving object output from the object attribute determination unit 22 is referred to as attributed fusion data.

  Since the hardware configuration of the moving object detection device 100b is the same as that described with reference to FIGS. 5A and 5B in the first embodiment, a duplicate description is omitted.

The operation of the moving object detection device 100b according to the third embodiment will be described.
FIG. 15 is a flowchart for explaining the operation of the moving object detection device 100b according to the third embodiment of the present invention.
In the following description, it is assumed that the moving speed and the attribute are estimated for a moving object whose center of gravity has passed the observation range 54 of the distance measuring sensor 200 at time k.

  The distance measurement data acquisition unit 11 and the object position data creation unit 12 of the distance measurement sensor processing unit 101 have the center of gravity position of the distance measurement sensor 200 at the time k based on the distance measurement information of the plurality of time frames acquired by the distance measurement sensor 200. A moving object that has passed through the observation range 54 is extracted, and object position data at time k is created (step ST1501). The specific operation is the same as step ST601 in FIG. 6 described in the first embodiment. The object position data creation unit 12 outputs the created object position data to the position movement direction correlation unit 15.

The object height data creation unit 20 of the ranging sensor processing unit 101 acquires ranging information of a plurality of time frames acquired by the ranging sensor 200 from the ranging data acquisition unit 11, and the center of gravity position is measured at time k. For a moving object that has passed through the observation range 54 of the sensor 200, the height of the moving object from the ground surface is calculated, and object height data at time k is created (step ST1502).
The object height data creation unit 20 uses, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2011-108223 described above as a method of calculating the height of the moving object from the distance measurement information of each time frame. In this method, ranging information of each moving object in the observation range 54 is divided in the height direction, and a range in which the object is detected at each height is calculated. For example, the height of a pedestrian is calculated. In this case, the maximum height at which the pedestrian is detected can be calculated as the pedestrian height.
The object height data creation unit 20 outputs the created object height data to the object attribute determination unit 22.

The object width data creation unit 21 of the ranging sensor processing unit 101 acquires ranging information of a plurality of time frames acquired by the ranging sensor 200 from the ranging data acquisition unit 11, and the center of gravity position at the time k is the ranging sensor. For a moving object that has passed 200 observation ranges 54, the widths of the moving object in the x and y directions are calculated, and object width data at time k is created (step ST1503).
The object width data creation unit 21 uses, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2011-108223 described above as a method of calculating the width of the moving object from the distance measurement information of each time frame. This method divides the ranging information of each moving object in the observation range 54 in the distance direction, and calculates the width of the pedestrian's body from the maximum value and the minimum value in the distance direction in which the moving object is detected in each distance direction. can do. Further, the width in the direction perpendicular to the distance direction can be calculated from the time when the moving object is continuously detected.
The object width data creation unit 21 outputs the created object width data to the object attribute determination unit 22.

  Video acquisition sensor processing section 102 creates pixel velocity data at time k from the videos of a plurality of time frames acquired by video acquisition sensor 300 (step ST1504). A specific method for creating pixel velocity data is the same as that in step ST602 of FIG. 6 described in the first embodiment. The pixel movement direction data creation unit 14 of the video acquisition sensor processing unit 102 outputs the pixel velocity data at time k to the position movement direction correlation unit 15.

The position movement direction correlation unit 15 uses the object position data at time k output from the object position data generation unit 12 in step ST1501 and the pixel velocity data at time k output from the pixel movement direction data generation unit 14 in step ST1504. Then, the estimated speed is estimated for the moving object specified by the object position data at time k, and fusion data at time k related to the moving object is created (step ST1505). The specific operation is the same as step ST603 in FIG. 6 described in the first embodiment.
The position movement direction correlation unit 15 outputs the created fusion data to the object attribute determination unit 22.

  The object attribute determination unit 22 includes the object height data at time k created by the object height data creation unit 20 at step ST1502, the object width data at time k created by the object width data creation unit 21 at step ST1503, and the step Based on the center of gravity position and the estimated speed of the moving object included in the fusion data at time k created by the position movement direction correlation unit 15 in ST1505, the attribute of the moving object, that is, the shape attribute is determined (step ST1506).

  For example, when a pedestrian is targeted as a moving object, the height of a moving object is less than the average height of an adult and the width of the moving object is less than the width of a typical adult body. When the speed is higher than the general walking speed, the object attribute determination unit 22 determines that the moving object has the shape attribute “child”. Information necessary for determining shape attributes such as average adult height, typical adult body width, and general walking speed information is set in advance and can be referred to by the object attribute determination unit 22 It is assumed that it is stored in a different place.

Also, for example, when a pedestrian is targeted as a moving object, the height of a moving object is less than the average height of an adult, and the width of the moving object near the ground is appropriate for a wheelchair wheel. If the movement speed is slower than the general walking speed, the object attribute determination unit 22 determines that the mobile object has the shape attribute “wheelchair user”.
In the above, an example for pedestrians has been given. However, the present invention is not limited to this, determination of vehicle types for vehicles, determination of models for aircraft, determination of ship types for ships, You may use for the determination etc. of the seed | species which made the animal object.

The object attribute determination unit 22 determines whether or not the estimated speed and shape attribute have been estimated for all of the one or more moving objects specified by the object position data at time k (step ST1507).
In step ST1507, when the estimated speed and the shape attribute are not estimated for all of one or more moving objects specified by the object position data at time k (in the case of “NO” in step ST1507), step Return to ST1505.

  In step ST1507, when the estimated speed and shape attribute are estimated for all of the one or more moving objects specified by the object position data at time k (in the case of “YES” in step ST1507), the object attribute The determination unit 22 determines the relative position of each moving object from the position, estimated speed, and shape attribute of each moving object at the time k estimated in Steps ST1505 to ST1506, and the fusion data in a time frame other than the time k. The attribute of the moving object based on the speed is determined (step ST1508). The attribute determined in step ST1508 is referred to as a group attribute.

  For example, when a pedestrian is targeted as a moving object, a moving object whose center of gravity has passed the observation range 54 of the distance measuring sensor at time k is used in a past or future time frame within a certain number from time k. When a certain number or more of moving objects in the same position and in the same moving direction are observed, the object attribute determining unit 22 determines that the moving object at the corresponding time k has the group attribute “one of group pedestrians”. .

Also, for example, when a pedestrian is targeted as a moving object, for a moving object whose shape attribute is “child”, a position in the vicinity of the moving object, the same moving speed, and a shape attribute other than “child” If a moving object has been observed, the group attribute of the moving object with the corresponding shape attribute “children” is set to “children with parents”, and the group attribute of the moving object with the corresponding shape attribute “non-children” is set to “children with children”. "Guardian of".
In the above, an example for pedestrians has been given. However, the present invention is not limited to this, but it is not limited to determination of traffic jams for vehicles, determination of formations for aircraft, determination of fleets for ships, animals You may use for the judgment etc. of the flock for the object.

  The object attribute determination unit 22 determines the position and moving speed of the moving object estimated in step ST1505, the shape attribute of the moving object determined in step ST1506, and the group attribute of the moving object determined in step ST1508. The data is output to the display unit 16 and the recording unit 17 as attached fusion data.

  In place of the position movement direction correlation unit 15 in FIG. 14, the movement direction hypothesis setting unit 18 and the movement direction determination unit 19 described in Embodiment 2 estimate the number, position, and movement speed of moving objects. Also good. In this case, the object attribute determination unit 22 determines the shape attribute and group attribute of the moving object based on the output of the movement direction determination unit 19.

Thus, according to the third embodiment, the distance measurement information of the distance measurement sensor 200, the position and movement speed of the moving object based on the image information of the image acquisition sensor 300, and the distance measurement of the distance measurement sensor 200. Since the attribute of the moving object is determined using the height and width of the moving object calculated from the information, when the resolution of the image acquisition sensor 300 is low or the contour of the moving object is low on the image acquisition sensor due to low illuminance. Even in the case of unclearness, it is possible to determine an attribute useful for analyzing a moving object.
Such a configuration is effective, for example, when estimating the proportion of children in pedestrians and the proportion of pedestrians acting as a group. The age structure of pedestrians or the presence or absence of group pedestrians that cause congestion is useful information in security systems, event management, market analysis, etc. In order to determine the attribute, it is necessary to extract the body part of the pedestrian from the contour on the video, and the accuracy of the determination deteriorates in the low-resolution video acquisition sensor. For example, in order to extract a child pedestrian from a video, it is necessary to extract the head position of the pedestrian from the video, which is difficult from a low resolution video.

  On the other hand, according to the third embodiment, each moving object obtained by fusing the height and width of the moving object acquired by the distance measuring sensor 200 and the observation results of the distance measuring sensor 200 and the image acquisition sensor 300. Therefore, even when the resolution of the image acquisition sensor 300 is low and the outline of the moving object is unclear, the probability of determining an erroneous attribute can be reduced.

  In addition, since the distance measuring sensor 200 can also observe the shape of the moving object in the depth direction, the probability of making an erroneous determination in attribute determination based on the position and speed of the moving object can be reduced compared to the image acquisition sensor 300.

  In the first embodiment, the moving object detection device 100 is configured as shown in FIG. 4, but the moving object detection device 100 includes the object position data creation unit 12, the pixel movement direction data creation unit 14, and the like. By providing the position movement direction correlation unit 15, the above-described effects can be obtained.

  In the above description, the case where a pedestrian is detected as a moving object has been described. However, the present invention also applies to a case where an automobile, a bicycle, a wheelchair, a railway vehicle, an aircraft, a ship, an animal other than a human, a robot, or the like is detected as a moving object. It is clear that can be used.

  Further, within the scope of the present invention, the invention of the present application can be freely combined with each embodiment, modified with any component in each embodiment, or omitted with any component in each embodiment. .

  The moving object detection device according to the present invention is configured so that the number, position, and moving direction of moving objects can be accurately estimated by one camera and one laser sensor installed at one place. The present invention can be applied to a moving object detection device that estimates the moving speed including the number, position, and moving direction of moving objects such as pedestrians.

  DESCRIPTION OF SYMBOLS 11 Distance data acquisition part, 12 Object position data preparation part, 13 Image | video acquisition part, 14 Pixel movement direction data preparation part, 15 Position movement direction correlation part, 16 Display part, 17 Recording part, 18 Movement direction hypothesis setting part, 19 Moving direction determination unit, 20 object height data creation unit, 21 object attribute determination unit, 53, 54 observation range, 61 object centroid position, 62, 65, 67a, 67b, 67c, 69a, 69b, 69c pixel velocity, 63 object Size, 66a, 66b, 66c, 68a, 68b, 68c Object position estimation value, 100, 100a, 100b Moving object detection device, 101 Distance sensor processing unit, 102 Video acquisition sensor processing unit, 103 Fusion processing unit, 200 Distance measurement Sensor, 300 video acquisition sensor, 501 processing circuit, 502 HDD, 503 display, 504 input interface Interface device, 505 output interface device, 506 memory, 507 CPU.

Claims (5)

An object position data creation unit that calculates position coordinates of a moving object based on distance measurement information measured by a distance sensor;
A pixel movement direction data creation unit that calculates an estimated movement direction of each region on the video based on video information captured by the video acquisition sensor;
A moving object detection device comprising: a position movement direction correlator for estimating the number of moving objects and the movement direction based on the position coordinates and the estimated movement direction. An object position data creation unit that calculates position coordinates of a moving object based on distance measurement information measured by a distance sensor;
A pixel movement direction data creation unit that calculates an estimated movement direction of each region on the video based on video information captured by the video acquisition sensor;
A moving direction hypothesis setting unit that sets a plurality of moving direction hypotheses for a moving object based on the position coordinates and the estimated moving direction;
A moving object detection device comprising: a moving direction determining unit that determines the number and moving direction of the moving objects based on the plurality of moving direction hypotheses. The moving direction determining unit
The moving object detection device according to claim 2, wherein likelihoods of the plurality of moving direction hypotheses are respectively calculated, and a moving direction of the moving object is determined based on the likelihoods. An object height data creation unit for calculating the height of the moving object based on the distance measurement information;
An object width data creation unit for calculating the width of the moving object based on the distance measurement information;
Attributes of the moving object based on the height and width of the moving object, the relative positional relationship between the moving objects when there are a plurality of moving objects, and the relative moving direction between the moving objects The moving object detection device according to claim 1, further comprising: an object attribute determination unit that determines An object height data creation unit for calculating the height of the moving object based on the distance measurement information;
An object width data creation unit for calculating the width of the moving object based on the distance measurement information;
Attributes of the moving object based on the height and width of the moving object, the relative positional relationship between the moving objects when there are a plurality of moving objects, and the relative moving direction between the moving objects The moving object detection device according to claim 2, further comprising: an object attribute determination unit that determines

Images