TW201804445A - Moving object detection device - Google Patents

Abstract

A moving object detection device is provided with: an object position data creation unit (12) for calculating positional coordinates of a moving object on the basis of ranging information measured by a ranging sensor (200); a pixel movement direction data creation unit (14) for calculating an estimated movement direction of each area on a video on the basis of video information captured by a video acquisition sensor (300); and a position movement direction correlation unit (15) for estimating the number of moving objects and movement directions thereof on the basis of the positional coordinates and the estimated movement direction.

Description

Moving object detection device

The present invention relates to a moving object detection device that estimates the number and position of moving objects, and the moving speed including the moving direction.

Conventionally, a technique is known in which various sensors are used to estimate the number, position, and moving direction of moving objects such as pedestrians within the sensor's observation range.

In estimating the number, position, or direction of moving objects, for example, there is a method of observing an object passing through a path by using a plurality of cameras, and estimating based on the deviation of the image of the moving object on the images captured by each camera. Number, location, etc.

However, in the method of estimating the number and position of moving objects by using multiple cameras in this way, and using a visible light camera at night, etc., the light reflection intensity caused by a moving object is low, and it cannot be accurately obtained from the image. The shape of a moving object significantly deteriorates the estimation accuracy. In addition, for example, when the shape of a moving object changes with time, such as when a pedestrian changes the direction of his face, the estimation accuracy is also deteriorated.

As another method for estimating the number, position, or direction of moving objects, for example, there are a plurality of laser sensors to observe the object, and the observation information of each laser sensor is combined on the same coordinate to estimate Moving object Number, position, etc.

In this method, a plurality of laser sensors are used to detect a moving object. In order to estimate the moving direction of the moving object, the laser sensor will be used to observe a wider range of movement than the moving object of the frame at each moment. As a prerequisite. However, the laser sensor that casts the laser light has a narrower observable range per unit time than the camera, so the fewer the number of laser sensors, the narrower the observation range. As a result, the accuracy of the estimation of the moving direction is significant. Worse.

Regarding the problems as described above, Patent Document 1 discloses a technology that uses a laser device that irradiates slit light on a path and an imaging device that observes the reflected light of the slit light to pass a person through the slit. The change in brightness at the light irradiation position estimates the number of people, the position, and the direction of travel.

In the technique disclosed in Patent Document 1, when a single imaging device and a laser disposed in the vicinity of the imaging device are used, the number of people passing through the path can be counted. In addition, in the case of using one imaging device and a plurality of lasers arranged in the vicinity of the imaging device, it is possible to count the number of passing people in each direction in which the person advances.

[Leading Patent Literature]

[Patent Literature]

[Patent Document 1] JP 2005-25593

Because it is necessary to observe the flow range of a moving object such as a pedestrian, it may be spread over a wide area, so it is better to have a low-cost moving direction detection device.

If it is based on the technology as disclosed in Patent Document 1, The number of moving directions of a moving object, etc., has a problem that lasers need to be arranged at at least two places.

The present invention was developed in order to solve the problems as described above, and an object thereof is to provide a camera and a laser sensor which can be installed at a place to estimate the number and position of moving objects with high accuracy And a moving object detection device at a moving speed.

The moving object detection device of the present invention includes: an object position data production unit that calculates the position coordinates of a moving object based on the ranging information measured by the borrowing range sensor; a pixel moving direction data production unit that acquires the senses based on the borrowed image The information of the image captured by the measuring device is used to calculate the estimated moving direction of each area on the image; and the position moving direction related section estimates the number of moving objects and the moving direction based on the position coordinates and the estimated moving direction.

According to the present invention, a camera and a laser sensor provided at one place can be used to estimate the number and position of moving objects and the moving speed including the moving direction with high accuracy.

11‧‧‧ Ranging data acquisition department

12‧‧‧ Object position data production department

13‧‧‧Image acquisition department

14‧‧‧ Pixel production direction data production department

15‧‧‧Position moving direction related department

16‧‧‧Display

17‧‧‧Recording Department

18‧‧‧ Movement direction hypothesis setting unit

19‧‧‧ Movement direction decision unit

20‧‧‧ Object Height Data Production Department

21‧‧‧Object attribute determination unit

53, 54‧‧‧ Observation range

61‧‧‧ Center of gravity

62, 65, 67a, 67b, 67c, 69a, 69b, 69c ‧‧‧ pixel speed

63‧‧‧Object size

66a, 66b, 66c, 68a, 68b, 68c‧‧‧Estimated object position

100, 100a, 100b ‧‧‧ mobile object detection device

101‧‧‧ ranging sensor processing unit

102‧‧‧Image acquisition sensor processing unit

103‧‧‧Fusion Processing Department

200‧‧‧ ranging sensor

300‧‧‧Image acquisition sensor

501‧‧‧processing circuit

502‧‧‧HDD

503‧‧‧ Display

504‧‧‧input interface device

505‧‧‧ output interface device

506‧‧‧Memory

507‧‧‧CPU

FIG. 1 is a configuration diagram of a moving object detection system including a moving object detection device according to a first embodiment of the present invention.

FIG. 2 is a diagram for explaining an example of the installation conditions of the ranging sensor and the image acquisition sensor and the observation conditions of the ranging sensor and the image acquisition sensor in the first embodiment.

FIG. 3 is a diagram for explaining an example of the installation conditions of the ranging sensor and the image acquisition sensor and the observation conditions of the ranging sensor and the image acquisition sensor in the first embodiment. FIG. The top view of FIG. 2 is seen from the positive direction of the z-axis.

Fig. 4 is a configuration diagram of a moving object detection device according to the first embodiment of the present invention.

5A and 5B are diagrams showing an example of a hardware configuration of a moving object detection device according to the first embodiment of the present invention.

Fig. 6 is a flowchart illustrating the operation of the moving object detection device according to the first embodiment of the present invention.

FIG. 7 is a schematic diagram illustrating a speed estimation operation of the moving object in step ST603 of FIG. 6.

Fig. 8 is a configuration diagram of a moving object detection device according to a second embodiment of the present invention.

Fig. 9 is a flowchart illustrating the operation of the moving object detection device according to the second embodiment of the present invention.

FIG. 10 is a schematic diagram showing the operation of setting the speed assumption value by the moving direction assumption value setting unit in step ST904 in FIG. 9 taking i = 1 as an example.

Fig. 11 is a flowchart illustrating the operation of step ST904 in Fig. 9 in detail.

FIG. 12 shows the case of j = 1 and j = 2 as an example, and shows the operation of updating the probability of the speed assumption values v ₀ , v ₁ , and v ₂ of the borrowing direction determining unit in step ST909 of FIG. 9. schematic diagram.

FIG. 13 is a flowchart illustrating detailed operations of step ST909 in FIG. 9.

Fig. 14 is a configuration diagram of a moving object detection device according to a third embodiment of the present invention.

Fig. 15 is a flowchart illustrating the operation of the moving object detection device according to the third embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First Embodiment

In the following description, as an example, a moving object detection device 100 according to the first embodiment of the present invention is applied to a pedestrian moving in a path extending in one direction as a detection target. That is, here, a pedestrian is used as a moving object, which will be described below.

FIG. 1 is a configuration diagram of a moving object detection system including a moving object detection device 100 according to the first embodiment of the present invention.

As shown in FIG. 1, in the moving object detection system, the moving object detection device 100 is connected to the distance measurement sensor 200 and the image acquisition sensor 300 via a network.

The range-finding sensor 200 is, for example, a laser rangefinder, a radar, an ultrasonic sensor, an active stereo sensor, etc., and scans within a time frame of an image captured by the image acquisition sensor 300 to obtain Ranging data.

The image acquisition sensor 300 is a visible light camera, an infrared camera, or the like, and is provided to detect a pedestrian path in a plan view and capture the path.

The moving object detection device 100 is based on the ranging information of the observation range of the ranging sensor at the time frame obtained from the ranging sensor 200 and the time at the time frame obtained from the image acquisition sensor 300. Image acquisition sensor within the observation range Image, the position of the pedestrian moving along the path, and the moving speed including the moving direction are estimated.

In addition, one ranging sensor 200 and one image acquisition sensor 300 form a group, and for a moving object detection target area, the group of ranging sensors 200 and image acquisition sensors 300 are set.

In addition, a group consisting of the ranging sensor 200 and an image acquisition sensor 300 is a complex array, and is connected to a moving object detection device 100 via a network. The moving object detection device 100 may be Each group consisting of a distance measuring sensor 200 and an image acquisition sensor 300 is subjected to a process of estimating the position, moving speed, and the like of a pedestrian moving along the path. The specific processing content will be described later.

FIG. 2 and FIG. 3 are diagrams for explaining the installation conditions of the ranging sensor 200 and the image acquisition sensor 300 in the first embodiment, and how the distance measurement sensor 200 and the image acquisition sensor 300 are installed. An example of observation conditions. FIG. 3 is a plan view of FIG. 2 viewed from the positive direction of the z-axis.

In Figures 2 and 3, the x-axis direction is the direction of the path extension, the z-axis direction is the direction perpendicular to the ground, the y-axis direction is the direction perpendicular to the path's extending direction and parallel to the ground, and the x-axis and y-axis The positive directions of the z-axis are orthogonal coordinate systems using a right-handed system.

As shown in FIG. 2 and FIG. 3, the image acquisition sensor 300 and the distance measurement sensor 200 are provided in a plan view path.

The image acquisition sensor 300 acquires the image of the channel at a preset angle of view, and the range projected from the setting position of the image acquisition sensor 300 to the xy plane on the channel is the observation range 53 of the image acquisition sensor 300 .

The ranging sensor 200 is scanned at least once in the y-axis direction within a time frame to obtain ranging data, and the position of the ranging sensor 200 is set to a one-dimensional line on the xy plane parallel to the y-axis. The projected range serves as the observation range 54 of the ranging sensor 200.

The observation range 53 of the image acquisition sensor 300 overlaps with the observation range 54 of the ranging sensor 200, and the width in the x-axis direction of the observation range 54 of the ranging sensor 200 is larger than that of the image acquisition sensor 300. The width in the x-axis direction of the observation range 53 is sufficiently narrow.

In addition, the one-dimensional line of the observation range of the ranging sensor 200 may be selected according to the observation environment. For example, the distance measuring sensor 200 can also observe a plurality of one-dimensional lines and select from each one-dimensional line.

One-dimensional lines that are less likely to be obscured by obstacles, etc., are used as the observation range.

Fig. 4 is a configuration diagram of a moving object detection device 100 according to the first embodiment of the present invention.

As shown in FIG. 4, the moving object detection device 100 includes a ranging data acquisition unit 11, an object position data creation unit 12, an image acquisition unit 13, a pixel movement direction data creation unit 14, a position movement direction correlation unit 15, and a display unit 16 And the recording section 17.

The ranging data acquisition unit 11 and the object position data production unit 12 constitute a ranging sensor processing unit 101, the image acquisition unit 13 and the pixel movement direction data production unit 14 constitute an image acquisition sensor processing unit 102, and a position movement direction correlation unit 15 constitutes a fusion processing unit 103.

The ranging data acquisition unit 11 sends a command to the ranging sensor 200, and obtains the ranging sensor observation at the time frame from the ranging sensor 200. Range ranging information. The ranging data acquisition unit 11 outputs the acquired ranging information to the object position data creation unit 12.

The object position data creation unit 12 is a time frame based on the distance measurement information obtained from the distance measurement data acquisition unit 11 at a plurality of time frames, and the position of the center of gravity of a specific moving object passes through the observation range 54 of the distance sensor 200 . The object position data creation unit 12 calculates the center of gravity positions of the x-axis and y-axis of a moving object from a plurality of time frames obtained by the distance measurement data acquisition unit 11 and generates object position data indicating the position of the center of gravity.

The object position data creation unit 12 outputs the time frame through which the center of gravity of the moving object passes to the position moving direction correlation unit 15, that is, the passing time and the object position data of the observation range 54 of the distance measuring sensor 200.

Here, as an example of a physical quantity indicating the position coordinates of a moving object, a device that calculates the position of the center of gravity of the entire moving object is used, but instead of the position of the center of gravity, a position coordinate about a part of the moving object may be used. For example, the position on the shoulders of a pedestrian or the position on the wheels of a car may be used.

The image acquisition unit 13 sends a command to the image acquisition sensor 300, and acquires an image within the observation range 53 of the image acquisition sensor 300 in the time frame from the image acquisition sensor 300. The image acquisition unit 13 outputs the acquired image to the pixel moving direction data creation unit 14.

The pixel moving direction data creation unit 14 calculates the estimated speeds of the x-axis and y-axis directions of each pixel based on the images obtained at the frames at a plurality of times obtained from the image acquisition unit 13, and creates and displays the estimates for each time frame Pixel speed data for speed. The pixel moving direction data creation unit 14 outputs pixel speed data to the position moving direction correlation unit 15.

Here, as an example of a device that calculates an estimated speed for each area on an image, the pixel moving direction data creation unit 14 that is a device that calculates an estimated speed for each pixel is used. The legal system is not necessarily every pixel. For example, it is also possible to reduce the processing load by calculating the estimated speed for each region of 10 pixels in length and 10 pixels in width.

Here, as an example of the physical quantity indicating the moving direction of the object, the pixel moving direction data creation unit 14 uses the estimated velocity in the x-axis and y-axis directions. However, the purpose of estimating only the moving direction of each moving object is not It is necessary to calculate the speed. For example, the pixel moving direction data creation unit 14 may calculate a fixed vector representing only the moving direction or an azimuth indicating the moving direction.

The position moving direction correlation unit 15 receives the object position data output from the object position data creation unit 12 and the pixel velocity data output from the pixel movement direction data creation unit 14 and estimates the position of the moving object specified by the object position data. Moving speed.

The position moving direction related unit 15 causes the display unit 16 to display the time when the position of the center of gravity passes through the observation range 54 of the distance measuring sensor 200, the position of the center of gravity on the x-axis and y-axis, and indicates the directions of the x-axis and y-axis. Information on the estimated speed. The position-moving-direction correlation unit 15 causes the recording unit 17 to memorize the time when the position of the center of gravity passes through the observation range 54 of the distance measuring sensor 200, the position of the center of gravity on the x-axis and y-axis, and indicates the x-axis and Information on the estimated speed in the y-axis direction.

Here, the time when the position of the center of gravity of each moving object passes the observation range 54 of the ranging sensor 200, the position of the center of gravity on the x-axis and y-axis, and the x-axis and The data of the estimated velocity in the y-axis direction are collectively called fusion data.

The display unit 16 is a display device such as a display, and displays the fusion data output from the position moving direction correlation unit 15.

The recording unit 17 stores the fusion data output from the position moving direction correlation unit 15.

Here, as shown in FIG. 4, the display unit 16 and the recording unit 17 are those included in the moving object detection device 100, but the present invention is not limited to this, and the display portion may be provided outside the moving object detection device 100. 16 and the recording unit 17, and the moving object detection device 100, the display unit 16, and the recording unit 17 are connected via a network.

5A and 5B are diagrams showing an example of a hardware configuration of the moving object detection device 100 according to the first embodiment of the present invention.

In the first embodiment of the present invention, each function of the ranging sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103 is implemented by the processing circuit 501. That is, the moving object detection device 100 is provided with a processing circuit 501 that performs control of displaying or memorizing information on the position, speed, etc. of the moving object based on the obtained ranging information and image information.

The processing circuit 501 may be dedicated hardware as shown in FIG. 5A, or a CPU (Central Processing Unit) 507 that executes a program stored in the memory 506 as shown in FIG. 5B.

When the processing circuit 501 is dedicated hardware, the processing circuit 501 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate). Array), or a combination of these elements Together.

When the processing circuit 501 is a CPU 507, the functions of the ranging sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103 are implemented by software, a carcass, or a combination of software and carcass . In other words, the ranging sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103 are CPU507 and system LSI (Large) that execute programs stored in the HDD (Hard Disk Driver) 502 and the memory 506. -Scale Integration). The programs stored in the HDD 502, the memory 506, and the like can be said to be programs or methods that cause a computer to execute the ranging sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103. Here, the memory 506 is nonvolatile such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Or volatile semiconductor memory, or magnetic disks, floppy disks, optical disks, small optical disks, mini optical disks, DVD (Digital Versatile Disc), etc.

In addition, each of the functions of the ranging sensor processing unit 101, the image acquisition sensor processing unit 102, and the fusion processing unit 103 may be implemented as part of dedicated hardware and part of software or hardware. For example, the processing unit 501 of the ranging sensor processing unit is a dedicated hardware processing circuit 501 to realize its function, and the processing unit 102 and the fusion processing unit 103 of the image acquisition sensor are read out by the processing circuit. And execute the program stored in the memory 506 to realize its function.

The display unit 16 is, for example, a display 503.

The recording unit 17 uses, for example, HDD502. In addition, this is just an example, and the recording unit 17 may be constituted by a DVD, a memory 506, or the like.

In addition, the moving object detection device 100 includes an input interface device 504 and an output interface device 505 that communicate with external devices such as the ranging sensor 200 and the image acquisition sensor 300. For example, the distance measuring sensor processing unit 101 uses an input interface device 504 such as USB (Universal Serial Bus), Aether (registered trademark, hereinafter omitted) to obtain the distance measured by the distance measuring sensor 200. Information. In addition, for example, the image acquisition sensor processing unit 102 uses an input interface device 504 such as DVI (Digital Visual Interface (registered trademark)), HDMI (High-Definition Multimedia Interface (registered trademark), USB, Ethernet, etc.) to acquire an image acquisition feeling. The image captured by the detector 300. In addition, for example, the fusion processing unit 103 is connected to the display 503 using an output interface device 505 such as USB and Ethernet, and outputs information such as the number, position, and speed of moving objects.

The operation of the moving object detection device 100 according to the first embodiment will be described.

FIG. 6 is a flowchart illustrating the operation of the moving object detection device 100 according to the first embodiment of the present invention.

Using FIG. 6, a case where the moving speed is estimated at the k-th (k-series natural number) time frame from the beginning of the observation, and the moving speed is estimated for the moving object having the center of gravity passing through the observation range 54 of the ranging sensor 200 is used as an example to explain the movement The operation of the object detection device 100.

Hereinafter, the k-th time frame from the start of observation is described as time k. In addition, the installation positions and observation ranges of the ranging sensor 200 and the image acquisition sensor 300 It is known as a preset, and a clock for measuring the observation time of the ranging sensor 200 and the image acquisition sensor 300 is used as synchronization between the ranging sensor 200 and the image acquisition sensor 300. .

The ranging sensor processing unit 101 is based on the ranging information obtained from the ranging sensor 200 at a plurality of time frames, and extracts the position of the center of gravity that has passed the observation range 54 of the ranging sensor 200 at time k. The object is moved to create object position data at time k (step ST601). Specifically, the object position data production unit 12 extracts the moving objects that have passed through the observation range 54 of the distance measurement sensor 200 based on the distance measurement information at a plurality of time frames output by the distance measurement data acquisition unit 11. The position of the center of gravity of a specific moving object is passed through the time frame within the observation range of the ranging sensor 200, and object position data is generated. When a plurality of moving objects have been extracted, the time frame is specified for each moving object, and the object position data is produced. In addition, the object position data creation unit 12 creates the distance measurement information storage unit (not shown) to store the distance measurement information of the time frame output by the distance measurement data acquisition unit 11 and stores the distance information based on the distance information storage unit. The plurality of ranging information can be used to create the object position data. The object position data creation unit 12 outputs the created object position data to the position moving direction correlation unit 15.

As a method of extracting the position of the center of gravity of the moving object from the ranging information of the frame at each time, for example, the method disclosed in Japanese Patent Application Laid-Open No. 2011-108223 is used. The method disclosed in this document is a method of extracting the number and position of moving objects at a specific height. For example, when the position of a pedestrian is extracted, the number of objects existing at the height of the head of the pedestrian can be obtained. Number and location.

In addition, it is not limited to this method, and the object position data creation section 12 series may also be used. Obtain the position data of the object that was created first at time k. Specifically, the object position data creation unit 12 may not create the object position data at time k each time in step ST601, but may memorize the created object position data in advance, and then obtain the memorized object position data.

The image acquisition sensor processing unit 102 generates the pixel speed data at time k from the images of the frame at each time acquired by the image acquisition sensor 300 (step ST602). Specifically, the pixel moving direction data creation unit 14 generates pixel speed data based on the images at the frames at a plurality of times output by the image acquisition unit 13. In addition, the pixel moving direction data creation unit 14 creates an image information storage unit (not shown) to store the time frame image output by the image acquisition unit 13 and creates pixel speed data based on the image stored in the image information storage unit. Just fine. The pixel moving direction data creation unit 14 outputs the created pixel speed data to the position moving direction correlation unit 15.

As a method of producing pixel velocity data from the image of the frame at each time, for example, B. Lucas, T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proceedings DARPA Image Understanding Workshop, pp. 121 ~ 130, method disclosed in April 1981. The method disclosed in this document is based on the premise that the brightness intensity of each pixel is spatially continuous and linear with the movement of the object. The residual vector of the brightness intensity between the time frames is estimated to be the minimum velocity vector by the least square method. According to this method, the velocity vector of a pixel representing a stationary object becomes a value close to zero, and pixel velocity data indicating that the velocity vector of a pixel of a moving object becomes a value close to a moving speed can be obtained.

In addition, the method is not limited to this method, and the pixel moving direction data creation unit 14 may be used. The obtained pixel velocity data at time k is obtained in advance. Specifically, for example, the pixel moving direction data creation unit 14 may not create the pixel speed data at time k at step ST602, but may store the created pixel speed data in advance and obtain the stored pixel speed data .

In addition, the pixel moving direction data creation unit 14 may also create pixel velocity data for only a part of the area on the image according to the observation environment. For example, if the area where the moving object does not exist, such as the area where pedestrians cannot pass, is known from the data of the frame before or after the data is obtained, the creation of pixel velocity data in the area where the moving object does not exist may be omitted.

The position moving direction correlation unit 15 uses the object position data at time k output by the object position data generating unit 12 in step ST601 and the pixel velocity data at time k output by the pixel moving direction data generating unit 14 in step ST602. For a moving object specified by the object position data at time k, a moving speed is estimated, and fusion data about the moving object at time k is created (step ST603).

Here, FIG. 7 is a schematic diagram illustrating a speed estimation operation of the moving object in step ST603 of FIG. 6. In addition, in FIG. 7, two moving objects that have passed through the observation range 54 of the range-finding sensor 200 at time k are extracted.

The position-moving-direction correlation unit 15 extracts a pixel velocity 62 within the object size 63 centered on the center of gravity position 61 of each moving object at time k. The position moving direction correlation unit 15 calculates the estimated speeds v ₁ and v _{2 of} each moving object from the pixel speed within the object size 63.

Here, the x-coordinates and y-coordinates in real space are If the x-coordinate and y-coordinate on the frame of the image obtained by the sensor 300 are the same.

In the following, the calculation of the estimated velocity v ₁ = (v _{1, x} , v _{1, y} ) will be described by taking the position of the center of gravity at time k as a moving object of (x _d , y _d ) as an example. method. Here, x _d is the position of the x-axis as the position of the center of gravity of the moving object, and y _d is the position of the y-axis as the position of the center of gravity of the moving object. In addition, v _{1, x} and v _{1, y} are regarded as the x-axis direction component and the y-axis direction component of the estimated speed. In addition, since the observation range 54 of the ranging sensor 200 is narrow in the x-axis direction, x _d may also be used as the x-axis position of the center of the observation range of the ranging sensor 200.

In the following, the pixel velocity data at time k is represented by V _x (x _i , y _i ) and V _y (x _i , y _i ). Here, x _i and y _i represent the x-axis center position and the y-axis center position of the i-th pixel, V _x (x _i , y _i ) represents the x-axis direction component of the pixel velocity, and V _y (x _i , y _i ) represents the y-axis direction component of the pixel velocity. In addition, the total number of pixels is taken as N _P.

The estimated speed v ₁ of the moving object at the center of gravity position (x _d , y _d ) is calculated from the following mathematical formula (1) and mathematical formula (2).

Here, W (x _i , y _i ; x _d , y _d ) is a function representing the object size 63, and is a function that satisfies the following mathematical formula (3) for any (x _d , y _d ).

For example, when a circular object size 63 is defined with (x _d , y _d ) as the center, it is defined as shown in the following mathematical formula (4).

Here, a parameter representing a radius of the object size 63 is taken as R, and the number of pixels (x _i ', y _i ') satisfying the following mathematical expression (5) is taken as n (R).

( x _{i '} -x _d ) ² + ( y _i' - y _d ) ² < R ² (5)

In addition, the size of the object size 63 can also be changed according to the type of the moving object envisaged, or the error of the pixel speed data of the sensor 300 acquired in the image and the error of the position data of the distance measuring sensor 200, Make the shape oval or rectangular. As shown in this embodiment, when a pedestrian is used as a detection target, the object size 63 may be regarded as an average size of pedestrians viewed from above.

In addition to performing the average addition on the pixel velocity data inside the object size 63 as described above, the function W (x _i , y _i ; x _d , y The value of _d ) has a slope.

In addition, it is also possible to discard unnecessary data based on the estimated speed on the premise that the position data of the object also contains unnecessary data other than stationary objects or actual objects. For example, if the estimated speed is less than the threshold value, it may be regarded as a stationary object and removed from the number of moving objects. That is, the number of moving objects calculated by the object position data creation unit 12 may be regarded as the number of temporary moving objects, and the final number of moving objects may be estimated by the position moving direction correlation unit 15.

Return to the flowchart in FIG. 6.

The position moving direction correlation unit 15 determines whether or not the estimated speed has been estimated for all moving objects specified by the object position data at time k (step ST604).

In step ST604, when it is determined that the estimated speed has not been estimated for all moving objects specified by the object position data at time k (in the case of "NO" in step ST604), the process returns to step ST603 and repeats subsequent processing.

In step ST604, when it is determined that the estimated speed has been estimated for all the moving objects specified by the object position data at time k (in the case of "YES" in step ST604), the position moving direction related unit 15 is based on the For all moving objects specified by the object position data of k, the passing time, the position of the center of gravity, and the estimated speed estimated in step ST603 are combined for each moving object and output to the display unit 16 and the recording unit 17 as fusion data. (Step ST605).

The display unit 16 displays information based on the fusion data output from the position moving direction correlation unit 15. The information displayed on the display unit 16 is, for example, the number of moving objects, the position of the center of gravity of each moving object, and the estimated speed. In addition, because the estimated speed also includes information on the direction of movement, the display unit 16 can also display, for example, a moving object that passes through the observation range 54 of the distance measuring sensor 200 toward the positive x direction or negative x direction of the path per unit time. Number of.

The information displayed on the display unit 16 may be determined based on the preset conditions. The information is not limited to this, and the information displayed on the display unit 16 may be set in advance by the user from an input device (not shown), or the user may switch the information displayed on the display during the display.

The recording unit 17 records the fusion data output from the position moving direction correlation unit 15. The information recorded by the recording unit 17 is, for example, the number of moving objects, the position of the center of gravity of each moving object, and the estimated speed. Also, in the recording section 17, It is also possible to record the number of moving objects that pass through the observation range 54 of the ranging sensor 200 toward the positive x direction or negative x direction of the path per unit time.

The information recorded by the recording unit 17 may be determined based on preset conditions. Not limited to this, the information recorded by the order recording unit 17 may be set in advance by the user from an input device (not shown), or the information recorded by the order may be changed by the user.

As shown above, according to the first embodiment, an image acquisition sensor 300 and a distance measuring sensor 200 disposed at a detection position can be used to estimate the number, position, and movement of moving objects. The speed of movement in the direction.

Generally, compared with image acquisition sensors that only require light-receiving elements, because the number of components of the ranging sensor that are provided by both the light-emitting element and the light-receiving element is larger, and the frequency of correction and aging is higher, it is being manufactured. Expenses are expensive when using it. In the moving object detection device 100 according to the first embodiment, since only one distance sensor 200 such as a laser sensor is required for one detection position, it is compared with a method using a plurality of distance sensors. , Can achieve the same function at a low cost.

In addition, according to the first embodiment, the light emitting element provided at one of the distance measuring sensors 200 can be used to estimate the moving speed including the moving direction of the moving object. Compared with the plural methods, there are fewer restrictions on setting conditions, and the cost of setting up the project can be reduced.

In addition, according to the first embodiment, the number, position, and direction of moving objects can be estimated by the image acquisition sensor 300 and the distance measuring sensor 200 that scans in only one dimension. Ranging sensing in one direction Compared with a distance measuring sensor that scans in a two-dimensional direction, the device 200 can reduce the cost of scanning and the amount of machinery and processing required during manufacturing, installation, and operation. Therefore, it is possible to realize the same function at a lower cost than a method using a ranging sensor such as a laser sensor that scans in a two-dimensional direction.

The distance measuring sensor 200 is also capable of observing the shape of a moving object in an irradiation direction such as a laser irradiated by the distance measuring sensor 200, so the position estimation error of the moving object is smaller than that of the image acquisition sensor 300. , And it is easy for the area of each moving object of a plurality of individuals approaching. Therefore, according to the first embodiment, compared with the method of acquiring the sensor 300 using only an image of a visible light camera or the like, the number and position of moving objects can be estimated with higher accuracy.

In addition, according to the first embodiment, an image of the image acquisition sensor 300 is created, an estimated speed of each pixel is calculated based on the brightness and intensity of the pixel, and a moving direction of the moving object is estimated. Therefore, when the resolution of the image acquisition sensor 300 is low and the shape of the moving object is not clear, such as when the size of the moving object, the pixels of the image are thick, or a part of the outline of the moving object is missing, Compared with the method of acquiring the sensor 300 using only an image of a visible light camera or the like, the deterioration of the estimation accuracy of the movement direction is small.

In addition, according to the first embodiment, the number of moving objects is estimated based on the observation results of the ranging sensor 200. Therefore, when a shadow of a moving object is captured in an image, or played on a screen on the background In the case of moving images, etc., the number of moving objects can be estimated without misjudging changes in the brightness intensity of the backgroundless entity as moving objects.

Second embodiment

In the first embodiment, the sensor 300 is obtained from the borrowed image. The premise that the brightness and intensity of the moving object on the acquired image is continuous. At the same time, the position of the center of gravity of the moving object based on the ranging information from the ranging sensor 200 at the same time frame and the sensing based on the obtained image The pixel speed of the image of the processor 300 corresponds.

However, in fact, the brightness intensity of a moving object changes continuously due to, for example, a bias in the illumination of the light source, a change in background color, rotation and deformation of a moving object, exaggerated noise of night photography, and the like. In this case, the pixel speed of the moving object temporarily becomes a value that is greatly different from the speed of the real moving object. At some point, the pixel speed of the frame may disappear.

Therefore, in this second embodiment, an embodiment will be described in which a plurality of candidates of estimated speeds of each moving object are set from the pixel speeds of a plurality of time frames, and the most reasonable estimated speed is determined from each candidate.

The configuration of the moving object detection system according to the second embodiment is the same as that described with reference to FIG. 1 in the first embodiment, and redundant description is omitted.

In addition, the installation conditions of the ranging sensor 200 and the image acquisition sensor 300 and the observation conditions of the ranging sensor 200 and the image acquisition sensor 300 are also used because, for example, the second embodiment is used in the first embodiment. As shown in the description of FIG. 3 and FIG. 3, overlapping description is omitted.

Fig. 8 is a configuration diagram of a moving object detection device 100a according to a second embodiment of the present invention.

In FIG. 8, the same configurations as those of the moving object detection device 100 according to the first embodiment described with reference to FIG. 4 are assigned the same reference numerals, and descriptions thereof are omitted.

Compared with the moving object detection device 100 according to the first embodiment, the moving object detection device 100a according to the second embodiment of the present invention only has a moving direction correlation section 15 at a substitute position, and the moving direction assumption value setting section 18 and the moving direction The determination unit 19 differs from the fusion processing unit 103.

In the second embodiment, the object position data creation unit 12 outputs the object position data to the movement direction assumption value setting unit 18. The pixel moving direction data creating unit 14 outputs the pixel speed data to the moving direction assumption value setting unit 18 and the moving direction determining unit 19.

The movement direction hypothesis setting unit 18 obtains the object position data output from the object position data creation unit 12 and the pixel speed data of the time frame output from the pixel movement direction data creation unit 14 and sets a plurality of types for distance measurement. Candidate value of the moving speed of the moving object observed by the sensor 200. Here, the candidate value of the moving speed of the moving object set by the moving direction assumption value setting unit 18 is referred to as a speed assumption value.

The movement direction assumption value setting unit 18 outputs a speed assumption value to the movement direction determination unit 19.

The moving direction determining unit 19 obtains the pixel speed data of the frame at the time output from the pixel moving direction data creating unit 14 and the speed assumed value of the moving object output from the moving direction assumed value setting unit 18, and obtains the speed assumed value from the speed assumed value. Among them determines the moving speed of the moving object. The movement direction determining unit 19 outputs to the display unit 16 and the recording unit 17 the time when the position of the center of gravity of the moving object passes through the observation range 54 of the distance measuring sensor 200, the position of the center of gravity on the x-axis and y-axis, and indicates the x-axis and y Fusion data of estimated velocity in the axial direction.

The hardware configuration of the moving object detection device 100a is similar to that of the first embodiment. The first embodiment is the same as that described with reference to Figs. 5A and 5B, so redundant descriptions are omitted.

The operation of the moving object detection device 100a according to the second embodiment will be described.

Fig. 9 is a flowchart illustrating the operation of the moving object detection device 100a according to the second embodiment of the present invention.

In the following description, k is taken as a natural number equal to or greater than N + 1. The parameter N will be described later.

The ranging sensor processing unit 101 is based on the ranging information obtained from the ranging sensor 200 at a plurality of time frames, and extracts a moving object passing through the observation range 54 of the ranging sensor 200 at the position of the center of gravity at time k. And create object position data at time k (step ST901). The specific operation is the same as step ST601 of FIG. 6 described in the first embodiment. The object position data creation unit 12 of the ranging sensor processing unit 101 outputs the created object position data to the movement direction assumption value setting unit 18.

The control unit (not shown) of the moving object detection device 100a initializes the variable i (step ST902). Here, the starting value of the variable i is taken as 0.

The image acquisition sensor processing unit 102 generates the pixel velocity data at the time k-i from the images obtained at the frames at a plurality of times from the image acquisition sensor 300 (step ST903). A specific method of generating the pixel speed data is the same as step ST602 of FIG. 6 described in the first embodiment. The pixel movement direction data creation unit 14 of the image acquisition sensor processing unit 102 outputs the pixels at the time k-i to the movement direction assumption value setting unit 18 and the movement direction determination unit 19 Speed data.

The movement direction assumed value setting unit 18 uses the object position data at time k output by the object position data creation unit 12 in step ST901 and the pixel velocity data at time ki output by the pixel movement direction data creation unit 14 in step ST903 , A speed assumed value is set for the moving object included in the object position data at time k (step ST904).

Here, FIG. 10 is a schematic diagram showing the operation of setting the speed assumption value by the movement direction assumption value setting unit 18 in step ST904 of FIG. 9 taking i = 1 as an example.

The movement direction hypothesis setting unit 18 extracts the estimated value of the position of the center of gravity of the moving object at time k-1 from the position of the center of gravity 61 of the moving object at time k, that is, the estimated value of the position of the object at time k-1 is 64, and The pixel speed 65 at time k-1 is within the object size 63 centered on the object position estimated value 64 at time k-1. Further, the movement direction assumption value setting unit 18 calculates a speed assumption value v _{1 of the} moving object from the extracted pixel speed 65 at time k- ₁ .

Fig. 11 is a flowchart illustrating the operation of step ST904 in Fig. 9 in detail.

Here, the position of the center of gravity of a moving object as an object is taken as (x _d , y _d ), and the speed assumption value is taken as v _u = (v _{u, x} , v _{u, y} ). x _d is the x-axis position of the center of gravity of the moving object, y _d is the y-axis position of the center of gravity of the moving object, v _{u, x} , v _{u, y} are the x-axis direction components of the estimated velocity, respectively , Y-axis direction component. In addition, since the observation range 54 of the ranging sensor 200 is narrow in the x-axis direction, x _d may also be used as the x-axis position of the center of the observation range of the ranging sensor 200.

In addition, V _{ki, x} (x _q , y _q ) and V _{ki, y} (x _q , y _q ) represent the pixel velocity data at time k -i. Here, x _q and y _q represent the x-axis center position and the y-axis center position of the q-th pixel, respectively, and V _{ki, x} (x _q , y _q ) represents the x-axis direction component of the pixel velocity, and V _{ki, y} (x _q , y _q ) represents the y-axis direction component of the pixel velocity. In addition, the total number of pixels is taken as N _P.

First, the movement direction assumption value setting unit 18 selects an unselected pixel from the image obtained from the pixel movement direction data creation unit 14, that is, the image obtained by the image acquisition sensor 300 (step ST1101). Hereinafter, the pixel selected by the movement direction assumption value setting unit 18 in step ST1101 is referred to as a "selected pixel", and the x-axis center position and the y-axis center position of the selected pixel are referred to as (x _s , y _s ).

The movement direction hypothesis setting unit 18 calculates the average velocity v _m = (v _{m, x} , v _{m, y} ) of the selected pixels from the pixel velocity data at the time ki in the vicinity of the selected pixels (step ST1102). Here, v _{m, x} and v _{m, y} are considered as the x-axis direction component and the y-axis direction component of the average speed.

The average speed v _m = (v _{m, x} , v _{m, y} ) is calculated from the following mathematical formula (6) and mathematical formula (7).

Here, W (x _q , y _q ; x _s , y _s ) is a function representing the object size 63, and is a function that satisfies the following mathematical formula (8) for any (x _s , y _s ).

For example, when a circular object size 63 is defined with (x _s , y _s ) as the center, it is defined as shown in the following mathematical formula (9).

Here, the parameter representing the radius of the object size 63 is taken as R, and the number of pixels (x _q ', y _q ') satisfying the following mathematical expression (10) is taken as n (R).

( x _{q '} -x _s ) ² + ( y _q' - y _s ) ² < R ² (10)

In addition, the size of the object size 63 may be changed according to the type of the moving object envisaged, or the error of the pixel speed data of the sensor 300 obtained in the image, and the error of the position data of the distance measuring sensor 200, or Make the shape oval or rectangular. As shown in this embodiment, when a pedestrian is used as a detection target, the object size 63 may be regarded as an average size of pedestrians viewed from above.

In addition to performing the average addition on the pixel velocity data in the object size 63 as described above, the function W (x _q , y _q ; x _s , y The value of _s ) has a slope.

Based on the average speed calculated in step ST1102, the movement direction assumption value setting unit 18 calculates the position of the selected pixel by extrapolating the time k with the average speed (step ST1103). Here, the position where the selected pixel is extrapolated at an average speed from time k is referred to as an extrapolated position.

The movement direction hypothesis value setting unit 18 assumes that the moving object moves straight at a constant speed from time ki to time k, and calculates the extrapolated position (x _e , y _e ) from the following mathematical formula (11) and mathematical formula (12).

x _e = x _s + v _{m , x} T _{k - i , k} (11)

y _e = y _s + v _{m , y} T _{k - i , k} (12)

Here, T _{ki, k} represents the elapsed time between the frames from time ki to time k. In particular, in the case of i = 0, it is regarded as T _{k, k} = 0.

Also, in Mathematical Formula (11) and Mathematical Formula (12), as a motion model of a moving object, a constant-speed straight-forward motion is assumed, but it can also be adapted to the type of moving object. Motion model to calculate extrapolated position.

The movement direction hypothesis value setting unit 18 calculates a residual from the extrapolated position calculated in step ST1103 and the position of the center of gravity of the moving object (step ST1104).

The moving direction hypothesis value setting unit 18 calculates the moving object based on the following mathematical formula (13) from the extrapolated position (x _e , y _e ) calculated from the pixel velocity data at time ki and the object position data at time k. The residual Δr of the position of the center of gravity (x _d , y _d ).

Here, t in the upper right of the matrix represents the transpose of the matrix, and "-1" in the upper right of the matrix represents the inverse matrix.

_{^{And, σ x 2, σ y 2}} , σ xy 2 based on the gravity center position as a parameter extrapolated position of the moving object changes in size of the residuals, σ _x ² are diagrams dispersive x-axis component, σ _y ² Indicates the dispersion of the y-axis component, and σ _xy ² indicates the co-dispersion of the x-axis and y-axis components. σ _y ^{2 is} set to a value larger than σ _x ² , and a residual can be calculated in consideration of the difference in the variation of each component.

Or, the change in the residuals about the extrapolated position and the position of the center of gravity of the moving object In cases where it is difficult to digitize, it can be assumed that the variation of each component is independent and the same, and the residual Δr can be obtained from the simpler mathematical formula (14) below.

△ r ² = ( x _d - x _e ) ² + ( y _d - y _e ) ² (14)

The movement direction hypothesis value setting unit 18 determines whether the residual Δr calculated in step ST1104 is the smallest compared to other residuals calculated in the loop processing of steps ST1101 to ST1107 (step ST1105).

In step ST1105, compared with other residuals, when the determination is not the smallest, that is, when the residual calculated in step ST1104 immediately before is not tentatively the minimum value (in the case of "NO" in step ST1105), The process of step ST1106 is skipped, and the process proceeds to step ST1107.

In step ST1105, compared with other residuals, when the determination is the smallest, that is, when the residual calculated in step ST1104 immediately before is tentatively the minimum value (in the case of "YES" in step ST1105), The moving direction hypothesis value setting unit 18 sets the average speed v _m calculated in step 1102 to the speed hypothesis value v _u of the moving object whose center of gravity position at time k is (x _d , y _d ) (step ST1106). In addition, if another value has been set as the speed assumed value of the moving object in the loop processing of steps ST1101 to ST1107, the movement direction assumed value setting unit 18 sets the average speed v _m calculated in step ST1102 as a new value. Average speed replacement.

The movement direction assumption value setting unit 18 determines whether or not all pixels in the pixel speed data at time k-i have been selected (step ST1107).

In step ST1107, it is determined that all the pixels have not been selected (in the case of "NO" in step ST1107), the process returns to step ST1101, and repeats thereafter. Processing.

In step ST1107, it is determined that all the pixels have been selected (in the case of "YES" in step ST1107), and the processing of Fig. 11 is ended.

As shown above, by executing the processing of FIG. 11, the speed assumed value of the moving object is set based on the pixel speed data at time k-i.

In addition, in the processing of FIG. 11 above, for all pixels in the pixel speed data at time ki, the average speed, the extrapolated position, and the residual from the position of the center of gravity of the moving object are set, and the speed assumed value is set, but It is also possible to reduce the processing load, for example, by limiting the selected pixels from the maximum speed assumed for the moving object.

In the processing in FIG. 11 above, only one speed hypothesis value is set based on the pixel speed data at time ki, but a plurality of speeds may be set from the pixel speed data at time ki according to the size of the residual. Assumed value.

In the processing in FIG. 11 described above, the case where the pixel movement direction data creation unit 14 calculates the speeds in the x-axis and y-axis directions is described. However, the case where the pixel movement direction data creation unit 14 calculates only the movement direction. The estimated moving direction is converted into speeds in the x-axis and y-axis directions based on a parameter indicating a typical moving speed of the pedestrian, and the same processing is performed.

Here, the pixel moving direction data creation unit 14 is an example of the physical quantity indicating the moving direction of the object, and calculates the speed in the x-axis and y-axis directions. However, it is not necessary to calculate the speed for the purpose of estimating only the moving direction of the object. . For example, it is also possible to use only a fixed vector or table that calculates the direction of movement. Shows the azimuth of the moving direction. In this case, the speed assumed value of the object set by the movement direction assumed value setting unit 18 is also referred to as a movement direction assumed value.

Return to the flowchart of FIG. 9.

The control unit (not shown) of the moving object detection device 100a determines whether the variable i is greater than or equal to the parameter N (step ST905).

The parameter N represents the number of past time frames that are referred to when setting the speed assumption value. The parameter N is set, for example, from the time k-N to the time k with the assumption that the movement of the target is a constant-speed linear motion, and the maximum number of frames at the appropriate time is assumed.

In step ST905, when it is determined that the variable i is an underfill parameter N (in the case of "NO" in step ST905), the movement direction assumption value setting unit 18 adds 1 to the variable i (step ST906), and returns to step ST903. .

In step ST905, when it is determined that the variable i is greater than the parameter N (in the case of "YES" in step ST905), the control unit of the moving object detection device 100a initializes the variable j (step ST907). The starting value of the variable j is 1. That is, the control unit substitutes 1 into the variable j. The control unit outputs information to the image acquisition sensor processing unit 102 on the purpose of initializing the variable j.

The image acquisition sensor processing unit 102 generates the pixel velocity data at the time k + j from the images obtained at the frames at a plurality of times from the image acquisition sensor 300 (step ST908). A specific method of generating the pixel speed data is the same as step ST602 of FIG. 6 described in the first embodiment. The pixel movement direction data creation unit 14 of the image acquisition sensor processing unit 102 outputs pixel speed data at the time k + j to the movement direction assumption value setting unit 18 and the movement direction determination unit 19.

The movement direction determination unit 19 updates the plurality of speed assumption values of the moving object set by the movement direction assumption value setting unit 18 in step ST904 using the pixel speed data at time k + j obtained in step ST908, and displays the respective speeds. The value of the probability of the hypothetical value (step ST909). Here, the value which represents the probability of each speed hypothesis value is called the probability of speed hypothesis value.

In addition, the probability of each speed hypothesis value is successively updated in the loop processing on the variable j in steps ST908 to ST911. For example, the assumed value based certain velocity v _i, at time k + j in accordance with the speed of the pixel data of the updated probability L _{i, k + j} j based on a probability-based variable of the previous loop processing of the calculated L _{i, k +} calculated by _j-1 .

FIG. 12 shows the case of j = 1 and j = 2 as an example, and shows the operation of updating the probability of the speed assumption values v ₀ , v ₁ , and v ₂ of the borrowing direction determining unit in step ST909 of FIG. 9. schematic diagram.

The moving direction determining unit 19 calculates the position of the center of gravity 61 of the moving object at time k, and calculates the estimated positions 66a to 66c of the object at time k + 1 and the estimated positions 68a of the object at time k + 2 based on the speed assumptions. ~ 68c. In addition, the movement direction determination unit 19 extracts pixels at time k + 1 and time k + 2 within the object size 63 centered on the estimated value of each object position from the pixel velocity data at time k + 1 and time k + 2. Speeds 67a ~ 67c, 69a ~ 69c.

Further, the moving direction determination unit 19 calculates the probability of the speed assumed value from the difference between the speed assumed value and the pixel speed within the object size 63 at each time frame. This probability indicates the degree of agreement between the speed assumed value set by the pixel speed before time k and the pixel speed after time k + 1, and the speed assumed value through the time frame after time k + 1. Within object size 63 The smaller the difference in pixel speed becomes, the higher the value becomes.

In the example in FIG. 12, the difference between the speed assumption value v ₁ and the pixel speed 67a around the object position estimation value 66a and the pixel speed 69a around the object position estimation value 68a is smaller than the remaining speed assumption values v ₀ and v ₂ Therefore, the probability of the velocity assumed value v ₁ of the moving object is larger than the probability of the velocity assumed values v ₀ and v ₂ .

Hereinafter, the operation of step ST909 in FIG. 9 will be described in detail.

FIG. 13 is a flowchart illustrating detailed operations of step ST909 in FIG. 9.

Here, the position of the center of gravity of a moving object as an object is taken as (x _d , y _d ), and the speed assumption value is taken as v _u = (v _{u, x} , v _{u, y} ). x _d is the x-axis position of the center of gravity of the moving object, y _d is the y-axis position of the center of gravity of the moving object, v _{u, x} , v _{u, y} are the x-axis direction components of the estimated velocity, The y-axis component. In addition, since the observation range 54 of the ranging sensor 200 is narrow in the x-axis direction, x _d can also be used as the center of the observation range of the ranging sensor 200.

In addition, v _{k + j, x} (x _q , y _q ) and v _{k + j, y} (x _q , y _q ) represent pixel velocity data at time k + j. Here, x _q and y _q are the x-axis center position and y-axis center position of the q-th pixel, and v _{k + j, x} (x _q , y _q ) are the x-axis direction components of the pixel velocity, and v _{k + j, y} (x _q , y _q ) represents the y-axis direction component of the pixel velocity. In addition, the total number of pixels is taken as N _p .

Moving direction determination unit 19 based gravity position of the moving object free time k is (x _d, y _d) and the velocity of hypothetical values v _u is calculated on the velocity of hypothetical values v _u sum at time k + j of the object position estimation value (x _{e, k + j} , y _{e, k + j} ) (step ST1301).

The moving direction determining unit 19 assumes that the moving object is moving straight at a constant speed from time k to time k + j, and calculates the estimated position of the object (x _{e, k + j} ) from the following mathematical expressions (15) and (16). , y _{e, k + j} ) (step ST1301).

x _{e , k + j} = x _d + v _{u , x} T _{k , k + j} (15)

y _{e , k + j} = y _d + v _{u , y} T _{k , k + j} (16)

Here, T _{k, k + j} represents the elapsed time between the frames from time k to time k + j.

Also, in Mathematical Expressions (15) and (16), as a motion model of a moving object, it is assumed that a linear motion is performed at a constant speed. However, the movement direction determining unit 19 may be adapted to the type of the moving object. For example, a constant acceleration is assumed. Calculate the extrapolated position by other motion models such as linear motion.

The moving direction determining unit 19 calculates the average velocity near the moving object at time k + j from the estimated position of the object calculated at step ST1301 and the pixel velocity data at time k + j, and _{m + k} = (v _{m, k + j, x} , v _{m, k + j, y} ) (step ST1302). v _{m, k + j, x} , v _{m, k + j, y} are regarded as the x-axis component and the y-axis component of the average speed.

The moving direction determination unit 19 calculates the average speed v _{m, k + j} = (v _{m, k + j, x} ) from the following mathematical formula (17) and mathematical formula (18), for example, as in step ST1102 in FIG. , v _{m, k + j, y} ).

Here, W (x _q , y _q ; x _{e, k + j} , y _{e, k + j} ) is a function representing the size of the object 63, which is arbitrary for (x _{e, k + j} , y _{e, k + j} ) A function that satisfies the following formula (19).

For example, when a circular object size 63 is defined with (x _{e, k + j} , y _{e, k + j} ) as the center, it is defined as shown in the following mathematical formula (20).

Here, R is taken as a parameter representing the radius of the object size 63, and n (R) is taken as the number of pixels (x _q ', y _q ') satisfying the following mathematical formula (21).

( x _{e , k + j} - x _{q '} ) ² + ( y _{e , k + j} - y _{q '} ) ² < R ² (21)

In addition, the size of the object size 63 may be changed according to the type of the moving object envisaged, or the error of the pixel speed data of the sensor 300 obtained in the image, and the error of the position data of the distance measuring sensor 200, or Make the shape oval or rectangular. As shown in this embodiment, when a pedestrian is used as a detection target, the object size 63 may be regarded as an average size of pedestrians viewed from above.

In addition to performing the average addition on the pixel velocity data inside the object size 63 as described above, the function W (x _q , y _q ; x _{e, k + j} , y _{e, k + j} ) have skew.

The moving direction determination unit 19 is based on the speed assumption v _u = (v _{u, x} , v _{u, y} ) and the average speed v _{m, k + j} = (v _{m, k + j, x} , v _{m, k + j, y} ), and calculate the probability L _{u, k + j of the} speed assumed value (step ST1303).

The probability L _{u, k + j} of the speed assumption v _u at time k + j is calculated from the following mathematical formula (22).

Here, the speed assumption values v _u and v _{m, k + j} are taken as the longitudinal quantities represented by the following mathematical expressions (23) and (24).

_{Sk + j} is a parameter representing two columns and two rows of the co-dispersion matrix of the residuals of v _u -v _{m, k + j} . In addition, regarding the probability _{Lu, k + j-1} at the time k + j- ₁ , especially in the case of j = 1, it is considered as follows.

L _{u , k} = 1 (25)

In addition, the above probability _{Lu, k + j} is the average speed vm _{, k + j} is the value of the model of the real speed of the moving object plus the white Gaussian noise. The error distribution of the average speed can also be used for colored non- Gaussian noise model.

The movement direction determination unit 19 determines whether or not the assumed value probability at time k + j has been updated for all the speed assumed values (step ST1304).

In step ST1304, it is determined that the hypothesis value probability at time k + j has not been updated for all the speed assumed values (in the case of "NO" in step ST1304), and the process returns to step ST1301 and repeats subsequent processing.

In step ST1304, it is determined that the hypothesis value probability at time k + j has been updated for all the speed assumed values (in the case of "YES" in step ST1304), and the processing of Fig. 13 is ended.

By executing the processing of FIG. 13 described above, the movement direction determination unit 19 updates the values representing the probabilities for each of the assumed speed values of the moving object based on the pixel speed data at time k + j.

Return to the flowchart of FIG. 9.

The control unit determines whether the variable j is greater than or equal to the parameter M (step ST910). The parameter M represents the number of time frames used in the calculation of the probability of the speed hypothesis value. In the parameter M, for example, it is preset that the assumption that the movement of the target is a constant-speed straight-line movement from time k to time k + M becomes the maximum value of the number of frames at the appropriate time.

In step ST910, when it is determined that the variable j is an underfill parameter M (in the case of NO in step ST910), the movement direction determination unit 19 adds 1 to the variable j (step ST911), and returns to step ST908.

In step ST910, when it is determined that the variable j is greater than or equal to the parameter M (in the case of "YES" in step ST910), the movement direction determination unit 19 calculates each of the speed hypothesis values calculated and updated in step ST909. Post hoc probability of speed hypothesis (step ST912).

Specifically, the moving direction determination unit 19 is a probability L _{u, k + M} (u = 1,2, ..) of each speed assumption value updated based on the pixel speed data at time k + 1 to time k + M. ., N _H , N _H are the number of speed hypotheses), and calculate the ex post probability P (H _u | V _{k: k + M} ) from the following mathematical formula (26).

Here, P (H _u) represents a priori probability-based value is assumed velocity v _u, the system parameters for each velocity set of hypothetical values of v _u.

As the setting method of P (H _u ), for example, according to the magnitude of the residual Δr of mathematical formula (26) or the value of v ₁ when the speed assumption value v _u is set, the light source or Set the background color, etc. Or, all speed hypotheses can be regarded as the same at the set time point, and P (H _u ) can be regarded as a constant.

The moving direction determination unit 19 determines the speed assumed value v _u with the greatest hindered probability from the hindered probability P (H ₁ | V _{k: k + M} ) of the speed assumed values calculated in step ST912, as the estimated speed of the moving object. (Step ST913).

The moving direction determination unit 19 determines whether or not the estimated speed has been determined for all moving objects among one or a plurality of moving objects specified by the object position data at time k (step ST914).

In step ST914, when the estimated speed has not been determined for all the moving objects (in the case of "NO" in step ST914), the process returns to step ST902.

In step ST914, when the estimated speed has been determined for all the moving objects (in the case of "YES" in step ST914), the moving direction determination unit 19 outputs the moving objects determined in step ST913 to the display unit 16 and the recording unit 17. Fusion data of the moving speed and the position of the center of gravity (step ST915).

In addition, as described above, the object position data creation unit 12 has estimated the number of moving objects. However, on the premise that the object position data also contains unnecessary data other than stationary objects or objects that actually exist, the movement direction determination unit may also be used. 19 Discard unnecessary data based on the estimated speed. For example, if the estimated speed is below the threshold, it can also be regarded as a stationary object and removed from the number of moving objects. In other words, the number of moving objects calculated by the object position data creating unit 12 may be regarded as the number of temporary moving objects, and the final number of moving objects may be estimated by the movement direction determining unit 19.

In addition, in the above, the fusion processing unit 103 sets the speed assumed value of the moving object from the pixel speed data before time k, and determines the speed of the moving object based on the pixel speed data after time k + 1. The frame selection method used at the moment is not limited. For example, the fusion processing unit 103 may set the speed false based on the pixel speed data after time k + 1. Set a value and determine the moving speed of the moving object from among the speed assumptions based on the pixel speed data before time k.

As described above, according to the second embodiment, since the moving speed of a moving object is estimated from the pixel speed of the frame at a plurality of times, the pixel speed data calculated from the image acquisition sensor 300 is used for moving. The pixel speed around the object will temporarily become a value that is very different from the real object's moving speed. It can also be estimated by a distance measuring sensor and an image acquisition sensor installed at a detection position. Number, position, and moving speed including the moving direction.

This effect is effective when, for example, the number and position of moving objects and the moving speed including the moving direction are estimated at night. The image acquisition sensor 300 of a camera or the like has a characteristic that the illumination intensity of a light source is low at night or in a dark place, and the brightness of each pixel is more likely to change due to noise inside the sensor. In this case, since the brightness of a moving object changes due to noise, the pixel speed data is likely to have a value that deviates from the true object moving speed of the moving object. However, according to the configuration of the first embodiment, the candidate speed of the object is set based on the pixel speed data of the frame at a plurality of time points, and the candidate values are compared to thereby estimate the probability of the speed of movement, so it can be reduced. Probability of estimating incorrect moving speed.

In addition, the image acquisition sensor 300 with small changes in luminance intensity caused by noise in low-light environments is more expensive because the general manufacturing process is more complicated. Therefore, according to the second embodiment, compared with the method of using the image acquisition sensor 300 which is hardly affected by noise in a low-light environment, it is epoch-making in terms of achieving the same accuracy with less cost.

This effect is also effective when, for example, a pedestrian is targeted as a type of moving object. Due to the movement of the hands and feet or the change in the direction of the body, the pedestrian is more likely to irregularly change the brightness of the surface of the object compared to inorganic objects such as vehicles. Therefore, for example, at a point in time when a pedestrian changes the direction of the face, the pixel speed data tends to be a value that is significantly different from the moving speed of the object. However, according to the configuration of the second embodiment, the pixel speed at the moment when the temporary pixel speed data becomes unstable is discarded because the pixel speed data of the time frame is low, and therefore the pixel speed is discarded. Irregular changes can also reduce the chance of miscalculating the speed of movement.

In addition, because pedestrians have clothing and the like worn on them, the brightness distribution is more diverse than that of inorganic substances such as vehicles. Therefore, the selection of an environment where it is difficult for the background to be confused with pedestrians is difficult in principle. As a result, when the background brightness becomes the same frame as the pedestrian brightness, the pixel velocity data will be very different from the pedestrian's moving speed. Value. However, according to the second embodiment, among the plurality of speed assumption values, the speed assumption value set from the pixel velocity data of the frame at the time when the background is confused with the background is because it is regarded as being different from the other time frame. Hypothetical values with low consistency are discarded, so the probability of moving speed of estimation errors can also be reduced in situations where pedestrians and backgrounds are temporarily confusing.

This effect is also effective when the brightness distribution on the image is greatly biased, for example, when the outdoor environment on a sunny day or when the background contains a light source. In the case where the brightness distribution on the image is greatly biased in outdoor locations such as the sunny place and the shaded place, because the continuity of the brightness distribution is the premise when calculating the pixel speed, the pixel speed data around the moving object will be obtained and The speed of moving objects is very different. However, according to the second embodiment, from When the moving object passes through the area where the brightness distribution is largely biased, the estimated speed value of the frame's pixel speed data is discarded because it is assumed to have low consistency with the pixel speed of the frame at other times. A situation where the brightness distribution is largely biased can also reduce the probability of estimating the moving speed by mistake.

Third Embodiment

In the first and second embodiments, the number of moving objects is estimated from the position of the center of gravity of the moving object based on the ranging information of the distance measuring sensor 200 and the pixel speed of the image obtained by the sensor 300 based on the image. Position and moving speed including the moving direction.

In the third embodiment, an embodiment will be described in which the height information from the ground plane of the moving object and the data about the width of the moving object are made into the attribute data of the moving object based on the distance measurement information of the distance measuring sensor 200.

The configuration of the moving object detection system according to the third embodiment is the same as that described with reference to FIG. 1 in the first embodiment, and therefore redundant description is omitted.

In addition, the installation conditions of the ranging sensor 200 and the image acquisition sensor 300 and the observation conditions of the ranging sensor 200 and the image acquisition sensor 300 are also used because, for example, the second embodiment is used in the first embodiment. As shown in the description of FIG. 3 and FIG. 3, overlapping description is omitted.

Fig. 14 is a configuration diagram of a moving object detection device 100b according to a third embodiment of the present invention.

In FIG. 14, the same configurations as those of the moving object detection device 100 according to the first embodiment described with reference to FIG. 4 are denoted by the same reference numerals, and descriptions thereof will be omitted.

Compared with the moving object detection device 100 according to the first embodiment, the moving object detection device 100b according to the third embodiment of the present invention includes an object height data creation unit 20 and an object width data only in the ranging sensor processing unit 101. The creation section 21 and the fusion processing section 103 further include an object attribute determination section 22 that is different from each other.

The object height data creation unit 20 obtains the distance measurement information of the frame at a plurality of times from the distance measurement data acquisition unit 11 and generates data of the estimated height of the moving object from the ground surface. Here, the data of the estimated height of the moving object from the ground surface is referred to as object height data.

The object height data creation unit 20 outputs the created object height data to the object attribute determination unit 22.

In addition, the moving object system of the object height data and the moving object of the object position data produced by the object position data creating section 12 are assigned correspondences, and the correspondence relationship can be referred from the object height data.

The object width data creation unit 21 obtains the distance measurement information of the frame at a plurality of times from the distance measurement data acquisition unit 11 and generates data of the estimated widths of the x-axis direction and the y-axis direction of the moving object. In addition, data for estimating the width in the x-axis direction and the y-axis direction of a moving object is referred to as object width data.

The object width data creation unit 21 outputs the created object width data to the object attribute determination unit 22.

In addition, the moving object system of the object width data and the moving object of the object position data produced by the object position data creation section 12 are assigned correspondence, and the correspondence relationship can be referred from the object width data.

The object attribute determination unit 22 obtains the position and movement speed data of the body from the position movement direction correlation unit 15 and the object height data creation unit 20 Obtain the object height data, obtain the object width data from the object width data creation section 21, and determine the attributes of the moving object based on the acquired data.

The object attribute determination unit 22 outputs data indicating the position, movement speed, and attributes of a moving object to the display unit 16 and the recording unit 17. Here, the data indicating the position, moving speed, and attributes of the moving object output by the object attribute determination unit 22 is referred to as attribute fusion data.

Since the hardware configuration of the moving object detection device 100b is the same as that described with reference to Figs. 5A and 5B in the first embodiment, redundant description is omitted.

The operation of the moving object detection device 100b according to the third embodiment will be described.

Fig. 15 is a flowchart illustrating the operation of the moving object detection device 100b according to the third embodiment of the present invention.

In the following description, the moving speed and attributes of the moving object having passed through the observation range 54 of the distance-measuring sensor 200 at time k are estimated.

The ranging data acquisition unit 11 and the object position data production unit 12 of the ranging sensor processing unit 101 are ranging information obtained from the ranging sensor 200 at a plurality of time frames, and the position of the center of gravity at time k is extracted. The moving object that has passed the observation range 54 of the ranging sensor 200 and generates object position data at time k (step ST1501). The specific operation is the same as step ST601 of FIG. 6 described in the first embodiment. The object position data creation unit 12 outputs the created object position data to the position moving direction correlation unit 15.

The object height data creation unit 20 of the distance measurement sensor processing unit 101 obtains the plurality of objects obtained by the distance measurement sensor 200 from the distance measurement data acquisition unit 11 Distance measurement information for each time frame, calculate the height of the moving object from the ground surface of the moving object that has passed the observation range 54 of the ranging sensor 200 at time k, and create the object height at time k Document (step ST1502).

The object height data creation unit 20 is a method for calculating the height of a moving object from the ranging information of the frame at each time, and for example, the method disclosed in Japanese Patent Application Laid-Open No. 2011-108223 is used. In this method, the distance measurement information of each moving object within the observation range 54 is divided in the height direction, and the range of the detected object is calculated at each height. For example, when the height of a pedestrian is calculated, the number of detected pedestrians can be calculated. The maximum height is taken as the height of the pedestrian.

The object height data creation unit 20 outputs the created object height data to the object attribute determination unit 22.

The object width data creation unit 21 of the ranging sensor processing unit 101 obtains the ranging information obtained by the ranging sensor 200 from the ranging data acquisition unit 11 at a plurality of time frames, and measures the center of gravity at time k. A moving object whose position has passed the observation range 54 of the distance measuring sensor 200, calculates the width in the x direction and the y direction of the moving object, and generates object width data at time k (step ST1503).

The object width data creation unit 21 is a method of calculating the width of a moving object from the ranging information of the frame at each time, and for example, the method disclosed in Japanese Patent Application Laid-Open No. 2011-108223 is used. This method divides the distance measurement information of each moving object within the observation range 54 in the distance direction, and can calculate the width of the pedestrian's body from the maximum and minimum values of the distance direction of the moving object detected in the name distance direction. The width in the direction perpendicular to the distance direction can be calculated from the time during which a moving object is continuously detected.

The object width data creation unit 21 outputs the created object width data to the object attribute determination unit 22.

The image acquisition sensor processing unit 102 generates the pixel velocity data at time k from the images at the frames of time obtained by the image acquisition sensor 300 (step ST1504). A specific method of generating the pixel speed data is the same as step ST602 of FIG. 6 described in the first embodiment. The pixel movement direction data creation unit 14 of the image acquisition sensor processing unit 102 outputs the pixel speed data at the time k to the position movement direction correlation unit 15.

The position moving direction correlation unit 15 uses the object height data at time k output by the object position data generating unit 12 in step ST1501 and the pixel velocity data at time k output by the pixel moving direction data generating unit 14 in step ST1504. The moving speed specified by the object position data at time k is estimated, and a fusion data about the moving object at time k is created (step ST1505). The specific operation is the same as step ST603 in FIG. 6 described in the first embodiment.

The position moving direction correlation unit 15 outputs the created fusion data to the object attribute determination unit 22.

The object attribute determination unit 22 is based on the object height data at time k produced by the object height data creation unit 20 in step ST1502 and the object width data at time k produced by the object width data creation unit 21 in step ST1503 and In step ST1505, the position of the center of gravity and the estimated speed of the moving object included in the fusion data at time k produced by the position-moving direction correlation unit 15 determine the attribute of the moving object, that is, the shape attribute (step ST1506).

For example, in the case of a pedestrian as a moving object, In the case where the height of a moving object is below the average height of an adult, and the width of the moving object is below the width of a typical body of an adult, and the moving speed is faster than the general walking speed, the object attribute determination unit 22 determines whether the moving object is The shape attribute is "kid". In addition, information required for determining shape attributes such as the average height of an adult, the width of a typical body of an adult, and information on general walking speed is preset and stored in a position that the object attribute determination unit 22 can refer to.

For example, when a pedestrian is used as a moving object, the height of a moving object is equal to or less than the average height of an adult, and the width near the ground of the moving object is the size of a wheel suitable for a wheelchair. When the walking speed is slow, the object attribute determination unit 22 determines that the shape attribute of the moving object is a "wheelchair user".

In addition, in the above, the examples of pedestrians are listed, but it is not limited to this, and it can also be used to determine the type of vehicle, the type of aircraft, the type of boat, Determination of the type of animal as a target.

The object attribute determination unit 22 determines whether or not the estimated velocity and shape attributes are all estimated for one or a plurality of moving objects specified by the object position data at time k (step ST1507).

In step ST1507, when all the estimated speed and shape attributes of one or a plurality of moving objects specified by the object position data at time k are not estimated (in the case of "NO" in step ST1507), the process returns to step ST1505.

In step ST1507, when all the estimated speed and shape attributes of all one or more moving objects specified by the object position data at time k are determined (in the case of "YES" in step ST1507), the object attribute is determined. The unit 22 determines the position of each moving object at time k estimated from steps ST1505 to ST1506, the estimated speed, shape attributes, and the fusion data of the frame at time other than time k, and determines whether the relative position of each moving object, Attributes of moving objects at speed (step ST1508). The attribute determined in this step ST1508 is called a group attribute.

For example, in the case of a pedestrian as a moving object, for a moving object that has passed the observation range 54 of the ranging sensor at time k, the past or future time frame within a fixed number from time k In the case where more than a fixed number of moving objects with the same position and the same moving direction are observed, the object attribute determining unit 22 determines that the group attribute of the moving object system at the time k is "one of the group pedestrians".

In addition, for example, when a pedestrian is used as a moving object, for a moving object whose shape attribute is "kid", a position near the moving object, the same moving speed, and the shape attribute is other than "kid" are observed. In the case of a moving object, the group attribute of the moving object with the matching shape attribute "kids" is regarded as a "child with a guardian", and the group attribute of the matching object with the shape attribute "other than children" is regarded as " Follow the child's guardian. "

In addition, in the above, the examples of pedestrians are listed, but it is not limited to this. It can also be used to determine traffic jams for vehicles, formations for aircrafts, fleets for ships, and Judgment of groups of animals as objects, etc.

The object attribute determination unit 22 is a group of the position and speed of the moving object estimated in step ST1505, the shape attributes of the moving object determined in step ST1506, and the group of moving objects determined in step ST1508. The attributes are output to the display unit 16 and the recording unit 17 as fusion data having attributes.

In addition, instead of the position moving direction correlation unit 15 in the position shown in FIG. 14, the number, position, and speed of moving objects can be estimated by the moving direction assumption value setting unit 18 and the moving direction determining unit 19 described in the second embodiment. . In this case, the object attribute determination unit 22 determines the shape attribute and the group attribute of the moving object based on the output of the movement direction determination unit 19.

In this way, if according to the third embodiment, the position and moving speed of the moving object are configured to use the distance measurement information of the distance measurement sensor 200 and the image information of the sensor 300 to obtain the distance from the distance measurement sensor. The height and width of the moving object calculated by the distance measurement information of the detector 200 determine the attributes of the moving object. Therefore, when the resolution of the image acquisition sensor 300 is low, or the contour of the moving object is acquired in the image due to low illumination Unclear conditions on the sensor can also determine attributes that are useful for analysis of moving objects.

Such a configuration is effective, for example, in a case where the proportion of children among pedestrians or the proportion of pedestrians acting in groups is estimated. The age structure of pedestrians or the existence of group pedestrians that are the cause of confusion are useful information in security systems, event management, market analysis, etc., but in order to determine the attributes of each pedestrian by using sensors such as visible light cameras to obtain sensors, The pedestrian's body part is extracted from the contour on the image, and the sensor is acquired in the low-resolution image, and the accuracy of the judgment is deteriorated. For example, in order to extract a pedestrian of a child from an image, it is necessary to extract a head position of the pedestrian from the image, and it is difficult to extract the image from a low-resolution image.

On the other hand, according to the third embodiment, the height and width of the moving object obtained by using the distance measuring sensor 200 and the observation results of the distance measuring sensor 200 and the image obtaining sensor 300 are merged. Gains The position and moving speed of the animal body determine the attributes of the moving object. Therefore, in the case where the resolution of the image acquisition sensor 300 is low and the outline of the moving object is not clear, the probability of determining the wrong attribute can also be reduced.

In addition, since the range-finding sensor 200 can also observe the shape in the depth direction of a moving object, compared with the image acquisition sensor 300, it is possible to reduce errors in the determination based on the attributes of the position and speed of the moving object. Probability of judgment.

In addition, in the first embodiment, the moving object detection device 100 is configured as shown in FIG. 4. However, the moving object detection device 100 includes an object position data creation unit 12, a pixel movement direction data creation unit 14, and a position. The moving direction correlation unit 15 can obtain the effects as described above.

The above description describes the case where a pedestrian is detected as a moving object. However, when a car, bicycle, wheelchair, train, airplane, ship, animal other than a human, a robot, etc. is detected as a moving object, the present invention can also be used. Invention, this is obvious.

The present invention is within the scope of the invention, and can be freely combined with each embodiment, modified with any constituent element of each embodiment, or omitted from any constituent element in each embodiment.

[Industrial applicability]

The moving object detection device of the present invention is configured to be able to estimate the number, position, and direction of moving objects with high accuracy by using a camera and a laser sensor installed at one place, so it can be applied to estimation The number and position of moving objects such as pedestrians, and moving object detection devices including the moving speed in the moving direction.

11‧‧‧ Ranging data acquisition department

12‧‧‧ Object position data production department

13‧‧‧Image acquisition department

14‧‧‧ Pixel production direction data production department

15‧‧‧Position moving direction related department

16‧‧‧Display

17‧‧‧Recording Department

100‧‧‧moving object detection device

101‧‧‧ ranging sensor processing unit

102‧‧‧Image acquisition sensor processing unit

103‧‧‧Fusion Processing Department

200‧‧‧ ranging sensor

300‧‧‧Image acquisition sensor

Claims (5)

A moving object detection device includes: an object position data production unit that calculates position coordinates of a moving object based on ranging information measured by a borrowing range sensor; a pixel moving direction data production unit that obtains sensing based on a borrowed image The image information captured by the camera calculates the estimated moving direction of each area on the image; and the position moving direction related part estimates the number of moving objects and the moving direction based on the position coordinates and the estimated moving direction. A moving object detection device includes: an object position data production unit that calculates position coordinates of a moving object based on ranging information measured by a borrowing range sensor; a pixel moving direction data production unit that obtains sensing based on a borrowed image The information of the image captured by the camera is used to calculate the estimated movement direction of each area on the image. The movement direction assumption value setting unit sets a plurality of movement direction assumption values for the moving object based on the position coordinates and the estimated movement direction. The determining unit determines the number of moving objects and the moving direction based on the plurality of moving direction hypotheses. For example, the moving object detection device according to item 2 of the patent application, wherein the moving direction determination unit calculates the probabilities of the plurality of moving direction hypotheses, and then determines the moving direction of the moving object based on the probability. For example, the moving object detection device of the scope of application for patent No. 1 further includes: an object height data production unit, which calculates the height of the moving object based on the ranging information; The object width data creation unit calculates the width of the moving object based on the ranging information; and the object attribute determination unit relies on the height and width of the moving object, and there are multiple cases where the moving object exists between the moving objects. The relative positional relationship and the relative moving direction between the moving objects determine the attributes of the moving object. For example, the moving object detection device under the scope of patent application No. 2 further includes: an object height data production unit, which calculates the height of the moving object based on the ranging information; an object width data production unit, which is based on the ranging information, Calculate the width of the moving object; and the object attribute determination unit is based on the height and width of the moving object, and when there are a plurality of moving objects, the relative positional relationship between the moving objects and the relative position between the moving objects The moving direction of, to determine the attributes of the moving object.