WO2018154654A1 - Moving object detection device and moving object detection program - Google Patents

Abstract

A difference image generation unit (110) generates a first difference image representing the difference in luminance between a target frame and a preceding frame, and a second difference image representing the difference in luminance between the target frame and a succeeding frame. An edge image generation unit (120) generates a first edge image representing edges in the first difference image, and a second edge image representing edges in the second difference image. A moving object image generation unit (130) generates a moving object image indicating edges common to the first edge image and the second edge image.

Description

Moving body detection apparatus and moving body detection program

The present invention relates to a technique for detecting a moving object shown in a video frame.

In order to detect a moving object such as a person by image processing, a technique using a difference between frames is widely used. The interframe difference is a difference in luminance information between images.
However, since luminance information changes due to changes in illuminance in the shooting environment, noise other than the movement of moving objects is included in the result of the inter-frame difference.

Patent Document 1 describes a technique for robustly recognizing differences in moving direction between moving objects and changes in moving objects through inter-frame differences and learning.
However, Patent Document 1 does not describe a technique for removing a change in luminance information caused by a change in illuminance when external light is generated.
Therefore, in the technique of Patent Document 1, a change in luminance information caused by a change in illuminance when external light occurs is detected as a motion of a moving object.

JP 2011-90708 A

When detecting a moving object by image processing, factors other than the movement of the moving object affect the luminance information of the image, so that the detection accuracy decreases. As a factor other than the movement of the moving body, for example, a change in illuminance due to turning on or off the illumination can be considered.

An object of the present invention is to prevent a decrease in detection accuracy caused by factors other than the movement of a moving object.

The moving object detection apparatus of the present invention is
Using a target frame that is a video frame, a previous frame that is a video frame before the target frame, and a rear frame that is a video frame after the target frame, the luminance of the target frame and the previous frame is determined. A difference image generation unit that generates a first difference image representing a difference and a second difference image representing a difference in luminance between the target frame and the subsequent frame;
An edge image generation unit that generates a first edge image representing an edge in the first difference image and a second edge image representing an edge in the second difference image;
A moving object image generation unit configured to generate a moving object image indicating an edge common to the first edge image and the second edge image;

According to the present invention, an image showing an edge common to the first edge image and the second edge image is generated as a moving object image. Therefore, factors other than the movement of the moving object are removed, and a decrease in detection accuracy can be suppressed.

1 is a configuration diagram of a moving object detection device 100 according to Embodiment 1. FIG. FIG. 6 is a relationship diagram of a previous frame 201, a target frame 202, and a first difference image 211 in the first embodiment. FIG. 6 is a relationship diagram among a target frame 202, a rear frame 203, and a second difference image 212 in the first embodiment. FIG. 4 is a government benefit diagram of the first difference image 211 and the first edge image 221 according to the first embodiment. FIG. 5 is a relationship diagram between a second difference image 212 and a second edge image 222 in the first embodiment. FIG. 6 is a relationship diagram between a first edge image 221, a second edge image 222, and a moving object image 230 in the first embodiment. 3 is a flowchart of a moving object detection method according to the first embodiment. The figure which shows the difference image 293 when brightness | luminance information changes resulting from external light. The block diagram of the moving body detection apparatus 100 in Embodiment 2. FIG. FIG. 10 is a relationship diagram among a moving object image 230, a moving object block 241 and a moving object region 242 according to the second embodiment. FIG. 10 shows tracking of a moving object region 242 in the second embodiment. FIG. 10 shows tracking of a moving object region 242 in the second embodiment. FIG. 10 is a diagram showing tracking of a stationary area 250 in the second embodiment. FIG. 10 is a diagram showing tracking of a stationary area 250 in the second embodiment. 10 is a flowchart of moving object region detection processing in the second embodiment. 10 is a flowchart of moving object region tracking processing according to the second embodiment. 10 is a flowchart of moving object region tracking processing according to the second embodiment. 10 is a flowchart of moving object region tracking processing according to the second embodiment. The hardware block diagram of the moving body detection apparatus 100 in embodiment.

In the embodiment and the drawings, the same reference numerals are given to the same elements and corresponding elements. Description of elements having the same reference numerals will be omitted or simplified as appropriate. The arrows in the figure mainly indicate the flow of data or the flow of processing.

Embodiment 1 FIG.
A mode for detecting a moving object shown in a video frame will be described with reference to FIGS.

*** Explanation of configuration ***
Based on FIG. 1, the structure of the moving body detection apparatus 100 is demonstrated.
The moving object detection device 100 is a computer including hardware such as a processor 901, a memory 902, an auxiliary storage device 903, and an input / output interface 904. These hardwares are connected to each other via signal lines.

The processor 901 is an IC (Integrated Circuit) that performs arithmetic processing, and controls other hardware. For example, the processor 901 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit).
The memory 902 is a volatile storage device. The memory 902 is also called main memory or main memory. For example, the memory 902 is a RAM (Random Access Memory). Data stored in the memory 902 is stored in the auxiliary storage device 903 as necessary.
The auxiliary storage device 903 is a nonvolatile storage device. For example, the auxiliary storage device 903 is a ROM (Read Only Memory), a HDD (Hard Disk Drive), or a flash memory. Data stored in the auxiliary storage device 903 is loaded into the memory 902 as necessary.

The input / output interface 904 is a port to which an input device and an output device are connected. For example, the input / output interface 904 is a USB terminal, the input device is a receiver, a keyboard and a mouse, and the output device is a transmitter and a display. USB is an abbreviation for Universal Serial Bus.

The moving object detection apparatus 100 includes software elements such as a difference image generation unit 110, an edge image generation unit 120, and a moving object image generation unit 130. A software element is an element realized by software.

The auxiliary storage device 903 stores a moving object detection program for causing a computer to function as the difference image generation unit 110, the edge image generation unit 120, and the moving object image generation unit 130. The moving object detection program is loaded into the memory 902 and executed by the processor 901.
Further, the auxiliary storage device 903 stores an OS (Operating System). At least a part of the OS is loaded into the memory 902 and executed by the processor 901.
That is, the processor 901 executes the moving object detection program while executing the OS.
Data obtained by executing the moving object detection program is stored in a storage device such as the memory 902, the auxiliary storage device 903, a register in the processor 901, or a cache memory in the processor 901.

The memory 902 functions as a storage unit 191 that stores data. However, another storage device may function as the storage unit 191 instead of the memory 902 or together with the memory 902.
The input / output interface 904 functions as a reception unit 192 that receives input. The input / output interface 904 functions as an output unit 193 that outputs data.

The moving object detection apparatus 100 may include a plurality of processors that replace the processor 901. The plurality of processors share the role of the processor 901.

The moving object detection program can be stored in a computer-readable manner on a nonvolatile storage medium such as a magnetic disk, an optical disk, or a flash memory. A non-volatile storage medium is a tangible medium that is not temporary.

*** Description of functions ***
The functions of the difference image generation unit 110, the edge image generation unit 120, and the moving object image generation unit 130 will be described.

Based on FIG. 2 and FIG. 3, the function of the difference image generation part 110 is demonstrated.
In FIG. 2, the difference image generation unit 110 generates a first difference image 211 using the target frame 202 and the previous frame 201.
The target frame 202 is a video frame. A video frame is image data. The target frame 202 in FIGS. 2 and 3 is data of an image obtained by photographing a room with a moving person from above.
The previous frame 201 is a video frame before the target frame 202. Specifically, the previous frame 201 is a video frame immediately before the target frame 202. In FIG. 2, the room in the target frame 202 is brighter than the room in the previous frame 201.
The first difference image 211 is an image representing a difference in luminance between the target frame 202 and the previous frame 201. In FIG. 2, since the room in the target frame 202 is brighter than the room in the previous frame 201, the first difference image 211 shows the illuminance difference of the room in addition to the moving person.

In FIG. 3, the difference image generation unit 110 generates a second difference image 212 using the target frame 202 and the rear frame 203.
The rear frame 203 is a video frame after the target frame 202. Specifically, the rear frame 203 is a video frame next to the target frame 202. In FIG. 3, the brightness of the room in the target frame 202 is not different from the brightness of the room in the rear frame 203.
The second difference image 212 is an image representing a difference in luminance between the target frame 202 and the rear frame 203. In FIG. 3, the brightness of the room in the target frame 202 is not different from the brightness of the room in the rear frame 203, and thus the moving person appears mainly in the second difference image 212.

The function of the edge image generation unit 120 will be described based on FIGS.
In FIG. 4, the edge image generation unit 120 generates a first edge image 221 using the first difference image 211.
The first edge image 221 is an image representing an edge in the first difference image 211.
An edge is a location that becomes a boundary of luminance. That is, the luminance changes greatly at the edge.

In FIG. 5, the edge image generation unit 120 generates a second edge image 222 using the second difference image 212.
The second edge image 222 is an image representing an edge in the second difference image 212.

Based on FIG. 6, the function of the moving object image generation unit 130 will be described.
The moving object image generation unit 130 generates an image indicating an edge common to the first edge image 221 and the second edge image 222. The generated image is referred to as a moving object image 230.
The moving object image 230 corresponds to an image showing the edge of the moving object shown in the target frame. A moving body image 230 in FIG. 6 shows an edge of a person reflected in the target frame 202 (see FIGS. 2 and 3).

*** Explanation of operation ***
The operation of the moving object detection apparatus 100 corresponds to a moving object detection method. The procedure of the moving object detection method corresponds to the procedure of the moving object detection program.

The moving object detection method will be described with reference to FIG.
In step S101, the reception unit 192 receives a video frame.
The storage unit 191 stores the video frame in association with the time when the video frame is received.

In step S <b> 111, the difference image generation unit 110 determines whether the target frame is stored in the storage unit 191.
The target frame is a video frame associated with the time before the video frame accepted in step S101.
If the target frame is stored in the storage unit 191, the process proceeds to step S112.
If the target frame is not stored in the storage unit 191, the process proceeds to step S101.

In step S112, the difference image generation unit 110 generates a difference image using the target frame and the subsequent frame. The subsequent frame is the video frame accepted in step S101.
The storage unit 191 stores the difference image in association with the time when the difference image is generated.

Specifically, the difference image generation unit 110 generates a difference image as follows.
The difference image generation unit 110 performs the following processing for each pixel of the target frame.
The difference image generation unit 110 selects a pixel of the subsequent frame corresponding to the pixel of the target frame, calculates a luminance difference between the pixel of the target frame and the pixel of the subsequent frame, and a pixel of the difference image corresponding to the pixel of the target frame Is selected, and the luminance difference is set to the pixels of the difference image.
The pixel of the Y frame corresponding to the pixel of the X frame is a pixel identified by the same coordinate value as the coordinate value for identifying the pixel of the X frame among the pixels included in the Y frame. That is, the Y frame pixel corresponding to the pixel located at (u, v) in the X frame is the pixel located at (u, v) in the Y frame.

In step S121, the edge image generation unit 120 generates an edge image using the difference image generated in step S112.
The storage unit 191 stores the edge image in association with the time when the edge image is generated.

Specifically, the edge image generation unit 120 generates an edge image by performing edge extraction on the difference image.
Edge extraction is a process for extracting a portion that becomes a boundary of luminance as an edge. Edge extraction is a conventional technique and is also called edge detection.

In step S131, the moving object image generation unit 130 determines whether the first edge image is stored in the storage unit 191.
The first edge image is an edge image associated with the time before the edge image generated in step S121.
If the first edge image is stored in the storage unit 191, the process proceeds to step S132.
If the first edge image is not stored in the storage unit 191, the process proceeds to step S101.

In step S132, the moving object image generation unit 130 generates a moving object image using the first edge image and the second edge image. The second edge image is the edge image generated in step S121.
The storage unit 191 stores the moving body image in association with the time when the moving body image is generated.

Specifically, the moving object image generation unit 130 generates a moving object image as follows.
The moving object image generation unit 130 performs the following processing for each pixel of the first edge image.
The moving object image generation unit 130 selects a pixel of the second edge image corresponding to the pixel of the first edge image, calculates a logical product of the pixel of the first edge image and the pixel of the second edge image, and calculates the first edge A moving image pixel corresponding to the image pixel is selected, and a logical product is set to the moving image pixel.
The pixel of the Y image corresponding to the pixel of the X image is a pixel identified by the same coordinate value as the coordinate value for identifying the pixel of the X image among the pixels included in the Y image. That is, the pixel of the Y image corresponding to the pixel located at (u, v) of the X image is the pixel located at (u, v) of the Y image.
For example, in the first edge image and the second edge image, the value of a pixel that is a part of the edge is “1”, and the value of a pixel that is not a part of the edge is “0”. When the values of the pixels of the first edge image and the pixels of the second edge image are “1”, the value of the pixel of the moving object image is “1”. When the value of at least one of the pixel of the first edge image and the pixel of the second edge image is “0”, the value of the pixel of the moving object image is “0”.

In step S141, the output unit 193 outputs the moving body image generated in step S132.
After step S141, the process proceeds to step S101.

*** Effects of Embodiment 1 ***
The first embodiment solves the problem of detecting only moving objects from luminance information that changes due to external light. Since the edge of an object does not change even when external light is generated, detection of a moving object and a moving object position is realized by using robust edge information for the external light.
According to the first embodiment, since the luminance change in the video frame caused by the illuminance change is removed, the motion of the moving object can be detected robustly.

FIG. 8 shows an example of a video frame whose luminance information has changed due to external light.
The front frame 291 is a video frame obtained before the rear frame 292, and the rear frame 292 is a video frame obtained after the front frame 291. The difference image 293 is an image representing a difference in luminance between the previous frame 291 and the subsequent frame 292.
Since the external light has increased between the time when the front frame 291 is obtained and the time when the rear frame 292 is obtained, the front frame 291 is darker than the rear frame 292 and the rear frame 292 is brighter than the front frame 291.
For this reason, the difference image 293 shows not only the moved person (right person) but also the entire person including the unmoved person (left person) and the unmoved background.
However, since the moving object detection apparatus 100 uses the first difference image and the second difference image, the moving object in the video frame is extracted as edge information even if the luminance information of the video frame changes greatly due to external light. Is possible.

Embodiment 2. FIG.
Regarding the form for tracking a moving object, differences from the first embodiment will be mainly described with reference to FIGS.

*** Explanation of configuration ***
Based on FIG. 9, the structure of the moving body detection apparatus 100 is demonstrated.
The moving object detection apparatus 100 includes a moving object region detecting unit 140 and a moving object region tracking unit 150 as software elements in addition to the difference image generating unit 110, the edge image generating unit 120, and the moving object image generating unit 130.
The moving object detection program is a program for causing a computer to function as the difference image generating unit 110, the edge image generating unit 120, the moving object image generating unit 130, the moving object region detecting unit 140, and the moving object region tracking unit 150.

*** Description of functions ***
As described in Embodiment 1, the moving object image generation unit 130 generates a moving object image for each set of three video frames that are consecutive in time series.

Based on FIG. 10, the function of the moving body area | region detection part 140 is demonstrated.
The moving object region detection unit 140 detects one or more moving object regions 242 corresponding to the one or more moving objects shown in the target frame from the moving object image 230 based on the edges indicated in the moving object image 230.
The moving object area 242 is an area representing a moving object.

Specifically, the moving object region detection unit 140 detects the moving object region 242 by the following procedure.
(1) The moving object region detection unit 140 divides the moving object image 230 into a plurality of blocks.
(2) The moving object region detection unit 140 calculates the number of edge pixels for each block. The number of edge pixels is the number of pixels indicating a part of the edge.
(3) The moving object region detection unit 140 identifies one or more moving object blocks 241 based on the number of edge pixels for each block. The moving object block 241 is a block corresponding to the number of edge pixels larger than the pixel number threshold.
(4) The moving object region detection unit 140 generates a rectangular region including adjacent moving object blocks 241. The generated area is the moving object area 242.

Based on FIGS. 11 and 12, the function of the moving object region tracking unit 150 will be described.
In FIG. 11, the moving object region tracking unit 150 detects a moving object region 242T corresponding to the previous moving object region 242B from the target moving object image for each previous moving object region 242B included in the previous moving object image.
In FIG. 12, the moving body region tracking unit 150 detects a previous moving body region 242B that does not correspond to any moving body region 242T included in the target moving body image from the previous moving body image.
Then, the moving body region tracking unit 150 detects a region corresponding to the detected previous moving body region 242B as a still region 250 from the target moving body image.
The stationary region 250 is a region corresponding to a moving object that is stationary.

Furthermore, the moving object region tracking unit 150 detects a moving object region corresponding to the still region for each still region included in the previous moving object image from the target moving object image.
Next, the moving body region tracking unit 150 detects a still region that does not correspond to any moving body region included in the target moving body image from the previous moving body image. The detected still area is called a target still area.
Then, the moving body region tracking unit 150 detects a region corresponding to the target stationary region from the target moving body image. The detected area is referred to as a still area in the target moving image.

However, when the target still region corresponds to a still region detected from a moving image corresponding to a time that is more than the tracking time before the time corresponding to the target moving image, the moving region tracking unit 150 may include the target moving image in the target moving image. Discard the static area.
The tracking time is a time during which the stationary region is tracked on the assumption that the moving body actively moving is stationary in the stationary region, and is a predetermined time.

Specifically, the moving object region tracking unit 150 operates as follows.
First, the moving object region tracking unit 150 sets the tracking time as the remaining time when a stationary region is detected for the first time for each stationary region.
Next, the moving object region tracking unit 150 performs, for each stationary region, moving object images from when the stationary region is detected for the first time until a moving object region corresponding to the stationary region is detected (or until the stationary region is discarded). Each time it is generated, the remaining time in the still area is reduced.
Then, the moving object region tracking unit 150 discards the still region in which the remaining time becomes zero.

In FIG. 13, the previous moving object image includes a moving object region 242B. There is no moving object region corresponding to the moving object region 242B in the target moving object image. Therefore, the still region 250 is detected from the target moving object image. The next moving object image includes a moving object region 242N. The moving object region 242N is a moving object region corresponding to the stationary region 250.
It is assumed that the moving object region 242N corresponding to the stationary region 250 is detected before the tracking time elapses. In that case, the moving object region 242B, the stationary region 250, and the moving object region 242N are considered to be active moving object regions. The active moving object area is an area representing an active moving object. An active moving object is a moving object that is not a moved object. That is, the active moving body is a moving body that moves by itself. For example, the active moving object is a person.

In FIG. 14, the target moving object image includes a moving object region 242B. There is no moving object region corresponding to the moving object region 242B in the target moving object image. Therefore, the still region 250 is detected from the target moving object image. There is no moving object region corresponding to the still region 250 in the next moving object image.
It is assumed that the moving object region corresponding to the stationary region 250 has not been detected before the tracking time has elapsed. In that case, the moving object region 242B and the stationary region 250 are considered to be passive moving object regions. The passive moving object area is an area representing a passive moving object. Passive moving objects are moved. In other words, passive moving objects are things that do not move by themselves. For example, the passive moving body is a chair.
When the tracking time has elapsed, the static area 250 is discarded.

*** Explanation of operation ***
Based on FIG. 15, a moving body area | region detection process is demonstrated.
The moving object region detection process is performed by the moving object region detection unit 140 every time a new moving object image is generated.

In step S201, the moving object region detection unit 140 divides the moving object image into a plurality of blocks.
Specifically, the moving object region detection unit 140 divides the moving object image for each region of W × H pixels. A region of W × H pixels is a block. W and H are arbitrary integers.

In step S202, the moving object region detection unit 140 calculates the number of edge pixels for each block.

Specifically, the moving object region detection unit 140 calculates the number of edge pixels for each block as follows.
When the value of the pixel indicating the edge is X, the moving object region detection unit 140 counts the number of pixels for which X is set. The number of pixels for which X is set is the number of edge pixels. X is a specific value.

In step S203, the moving object region detection unit 140 identifies moving object blocks included in a plurality of blocks.

Specifically, the moving object region detection unit 140 determines whether the target block is a moving object block as follows.
The moving object region detection unit 140 compares the number of edge pixels of the target block with a pixel number threshold value. The number of edge pixels is a predetermined value.
When the number of edge pixels of the target block is greater than or equal to the pixel number threshold, the moving object region detection unit 140 determines that the target block is a moving object block.
When the number of edge pixels of the target block is less than the pixel number threshold, the moving object region detection unit 140 determines that the target block is not a moving object block.

At step S203, one or more moving object blocks are specified.

In step S204, the moving object region detection unit 140 generates one or more moving object regions based on one or more moving object blocks.

Specifically, the moving object region detection unit 140 generates a moving object region for each moving object block as follows.
The moving object region detection unit 140 determines whether there is a moving object block adjacent to the target moving object block.
When there is a moving object block adjacent to the target moving object block, the moving object region detection unit 140 generates a rectangle that surrounds the target moving object block and the adjacent moving object block. A region surrounded by the generated rectangle is a moving object region.
When there is no moving object block adjacent to the target moving object block, the target moving object block is a moving object region.

In step S205, the moving object region detection unit 140 assigns an area identifier to the moving object region for each moving object region. The area identifier is an identifier for identifying a moving object area. For example, the area identifier is a serial number.
Specifically, the moving object region detection unit 140 stores the position information and the region identifier in the storage unit 191 in association with each other for each moving object region. The position information is information for specifying the position of the moving object region in the moving object image. For example, the position information is each coordinate value of four vertices in the moving object region.

The moving object region tracking process will be described with reference to FIGS.
The moving object region tracking process is executed by the moving object region tracking unit 150 after the moving object region detection process every time a new moving object image is generated.
In the description of the moving body region tracking process, a new moving body image is referred to as a target moving body region, and a moving body image generated immediately before the new moving body image is referred to as a previous moving body image.

In step S211 (see FIG. 16), the moving body region tracking unit 150 selects one unselected moving body region from the moving body regions included in the target moving body image.

In the description after step S211, the moving object region selected in step S211 is referred to as a target moving object region.

In step S212, the moving object region tracking unit 150 calculates the distance between the target moving object region and the tracking region for each tracking region included in the previous moving object image.
The tracking area is a moving area included in the previous moving body image or a still area included in the previous moving body image.
Specifically, the moving object region tracking unit 150 calculates the distance between the target moving object region and the tracking region using the position information of the target moving object region and the position information of the tracking region.
For example, the moving object region tracking unit 150 calculates the distance from the upper left vertex of the target moving object region to the upper left vertex of the tracking region.

In step S213, the moving object region tracking unit 150 determines whether there is a tracking region corresponding to the target moving object region. The tracking area corresponding to the target moving object area is referred to as a corresponding tracking area.
Specifically, the moving object region tracking unit 150 determines whether there is a tracking region whose distance from the target moving object region is equal to or less than the distance threshold. A tracking region whose distance from the target moving object region is equal to or smaller than the distance threshold is a corresponding tracking region.
If there is a corresponding tracking area, the process proceeds to step S214.
If there is no corresponding tracking area, the process proceeds to step S215.

In step S214, the area identifier of the target moving object area is updated to the area identifier of the corresponding tracking area.

In step S215, the moving object region tracking unit 150 determines whether there is an unselected moving object region in the moving object regions included in the target moving object image.
If there is an unselected moving body region, the process proceeds to step S211.
If there is no unselected moving object region, the process proceeds to step S221 (see FIG. 17).

In step S221 (see FIG. 17), the moving object region tracking unit 150 selects one unselected tracking region from the tracking regions included in the previous moving object image.

In the description after step S212, the tracking area selected in step S221 is referred to as a target tracking area.

In step S222, the moving object region tracking unit 150 determines whether a moving object region included in the target moving object image includes a moving object region corresponding to the target tracking region. The moving object area corresponding to the target tracking area is referred to as a corresponding moving object area.
Specifically, the moving object region tracking unit 150 determines whether a moving object region having the same region identifier as that of the target tracking region is included in the target moving object image. A moving object area having the same area identifier as the target tracking area is a corresponding moving object area.
If there is a corresponding moving body region, the process proceeds to step S223.
If there is no corresponding moving object region, the process proceeds to step S231 (see FIG. 18).

In step S223, the moving object region tracking unit 150 determines whether the target tracking region is a moving object region or a stationary region.
Specifically, the moving object region tracking unit 150 determines whether the remaining time is stored in association with the region identifier of the target tracking region. When the remaining time is recorded in association with the area identifier of the target tracking area, the target tracking area is a stationary area. When the remaining time is not recorded in association with the area identifier of the target tracking area, the target tracking area is a moving object area.
If the target tracking area is a moving object area, the process proceeds to step S235 (see FIG. 18).
If the target tracking area is a stationary area, the process proceeds to step S224.

In step S224, the moving object region tracking unit 150 initializes the remaining time of the target tracking region.
Specifically, the moving object region tracking unit 150 updates the remaining time associated with the region identifier of the target tracking region to the tracking time.
After step S224, the process proceeds to step S235 (see FIG. 18).

In step S231 (see FIG. 18), the moving object region tracking unit 150 generates a still region corresponding to the target tracking region in the target moving image. The stationary area corresponding to the target tracking area is called a corresponding stationary area.
Specifically, the moving object region tracking unit 150 stores the position information of the corresponding still region, the region identifier of the corresponding still region, and the remaining time of the corresponding still region in the storage unit 191 in association with the target moving object image. The position information of the corresponding still area is the same as the position information of the target tracking area. The area identifier of the corresponding still area is the same as the area identifier of the target tracking area. The remaining time of the corresponding still area is the same as the tracking time.

In step S232, the moving object region tracking unit 150 determines whether the target tracking region is a moving object region or a stationary region.
Specifically, the moving object region tracking unit 150 determines whether the remaining time is stored in association with the region identifier of the target tracking region. When the remaining time is recorded in association with the area identifier of the target tracking area, the target tracking area is a stationary area. When the remaining time is not recorded in association with the area identifier of the target tracking area, the target tracking area is a moving object area.
If the target tracking area is a moving object area, the process proceeds to step S235.
If the target tracking area is a static area, the process proceeds to step S233.

In step S233, the moving object region tracking unit 150 takes over the remaining time from the target tracking region to the corresponding still region.
Specifically, the moving object region tracking unit 150 updates the remaining time associated with the region identifier of the target still region to the same time as the remaining time associated with the region identifier of the target tracking region.

In step S234, the moving object region tracking unit 150 reduces the remaining time of the corresponding still region.
Specifically, the moving object region tracking unit 150 reduces the unit elapsed time from the remaining time associated with the region identifier of the corresponding still region. The unit elapsed time is a time determined in advance as a time interval at which a video frame is input or a time interval at which a moving object image is generated.

In step S235, the moving object region tracking unit 150 determines whether there is an unselected tracking region in the tracking regions included in the previous moving object image.
If there is an unselected tracking area, the process returns to step S221 (see FIG. 17).
If there is no unselected tracking area, the process proceeds to step S241.

In step S241, the moving object region tracking unit 150 discards a still region in which the remaining time is zero among the still regions included in the target moving object image.
Specifically, the moving object region tracking unit 150 refers to the remaining time associated with the region identifier of the still region for each still region included in the target moving image. If the remaining time is less than or equal to zero, the moving object region tracking unit 150 deletes the information on the still region (region identifier, position information, remaining time) from the storage unit 191.
After step S241, the moving object region tracking process ends.

*** Effects of Embodiment 2 ***
The moving object detection apparatus 100 can extract the moving object region in the video frame as edge information even if the luminance information of the video frame changes greatly due to disturbance.
Furthermore, the moving object detection apparatus 100 can detect the position of a stationary moving object using the edge information. The moving object detection apparatus 100 can determine whether the stationary object is an active moving object or a passive moving object.

*** Supplement to the embodiment ***
In the embodiment, the function of the moving object detection apparatus 100 may be realized by hardware.
FIG. 19 shows a configuration when the function of the moving object detection apparatus 100 is realized by hardware.
The moving object detection apparatus 100 includes a processing circuit 990. The processing circuit 990 is also called a processing circuit.
The processing circuit 990 is a dedicated electronic circuit that implements the difference image generation unit 110, the edge image generation unit 120, the moving object image generation unit 130, the moving object region detection unit 140, the moving object region tracking unit 150, and the storage unit 191.
For example, the processing circuit 990 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, GA, ASIC, FPGA, or a combination thereof. GA is an abbreviation for Gate Array, ASIC is an abbreviation for Application Specific Integrated Circuit, and FPGA is an abbreviation for Field Programmable Gate Array.

The moving object detection apparatus 100 may include a plurality of processing circuits that replace the processing circuit 990. The plurality of processing circuits share the role of the processing circuit 990.

The embodiment is an example of a preferred embodiment and is not intended to limit the technical scope of the present invention. The embodiment may be implemented partially or in combination with other embodiments. The procedure described using the flowchart and the like may be changed as appropriate.

100 moving object detection device, 110 difference image generating unit, 120 edge image generating unit, 130 moving object image generating unit, 140 moving object region detecting unit, 150 moving object region tracking unit, 191 storage unit, 192 receiving unit, 193 output unit, 201 previous frame 202 target frame, 203 post frame, 211 first difference image, 212 second difference image, 221 first edge image, 222 second edge image, 230 moving object image, 241 moving object block, 242 moving object area, 250 stationary area, 901 Processor, 902 memory, 903 auxiliary storage device, 904 input / output interface, 990 processing circuit.

Claims (7)

1. Using a target frame that is a video frame, a previous frame that is a video frame before the target frame, and a rear frame that is a video frame after the target frame, the luminance of the target frame and the previous frame is determined. A difference image generation unit that generates a first difference image representing a difference and a second difference image representing a difference in luminance between the target frame and the subsequent frame;
   An edge image generation unit that generates a first edge image representing an edge in the first difference image and a second edge image representing an edge in the second difference image;
   A moving object detection apparatus comprising: a moving object image generating unit that generates a moving object image indicating an edge common to the first edge image and the second edge image.
2. The moving object area detection part which detects one or more moving object area | regions corresponding to the one or more moving object reflected in the said object frame from the said moving object image based on the edge shown by the said moving object image. Moving object detection device.
3. The moving object region detection unit divides the moving object image into a plurality of blocks, calculates the number of edge pixels, which is the number of pixels indicating a part of the edge for each block, and corresponds to the number of edge pixels larger than the pixel number threshold. The moving body detection apparatus according to claim 2, wherein a moving body block that is a block is specified, and a rectangular area including adjacent moving body blocks is generated as the moving body area.
4. The moving image generation unit generates a moving image for each set of three consecutive video frames,
   The moving object detection device detects a moving object area corresponding to a previous moving object area for each previous moving object area included in a previous moving object image from a target moving object image, and detects any moving object area included in the target moving object image. A moving body region tracking unit that detects a previous moving body region that does not correspond to the previous moving body image and detects a region corresponding to the detected previous moving body region as a stationary region from the target moving body image. The moving object detection device according to claim 3.
5. The moving object region tracking unit detects a moving object region corresponding to a stationary region for each stationary region included in the previous moving object image from the target moving image, and corresponds to any moving object region included in the target moving image. The moving body detection device according to claim 4, wherein a still area that is not detected is detected from the previous moving body image, and an area corresponding to the detected still area is detected from the target moving body image as a still area in the target moving body image.
6. The moving object region tracking unit corresponds to a time that is equal to or longer than a tracking time before a time when a target still region that is a still region that does not correspond to any moving object region included in the target moving image corresponds to the target moving image. The moving object detection device according to claim 5, wherein when the object corresponds to a still area detected from the moving object image, the still area in the target moving object image is discarded.
7. Using a target frame that is a video frame, a previous frame that is a video frame before the target frame, and a rear frame that is a video frame after the target frame, the luminance of the target frame and the previous frame is determined. A difference image generation process for generating a first difference image representing a difference and a second difference image representing a difference in luminance between the target frame and the subsequent frame;
   Edge image generation processing for generating a first edge image representing an edge in the first difference image and a second edge image representing an edge in the second difference image;
   A moving object detection program for causing a computer to execute a moving object image generation process for generating a moving object image indicating an edge common to the first edge image and the second edge image.

Images