JP2020149221A - Image processing device and image processing program - Google Patents

Abstract

To provide an image processing device and an image processing program capable of detecting a low-speed moving body from a photographic image by using a result of inter-frame difference processing.SOLUTION: In a security system, difference processing means 44 of an image sensor makes a circumscribed rectangle of a background difference area to be a determination area on the basis of a photographic image 36 and a background image 38. The difference processing means 44 performs inter-frame difference processing between respective photographic images 36 photographed within a difference period, and acquires a plurality of inter-frame difference images including an inter-frame difference area. Low-speed moving body determination means 46 acquires a first area being an area of the inter-frame difference area within the determination area in an object inter-frame difference image and a second area being an area of a stored inter-frame difference area within the determination area in a stored difference image obtained by storing inter-frame difference areas within a plurality of inter-frame difference images, and determines that a low-speed moving body area exists within the determination area in the case that an area increasing rate from the first area to the second area is equal to or more than a prescribed increasing rate threshold.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing apparatus and an image processing program for detecting a moving body region indicating an image region of a moving body from within a captured image.

Conventionally, a process of detecting a moving body is performed from a captured image obtained by photographing a predetermined area (monitoring area) with a camera. For example, when a moving object such as a person invades the monitoring area, the person is extracted from the captured image.

As a process for detecting a moving object from a captured image, background subtraction processing for extracting a background subtraction region between the captured image and a background image (an image in which no moving object exists in the monitoring area) is generally performed. is there. However, in the background subtraction processing, when the door is opened or closed or the luggage is moved, the background subtraction region is continuously extracted, so that the detection accuracy of the moving object that is originally desired to be detected is lowered. Therefore, in addition to the processing result of background subtraction processing, a frame for extracting an inter-frame difference region between a captured image and a past image (for example, an image one to several frames before) that is captured in the past. A moving object may be detected in consideration of the processing result of the inter-subtraction processing. For example, a moving body is detected based on a pixel region extracted as a background subtraction region and extracted as an interframe subtraction region.

For example, Patent Document 1 describes a photographed image by background subtraction processing between the photographed image and the background image, and frame subtraction processing between the photographed image and a past image that is temporally adjacent to the photographed image. An image sensor that detects a detection target is disclosed.

JP-A-2015-46811

When a moving body (low-speed moving body) having an extremely slow moving speed exists in the monitoring area, the position of the moving body may be almost the same between the captured image and the past image. In this case, since the inter-frame difference processing extracts the inter-frame difference region between the captured image and the past image, the low-speed moving body may not be extracted as the inter-frame difference region in the inter-frame difference processing. .. As described above, conventionally, there have been cases where a low-speed moving object cannot be appropriately detected from a captured image by inter-frame difference processing.

An object of the present invention is to detect a low-speed moving object from a captured image by using the result of inter-frame difference processing.

The present invention is an image processing device that detects a low-speed moving body region indicating an image region of a low-speed moving body from a target image among captured images in which a predetermined region is sequentially captured, and is a background image obtained by capturing the background of the predetermined region. A determination area including a background subtraction region extracted by background subtraction processing between the target image and the target image is obtained, and frame-to-frame difference processing between the captured images captured within the difference period including the shooting time of the target image. From the difference processing means for obtaining a plurality of inter-frame subtraction regions and the first area which is the area of the inter-frame subtraction region between the target image and the other captured image in the determination region. When the area increase rate to the second area, which is the area of the inter-frame difference region in the accumulated difference image in which a plurality of inter-frame subtraction regions are accumulated, is equal to or greater than the increase rate threshold, the low-speed moving body region is present in the determination region. It is an image processing apparatus including a low-speed moving body determining means for determining.

Desirably, a label shaping means for shaping the background subtraction region based on the inter-frame subtraction region in the determination region in the accumulated difference image determined to have the low-speed moving body region is further provided. It is a feature.

Desirably, the image region outside the determination region determined to have the low-speed moving body region is determined to be the stable region by the inter-frame difference processing, and the image region not extracted as the inter-frame difference region during the stability determination period is determined. The background image update means for updating the background image using the image region determined to be the stable region is further provided.

Desirably, the difference processing means is characterized in that the smaller the ratio of the first area to the area of the determination area, the larger the difference period is set.

Desirably, a third area indicating the area of the background subtraction region in the determination region is obtained, and when the difference between the first area and the third area is equal to or less than the area difference threshold value, the slow movement to the determination region is performed. It is characterized in that it is determined that there is no body area.

Further, the present invention is an image processing program for detecting a low-speed moving body region indicating an image region of a low-speed moving body from a target image among captured images in which a predetermined region is sequentially captured. A determination area including a background subtraction area extracted by background subtraction processing between the background image obtained by photographing the background of the target image and the target image is obtained, and each of the above-mentioned photographs taken within the difference period including the shooting time of the target image. With the difference processing means for obtaining a plurality of inter-frame subtraction regions by inter-frame subtraction processing between images, and the area of the inter-frame subtraction region between the target image and the other captured image in the determination region. When the area increase rate from a certain first area to the second area, which is the area of the inter-frame difference area in the accumulated difference image in which the plurality of inter-frame difference areas are accumulated, is equal to or more than the increase rate threshold, the determination area is set to the determination area. It is an image processing program characterized by functioning as a low-speed moving body determination means for determining that there is a low-speed moving body region.

According to the present invention, a low-speed moving body can be detected from a captured image by using the result of inter-frame difference processing.

It is a block diagram of the security system which concerns on this embodiment. It is a block diagram of the image sensor which concerns on this embodiment. It is a figure which shows the target image (a), the background image (b), and the background subtraction image (c). It is a figure which shows a plurality of photographed images, and the difference image between a plurality of frames. It is a figure which shows the difference image (a) between target frames, and the accumulation difference image (b). It is a figure which shows a plurality of photographed images, and a plurality of frame-to-frame difference images when the speed of a moving body is high. It is a figure which shows the difference image (a) between target frames, and the accumulation difference image (b) when the speed of a moving body is fast. It is a figure which shows the inter-frame difference image (a) and background subtraction image (b) when the speed of a moving body is fast. It is a figure which shows the background subtraction image (a), the accumulation difference image (b), and the updated determination area. It is a flowchart which shows the flow of the low-speed moving body detection processing which concerns on this embodiment.

Hereinafter, embodiments of the present invention will be described.

FIG. 1 is a schematic configuration diagram of a security system 10 according to the present embodiment. The security system 10 is a security center connected to each security device 14 via a security device 14 installed in each monitored property 12 such as a store, office, condominium, warehouse, or house, and a communication network 16 such as a public telephone line. The device 18 and the user device 20 are included. Further, the security system 10 includes an image sensor 22 as one or more image processing devices for detecting an abnormality in the monitored property 12 based on a monitored image obtained by capturing the monitored area of the monitored property 12, and an image sensor 22. It is configured to include a recording device 24 for recording a surveillance image taken by the user. The image sensor 22 and the recording device 24 are communicably connected to the security device 14.

When the security device 14 receives an alarm signal from the image sensor 22 connected to itself via the premises LAN or the like, the security device 14 detects the alarm signal, the identification signal of the security device 14 itself, the monitored property 12, or an abnormality. The identification signal of the image sensor 22 is transmitted to the security center device 18. Therefore, the security device 14 has a communication interface for communicating with the image sensor 22, a communication interface for communicating with the security center device 18 and the user device 20, and a control unit for controlling them.

The security center device 18 is composed of a so-called computer, and includes a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a notification unit composed of a buzzer, an LED, or the like. .. When the security center device 18 receives an alarm signal from the security device 14 via the communication network 16, the security center device 18 notifies the monitored property 12 in which the security device 14 that has transmitted the alarm signal is installed, and the content of the detected abnormality. Notify the observer through the display device.

The user device 20 is also composed of a so-called computer, and remotely operates the security device 14 such as a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a keyboard and a mouse. It has a user interface for inputting operation commands for. When the user device 20 observes the pre-registered monitored property 12 via the user interface, the user device 20 currently photographs the security device 14 installed in the registered monitored property 12. Various image request signals requesting that the surveillance image inside or the surveillance image recorded in the recording device 24 be transmitted to the user device 20 are transmitted. Then, when the surveillance image is received from the security device 14, the user device 20 displays the requested surveillance image on the display device.

The recording device 24 is a recording medium that can be attached to and detached from the recording device 24, such as a magnetic disk device such as an HDD, a magnetic tape such as a DAT, and an optical recording medium such as a DVD-RAM, and data by accessing the recording medium. It consists of a device that reads and writes. The recording device 24 receives the surveillance image taken by the image sensor 22 from the security device 14 and records it in association with the shooting time.

FIG. 2 is a schematic configuration diagram of the image sensor 22. The image sensor 22 captures a predetermined region (monitoring region) to acquire a captured image, and then detects a moving body region indicating a moving body from the captured image. In the present embodiment, the image sensor 22 detects a moving body region indicating a person (intruder) among moving bodies from the captured image. In particular, the image sensor 22 according to the present embodiment can detect a low-speed moving body region indicating a low-speed moving body whose moving speed is extremely slow. The details of the image sensor 22 will be described below.

The communication unit 30 is an input / output interface for transmitting and receiving various setting signals and control signals between the image sensor 22 and the security device 14 via a communication network such as a premises LAN, and various types such as Ethernet (registered trademark). It is composed of the communication interface circuit of the above and the driver software that drives them. Specifically, when a person is detected by the signal processing unit 42 described later, the communication unit 30 outputs an alarm signal indicating that the person has been detected to the security device 14.

The photographing unit 32 includes a two-dimensional detector composed of a photoelectric converter having sensitivity to visible light such as a CCD, and an imaging optical system that forms an image of a monitoring area on the two-dimensional detector. It is composed. As the photographing unit 32, for example, a unidirectional camera or an omnidirectional camera can be used. The photographing unit 32 sequentially acquires captured images by sequentially photographing the monitoring area. In the present embodiment, the photographing unit 32 shoots at regular time intervals (for example, 1/5 second), but the photographing method of the photographing unit 32 is not limited to this. The acquired captured image is stored in the storage unit 34.

The storage unit 34 is composed of a semiconductor memory, a magnetic disk (HDD), an optical disk drive such as a CD-ROM or a DVD-RAM, and a recording medium thereof. The storage unit 34 stores an image processing program for operating each part of the image sensor 22. Further, as shown in FIG. 2, the storage unit 34 stores a plurality of captured images 36, background images 38, and reference information 40 sequentially acquired by the photographing unit 32.

The background image 38 is an image obtained by photographing the background of the monitoring area, and is created or updated by the background image updating means 54 described later based on the captured image 36 when no person exists in the monitoring area. The details of the update process of the background image 38 will be described later.

The reference information 40 is information representing the characteristics of the detection target (person in the present embodiment), and in the present embodiment, the reference height (for example, 180 cm) indicating the standard height of the person and the standard width of the person are indicated. It contains a reference width (eg, 60 cm) and information indicating a reference movement speed indicating the standard movement speed of the person.

The signal processing unit 42 has a built-in microprocessor unit, memories such as ROM and RAM, and peripheral circuits thereof, and executes various signal processing of the image sensor 22. As shown in FIG. 2, the signal processing unit 42 serves as a difference processing means 44, a low-speed moving body determining means 46, a label shaping means 48, a tracking means 50, an intruder determining means 52, and a background image updating means 54. Demonstrate function. When the signal processing unit 42 exerts these means, the moving body region, which is an image region showing a person (intruder), is detected in the captured image 36. In particular, in the present embodiment, a low-speed moving body region indicating a person as a low-speed moving body having an extremely low moving speed is detected. Hereinafter, each means included in the signal processing unit 42 will be described.

The difference processing means 44 compares the captured image 36 with the background image 38 stored in advance in the storage unit 34, extracts pixels that differ between the two images, and sets the extracted pixels to a first value (for example,). 1), a background subtraction process for obtaining a background subtraction image, which is a binary image in which the pixels not extracted is set to a second value (for example, 0), is performed. Here, the "pixels different between the two images" mean the pixels of the captured image 36 in which the difference in luminance value or color component between the captured image 36 and the background image 38 is equal to or greater than a predetermined value. The difference processing means 44 performs background subtraction processing between each of the plurality of captured images 36 that are sequentially acquired and the background image 38.

FIG. 3 shows examples of the target image 36a (a), the background image 38 (b), and the background subtraction image 60 (c), which are the captured images 36 for detecting the low-speed moving body region. By the way, the target image 36a does not necessarily have to be the latest captured image 36 among the plurality of captured images 36 stored in the storage unit 34. When a person is included in the target image 36a as in the example of FIG. 3A, it is composed of a pixel group corresponding to the person as shown in FIG. 3C by background subtraction processing. The background subtraction region 62 is extracted. The pixels included in the background subtraction region 62 are called background subtraction extraction pixels.

The difference processing means 44 obtains a determination area 66 including the extracted background difference area 62 in the background difference image 60. As will be described later, the determination area 66 is an area for determining the presence or absence of the low-speed moving body area. In the present embodiment, the subtraction processing means 44 uses the circumscribing rectangle of the background subtraction region 62 as the determination region 66. In the present embodiment, the determination area 66 is the circumscribing rectangle of the background subtraction area 62, but the present invention is not limited to this, and there is a margin as an upper limit range in consideration of the range in which the low-speed moving body to be detected can move within the difference period. You may let me. It is preferable that the margin is set in advance according to the characteristics of the low-speed moving body to be detected. The setting of such a determination area is intended to eliminate the influence of the high-speed moving body. As the information indicating the determination area 66, for example, the coordinates of the opposite vertices of the determination area 66 are held.

Further, the difference processing means 44 performs labeling processing for assigning an identifier (label) to each background subtraction extraction pixel. As the label, for example, a unique numerical value in the background subtraction image 60 can be used. In the labeling process, when attention is paid to a certain background subtraction pixel, a group of pixels composed of other background subtraction extraction pixels adjacent to the pixel is regarded as one background subtraction region 62. The pixels adjacent to the background subtraction extraction pixels may be pixels adjacent to the background subtraction extraction pixels in the vertical and horizontal directions, or may include pixels adjacent to the background subtraction extraction pixels in the oblique direction. Then, different labels are given to each background subtraction region 62, and processing is performed to store the background subtraction image 60 as a pixel value. Hereinafter, the background subtraction area 62 to which the label is attached may be referred to as a label area 64.

Further, the difference processing means 44 compares two captured images 36 captured at different time points, extracts pixels that differ between the two images, and sets the extracted pixels as the first value (for example, 1). Interframe difference processing is performed to obtain an interframe difference image which is a binary image in which the extracted pixels are set to a second value (for example, 0). Here, the difference processing means 44 performs inter-frame difference processing between each photographed image 36 photographed within the difference period including the image-taking time of the target image 36a, and obtains a plurality of inter-frame difference images. The difference period will be described in detail later. In the present embodiment, the difference processing means 44 performs inter-frame difference processing between each of the captured images 36 captured in the past difference period from the capture time of the target image 36a, but the captured image to be the target of the inter-frame difference processing. The 36 may include a captured image 36 captured in the future than the target image 36a.

FIG. 4 shows a plurality of captured images 36 and a plurality of inter-frame difference images 68-1 to 68-4 (hereinafter, may be comprehensively referred to as inter-frame difference images 68). In the example of FIG. 4, 5 frames from the target image 36a are captured images 36 captured within the difference period. In the example of FIG. 4, a plurality of captured images 36 include a person whose moving speed is extremely slow, and the pixels corresponding to the person included in each captured image 36 and the person in the captured image 36 one frame before the pixel are included. Some of the corresponding pixels overlap.

The difference processing means 44 sets the inter-frame difference image 68-1 between the target image 36a and the captured image 36-1 one frame before the target image 36a, the captured image 36-1 and the captured image 36-2 one frame before the target image 36a. Inter-frame difference image 68-2, inter-frame difference image 68-3 between captured image 36-2 and captured image 36-3 one frame before, and captured image 36-3 and one frame before The inter-frame difference image 68-4 between the captured image 36-4 and the captured image 36-4 is obtained.

As described above, since the pixels corresponding to the person included in each captured image 36 and the pixels corresponding to the person in the captured image 36 one frame before the pixel partially overlap, the difference image 68 between frames is used. Is a pixel corresponding to a person in one captured image 36 but not a pixel corresponding to a person in the other captured image 36, and a pixel not corresponding to a person in one captured image 36 but the other. Pixels that are pixels corresponding to a person in the captured image 36 are extracted. The pixels extracted in this way are called inter-frame difference extraction pixels, and the set of inter-frame difference extraction pixels is called an inter-frame difference region 70.

Further, in the present embodiment, among the plurality of inter-frame difference images 68, the inter-frame difference image 68 between the target image 36a and the other captured image 36 is referred to as the inter-frame difference image 68a.

The low-speed moving body determination means 46 uses a plurality of inter-frame difference images 68 to perform a low-speed moving body detection process for detecting a low-speed moving body from the target image 36a.

The low-speed moving body determination means 46 applies the determination area 66 obtained in the background subtraction image 60 within the target frame subtraction image 68a. FIG. 5A shows a determination region 66 applied to the inter-frame difference image 68a. On top of that, the low-speed moving body determination means 46, in the difference image 68a between the target frame, determining a first area S ₁ is the area of the inter-frame difference area 70 in the determination area 66. The first area S ₁ is, in other words, the number of inter-frame difference extraction pixels in the determination region 66 in the inter-frame difference image 68a.

Further, the low-speed moving body determining means 46 obtains the accumulated difference image in which the inter-frame difference region 70 in the plurality of inter-frame difference images 68 obtained by the inter-frame difference processing by the difference processing means 44 is accumulated (added). .. In the example of FIG. 4, the accumulated difference image is obtained by adding the inter-frame difference areas 70 of the target inter-frame difference image 68a and the inter-frame difference images 68-2 to 68-4. FIG. 5B shows the accumulated difference image 80 thus obtained. The area obtained by adding the inter-frame difference areas 70 of the plurality of inter-frame difference images 68 included in the accumulated difference image 80 is referred to as an inter-frame difference area 72.

On top of that, the low-speed moving body decision means 46 applies a determination region 66 in the accumulation subtraction image 80, in the accumulation subtraction image 80, second determination area is the area of the storage frame difference area 72 in the region 66 S ₂ Ask for.

When a person who has invaded the monitoring area is moving at a low speed, the positions of the pixels corresponding to the person in each captured image 36 are slightly different, so that the position of the inter-frame difference area 70 in each inter-frame difference image 68 is also slightly different. different. Therefore, when the inter-frame difference regions 70 in each inter-frame difference image 68 are added together, the area of the accumulated inter-frame difference region 72 gradually increases. Therefore, looking for the determination area 66, in comparison with the first area S ₁ is the area of the frame difference area 70 in the difference image 68a between the target frame, the area of the storage frame difference area 72 in the storage difference image 80 A certain second area S ₂ is larger.

Here, consider the case where the moving speed of the person who has entered the monitoring area is high, in other words, the case where the person is not a low-speed moving body. Similar to FIG. 4, FIG. 6 shows a plurality of captured images 36 and a plurality of inter-frame difference images 68. In the example of FIG. 6, a plurality of captured images 36 include a person having an extremely high moving speed, and the pixels corresponding to the person included in each captured image 36 and the person in the captured image 36 one frame before the pixel are included. Corresponding pixels do not overlap.

In this case, in each frame-to-frame difference image 68, both all the pixels corresponding to the person in one captured image 36 and all the pixels corresponding to the person in the other captured image 36 are extracted as the inter-frame difference region 70. Will be done.

In the example of FIG. 6, the inter-frame difference image 68a to which the determination area 66 is applied is shown in FIG. 7A, and the accumulated difference image 80 to which the determination area 66 is applied is shown in FIG. 7B. There is. When the moving speed of the person who has entered the monitoring area is high, the positions of the pixels corresponding to the person in each captured image 36 do not overlap. Therefore, as shown in FIG. 7B, when the inter-frame difference regions 70 in each inter-frame difference image 68 are added together, the accumulated inter-frame difference region 72 includes a pixel region corresponding to a person in each captured image 36. They just line up without duplication. In this case, looking at the the determination area 66, the first area S ₁ is the area of the frame difference area 70 in the difference image 68a between the target frame is the area of the storage frame difference area 72 in the storage difference image 80 The two areas S ₂ are equivalent.

Because are as above, low-speed moving body determination means 46, the first area S ₁ obtains the area increase ratio [Delta] S [%] of the second area S _2, obtained area increase rate [Delta] S is a predetermined increase rate threshold In the above case, it is determined that the low-speed moving body area exists in the determination area 66. In other words, the low-speed moving body determination means 46, when the area rate of increase from the first area S ₁ to the second area S ₂ [Delta] S [%] is less than the predetermined increase rate threshold, slow in the determination area 66 It is determined that there is no moving body area. Here, the area increase rate ΔS is calculated by the following formula.
_{ΔS = ((S 2 -S 1} ) / S 1) × 100
The rate of increase ΔS may be calculated using S ₃ described later instead of S ₁ as the denominator of the above equation.

When the speed of the person who has entered the monitoring area is considerably low, the area of the inter-frame difference area 70 in each inter-frame difference image 68 becomes considerably small. If the difference period is constant, as the area of the frame difference area 70 in the difference image 68 between the frames is small, the area of the storage frame difference area 72 in the storage difference image 80 (i.e., the second area S ₂₎ is reduced .. This acts in a direction that makes it difficult for the low-speed moving body to be determined to be in the determination area 66. Here, the longer the difference period, because the number of inter-frame difference image 68 to be storage processing is increased, it is possible to further increase the second area S _2.

Therefore, in the present embodiment, when the area of the inter-frame difference area 70 (that is, the first area S ₁ ) in the determination area 66 in the target inter-frame difference image 68a is calculated by the low-speed moving body determination means 46, the difference processing is performed. means 44, as the ratio of the first area S ₁ occupied in the area of the determination region 66 of the target inter-frame difference image 68a is small, may be set to a large value difference period. Simply do not change the difference period in accordance with the size of the first area S ₁ is because considering the size of the pixel area corresponding to the person in the captured image 36. Then, the difference processing means 44 obtains the inter-frame difference image 68 between the captured images 36 taken within the set difference period, and the low-speed moving body determining means 46 obtains the inter-frame difference image 68 between the newly obtained plurality of captured images. The above-mentioned low-speed moving body determination process is performed using the difference image 68.

Further, as described above, when the moving speed of the person who has entered the monitoring area is high, the pixels corresponding to the person included in each captured image 36 and the pixels corresponding to the person in the captured image 36 one frame before the pixel are Since there is no overlap, as shown again in FIG. 8A (see also FIG. 6), the inter-frame difference area 70 in the target frame-to-frame difference image 68a includes a pixel area corresponding to a person in the target image 36a and a captured image 36. Only the pixel areas corresponding to the person in -1 are lined up without overlapping. At this time, the area of the frame difference area 70 in the determination area 66 in the difference image 68a between the target frame (i.e., the first area S _1), the background of the determination area 66 in the background difference image 60 shown in FIG. 8 (b) the third area S ₃ is the area of the differential area 62 becomes equal.

Thus, low-speed moving body decision means 46 obtains the third area S ₃ is the area of the background difference region 62 within the determination region 66 in the background differential image 60, the difference between the first area S ₁ and the third area S ₃ When is less than or equal to a predetermined area difference threshold value, it may be determined that the moving speed of the person who has entered the monitoring area is high, that is, there is no low-speed moving body area in the determination area 66.

Returning to FIG. 2, the label shaping means 48 executes a label shaping process for shaping each label area 64 extracted by the background subtraction process of the difference processing means 44. For example, a label integration process for integrating a plurality of label areas 64 based on a predetermined condition such as whether or not a plurality of label areas 64 within a predetermined range become a human-like size or shape area when integrated. To execute. According to the label integration process, for example, when the upper body and the lower body of the same person are extracted as different label areas 64, the two label areas 64 are integrated to correspond to the image of one person. It can be the label area 64.

Further, for example, the label shaping means 48 may perform a label separation process of cutting out a human area from the label area 64 and using it as another label area 64. The label separation process includes a process of cutting out a human area from one label area 64 including a human area and a shadow area.

Further, for example, the label shaping means 48 may perform a label deletion process for deleting the label area 64. The label deletion process is a process of deleting a label area 64 that can be clearly determined not to be a person. For example, the label area 64, which is too large or too small for the size of the person, is deleted.

Further, in a plurality of captured images 36 included in the difference period, uniform noise may be superimposed due to, for example, lens flare. In this case, as shown in FIG. 9A, the background subtraction region 62 included in the background subtraction image 60 includes pixels corresponding to the noise, and the area of the background subtraction region 62 becomes large. The outer shape may also become unhuman.

In such a case, the label shaping means 48 is in the determination area 66 in the accumulated difference image 80 where it is determined that there is a low-speed moving body area prior to the above-mentioned label integration process, label separation process, and label deletion process. The background subtraction region 62 may be shaped based on the storage frame-to-frame difference region 72 (see FIG. 9B). In the present embodiment, the label shaping means 48 is extracted as the background subtraction area 62 in the original determination area 66 (see FIG. 9A) of the background subtraction image 60, and is applied to the accumulated difference image 80. The pixel area extracted as the accumulation frame subtraction area 72 in the determination area 66 (see FIG. 9B) is referred to as the updated background subtraction area 62'. Further, the determination area 66 may also be updated in accordance with the background subtraction area 62'after the update. Specifically, the circumscribing rectangle of the updated background subtraction region 62'may be used as the updated determination region 66'. The post-update determination area 66'can have an area smaller than the determination area 66, and thus an image area that can be used as the background image 38 in the background image update process by the background image update means 54 described later. Can be taken larger.

The tracking means 50 performs a process of temporally tracking the label area 64 extracted from each of the plurality of captured images 36 sequentially acquired by the photographing unit 32 and shaped by the label shaping means 48. Specifically, the tracking means 50 is extracted from each label area 64 extracted from the captured image 36 acquired this time and shaped by the label shaping means 48, and stored in the captured image 36 acquired in the past. A process of identifying the label area 64 (hereinafter referred to as “tracking label area”) stored as tracking information in the unit 34 (a process of determining that the label area 64 corresponds to the same person) is performed. Here, the tracking information is information related to the tracking ID, the position on the captured image 36, and the tracking feature amount with respect to the tracking label area. The tracking feature amount is the size and shape of the tracking label area in the real space, the brightness histogram, the texture information, and the like.

The tracking means 50 compares the tracking feature amount and position of the label area 64 of the captured image 36 acquired this time with the tracking feature amount and position of the tracking label area, and thereby, the captured image 36 acquired this time. The label area 64 and the tracking label area are identified. For example, identification is performed by those having similar tracking features and close positions. Then, the same tracking ID as the identified tracking label area is assigned to the label area 64 of the captured image 36 acquired this time, and the tracking of the label area 64 of the captured image 36 acquired this time is tracked in chronological order. After associating the ID with the position on the captured image 36 and the like, the ID is stored in the storage unit 34 as tracking information.

In the label area 64 of the captured image 36 acquired this time, those that could not be identified as the tracking label area are considered to have appeared as a new person, and are new to the tracking feature amount of the label area 64. Give a tracking ID. In addition, if there is a tracking label area that has been tracked so far that cannot be identified as the label area 64 of the captured image 36 acquired this time, it is assumed that the person corresponding to the tracking label area has disappeared. , Delete the tracking information.

Based on the processing results of each of the above-mentioned means, the intruder determining means 52 determines whether or not each of the label areas 64 included in the target image 36a corresponds to a person (intruder) who is a moving body. judge.

The intruder determination means 52 determines whether or not the label area 64 determined to be a moving body is human. For example, the estimated size (estimated height or estimated width) of the label area 64 is obtained, and it is determined whether or not the label area 64 is human by comparing it with the standard size of the person included in the reference information 40.

Further, the intruder determining means 52 obtains the estimated speed of the label area 64 based on the movement amount of the label area 64 from the result of the tracking process of the tracking means 50, and the standard moving speed of the person included in the reference information 40. It may be determined whether or not the label area 64 is human-like in consideration of the comparison result with. At this time, when the label area 64 is determined by the low-speed moving body determination means 46 to be the low-speed moving body area, it is determined whether or not the label area 64 is human-like in consideration of the determination result. It may be. For example, even if the movement amount of the label area 64 is slower than the standard movement speed, if it is determined to be a low-speed moving body region, it may be determined to be human-like (moving at a low speed). Alternatively, the value set as the standard moving speed may be changed to a smaller value for the label area 64 determined to be the low-speed moving body area.

When the label area 64 is determined to be human by the above processing, the intruder determination means 52 determines that the target image 36a includes a person (intruder), and causes the communication unit 30 to output an alarm signal. .. Here, further, the intruder determination means 52 communicates when it is determined that the label area 64 to which the same tracking ID is assigned is continuously determined to be human-like by a predetermined number of frames (for example, 5 frames) or more. The alarm signal may be output from the unit 30.

The background image updating means 54 creates or updates the background image 38 using the captured image 36. The background image updating means 54 may use the captured image 36 when there is no moving body in the monitoring area as the background image 38 as it is, but the background image 38 may be used as the background image 38 by using a portion of the captured image 36 other than the moving body region. It can also be partially updated.

The background image updating means 54 in the present embodiment updates the background image 38 based on the results of the inter-frame difference processing by the difference processing means 44 and the low-speed moving body determination processing by the low-speed moving body determination means 46. Specifically, first, the background image updating means 54 determines an image area that is not extracted as the inter-frame difference area 70 for a predetermined stability determination period as a stable area by the inter-frame difference processing by the difference processing means 44. On top of that, the image area outside the determination area 66 determined to have the low-speed moving body area by the low-speed moving body determination process by the low-speed moving body determination means 46, and the image area determined to be the stable area. Is used to update the background image 38. In other words, the background image updating means 54 does not use the determination area 66 determined to have the low-speed moving body region for updating the background image 38.

The background image 38 is preferably updated at regular intervals (for example, every 10 minutes) in order to cope with fluctuations in the monitoring region such as fluctuations in the lighting state and daily fluctuations in the sun.

The configuration of the security system 10 according to the present embodiment is as described above. Hereinafter, the flow of the low-speed moving body determination process (including the process by the partial difference processing means 44) will be described with reference to the flowchart shown in FIG.

In step S10, the difference processing means 44 compares the captured image 36 with the background image 38 and performs background subtraction processing for obtaining the background difference image 60.

In step S12, the difference processing means 44 sets the circumscribing rectangle of the background subtraction region 62 in the determination region 66 in the background subtraction image 60 obtained in step S10.

In step S14, the difference processing means 44 performs inter-frame difference processing between the target image 36a and the other captured image 36 to obtain the inter-frame difference image 68a.

In step S16, the low-speed moving object determination unit 46 calculates the third area _{S 3} is the area of the background difference region 62 in the determination area 66 determined in step S12.

In step S18, the low-speed moving body determining means 46 applies the determination area 66 obtained in step S12 to the inter-frame difference image 68a obtained in step S14. Furthermore, the slow moving object determination means 46, in the difference image 68a between the target frame, determining a first area S ₁ is the area of the inter-frame difference area 70 in the determination area 66.

In step S20, the low-speed moving object determination means 46, or the third area _{S 3} obtained in the first area _{S 1} and step S16 obtained in step S18 is equal, i.e., the first area _{S 1} and the third It is determined whether or not the difference from the area S ₃ is equal to or less than a predetermined area difference threshold. If the difference between the first area S ₁ and the third area S ₃ is less than or equal to the predetermined area difference threshold, the process proceeds to step S22, in step S22, the low-speed moving object determination means 46, the low-speed movement in the determination area 66 It is determined that there is no body area. If the difference between the first area S ₁ and the third area S ₃ is not less than a predetermined area difference threshold, the process proceeds to step S24.

In step S24, the difference processing means 44 sets a difference period including the shooting time of the target image 36a. Specifically, the difference processing unit 44, calculated in step S18, as the ratio of the first area S ₁ occupied in the area of the determination region 66 of the target inter-frame difference image 68a is small, set to a large value difference period ..

In step S26, the difference processing means 44 performs inter-frame difference processing between each of the captured images 36 captured within the difference period set in step S24, and obtains a plurality of inter-frame difference images 68.

In step S28, the low-speed moving body determining means 46 generates an accumulated difference image 80 in which the inter-frame difference regions 70 in the plurality of inter-frame difference images 68 obtained in step S26 are accumulated.

In step S30, the low-speed moving body determination means 46 applies the determination area 66 obtained in step S12 to the accumulated difference image 80 generated in step S28, and the difference between the accumulated frames in the determination area 66 in the accumulated difference image 80. determining a second area _{S 2} is the area of the region 72.

In step S32, the low-speed moving object determination unit 46 calculates an area increase ratio from the first area _{S 1} to the second area _{S 2 ΔS} [%]. As described above, the area increase rate ΔS is
_{ΔS = ((S 2 -S 1} ) / S 1) × 100
Is required by.

In step S34, the low-speed moving body determining means 46 determines whether or not the area increase rate ΔS obtained in step S32 is equal to or greater than a predetermined increase rate threshold value. When the area increase rate ΔS is equal to or greater than the increase rate threshold value, the process proceeds to step S36, and in step S36, the low-speed moving body determining means 46 determines that the low-speed moving body area exists in the determination area 66. On the other hand, when the area increase rate ΔS is less than the increase rate threshold value, the process proceeds to step S22, and in step S22, the low-speed moving body determining means 46 determines that there is no low-speed moving body area in the determination area 66.

Although the embodiments according to the present invention have been described above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the present invention.

10 security system, 12 monitored property, 14 security device, 16 communication network, 18 security center device, 20 user device, 22 image sensor, 24 recording device, 30 communication unit, 32 imaging unit, 34 storage unit, 36 captured image , 36a Target image, 38 background image, 40 reference information, 42 signal processing unit, 44 subtraction processing means, 46 low-speed moving object determination means, 48 label shaping means, 50 tracking means, 52 intruder determination means, 54 background image updating means. , 60 background subtraction image, 62 background subtraction area, 62'updated background subtraction area, 64 label area, 66 judgment area, 66' after update judgment area, 68 frame subtraction image, 68a target frame subtraction image, 70 frame interval Difference area, 72 accumulation frame difference area, 80 accumulation difference image.

Claims (6)

An image processing device that detects a low-speed moving body area indicating an image area of a low-speed moving body from a target image among captured images obtained by sequentially shooting a predetermined area.
A determination area including the background subtraction area extracted by background subtraction processing between the background image obtained by photographing the background of the predetermined area and the target image was obtained, and the image was taken within the difference period including the shooting time of the target image. A difference processing means for obtaining a plurality of inter-frame subtraction regions by inter-frame subtraction processing between the captured images.
The frame in the accumulated difference image in which the plurality of inter-frame difference areas are accumulated from the first area which is the area of the inter-frame difference area between the target image and the other captured image in the determination area. When the area increase rate to the second area, which is the area of the difference region, is equal to or greater than the increase rate threshold, the low-speed moving body determining means for determining that the low-speed moving body region exists in the determination region, and
An image processing apparatus comprising. A label shaping means for shaping the background subtraction region based on the inter-frame subtraction region in the determination region in the accumulated difference image determined to have the low-speed moving body region.
The image processing apparatus according to claim 1, further comprising. An image region outside the determination region determined to have the low-speed moving body region by determining the image region not extracted as the inter-frame difference region during the stability determination period by the inter-frame difference processing as a stable region. The image processing apparatus according to claim 1 or 2, further comprising a background image updating means for updating the background image using the image region determined to be the stable region. The difference processing means sets the difference period to a larger value as the ratio of the first area to the area of the determination area is smaller.
The image processing apparatus according to any one of claims 1 to 3, wherein the image processing apparatus is characterized. A third area indicating the area of the background subtraction region in the determination region is obtained, and when the difference between the first area and the third area is equal to or less than the area difference threshold value, the low-speed moving body region is included in the determination region. Judge no,
The image processing apparatus according to any one of claims 1 to 4, wherein the image processing apparatus is characterized. An image processing program for detecting a low-speed moving body area indicating an image area of a low-speed moving body from a target image among captured images obtained by sequentially shooting a predetermined area.
Computer,
A determination area including a background subtraction area extracted by background subtraction processing between the background image obtained by photographing the background of the predetermined area and the target image was obtained, and the image was taken within the difference period including the shooting time of the target image. A difference processing means for obtaining a plurality of inter-frame subtraction regions by inter-frame subtraction processing between the captured images, and
The frame in the accumulated difference image in which the plurality of inter-frame difference areas are accumulated from the first area which is the area of the inter-frame difference area between the target image and the other captured image in the determination area. When the area increase rate to the second area, which is the area of the difference region, is equal to or greater than the increase rate threshold, the low-speed moving body determining means for determining that the low-speed moving body region exists in the determination region, and
An image processing program characterized by functioning as.

Images