WO2012042705A1

WO2012042705A1 - Method of detecting moving object and moving object detecting device

Info

Publication number: WO2012042705A1
Application number: PCT/JP2011/003321
Authority: WO
Inventors: 恵酒井; 隆一宮腰; 北村　臣二; 邦博今村
Original assignee: パナソニック株式会社
Priority date: 2010-09-27
Filing date: 2011-06-10
Publication date: 2012-04-05

Abstract

The present invention detects a moving object using a normalized feature amount unsusceptible to a change in the brightness of a display screen. The normalized feature amount is obtained by dividing an image into block units, calculating the feature amount for each block and normalizing the feature amount with the average of the brightness and so forth. By comparing the normalized feature amount of the blocks of an input image with the normalized feature amount calculated from the same position in the background, a moving object can be detected without being affected by a change in the brightness of a display screen.

Description

Moving object detection method and moving object detection apparatus

The present invention relates to a moving object detection method and a moving object detection apparatus for detecting a moving object such as a person or a vehicle based on an input moving image.

In a surveillance system using a camera, a technique for automatically detecting a moving object in a screen is known. In general, a method has been used in which a difference image of luminance values existing at the same coordinates is generated in an input image and a background image, and a moving object is present in a portion having a large difference value. In this method, the brightness fluctuates due to the flicker of fluorescent lamps, the lighting device turning on / off, sunlight, clouds, etc., so the luminance values at the same coordinates all change with the lighting fluctuations, and the parts that are not moving are also moving objects Detected.

One of the solutions for erroneous detection of moving objects at the time of illumination fluctuation is disclosed in Patent Document 1. In the method described in Patent Document 1, when N is a natural number, an image is divided into N × N pixel block units, orthogonal transformation is performed, and among the AC components, the horizontal direction absolute value sum and the vertical direction absolute value sum The moving object is detected using the vertical / horizontal edge ratio based on. As one of orthogonal transforms, a discrete cosine transform (DCT) is used.

FIG. 17 shows an example of the DCT coefficient. Assuming that the DCT coefficient obtained by performing DCT on the two-dimensional signal f (i, j) is F (k, l), F (0,0) indicated by 1701 is a DC component, and the others The coefficient F (k, l) (k> 0, l> 0) is an AC component. k is a horizontal frequency, l is a vertical frequency, and the higher the k or l, the higher the frequency.

A method for calculating the vertical / horizontal edge ratio will be described. Among the AC components, the horizontal direction absolute value sum f (h) is the sum of the absolute values of the coefficients existing in 1702, and the vertical direction absolute value sum f (v) is the sum of the absolute values of the coefficients existing in 1703. is there. The aspect ratio BR is, for example,
BR = f (v) / f (h)
It is represented by

According to Patent Document 1, when there is no edge in a divided block, the AC components are all 0, and no false detection is made as a moving object even while illumination variation is occurring. Even if an edge is present in the divided block, the AC component changes, but by detecting the vertical / horizontal edge ratio, erroneous detection of a moving object is prevented.

FIG. 18 (a) and FIG. 18 (b) show examples of luminance of illumination variation in a 4 × 4 pixel block unit. In FIG. 18A, the left half has a luminance value of 200 and the right half has a luminance value of 100. In FIG. 18B, it is assumed that the illumination fluctuation occurs in the same block, the brightness in the block is halved, the left half has the luminance value 100, and the right half has the luminance value 50.

FIG. 19 (a) and FIG. 19 (b) show the DCT coefficients for FIG. 18 (a) and FIG. 18 (b), respectively. The arrangement of the coefficients in FIGS. 19A and 19B corresponds to FIG.

When calculating the aspect ratios BR (a) and BR (b) for FIG. 19A and FIG. 19B, respectively.
BR (a) = 0 / (| 184.8 | + | -76.5 |) = 0
BR (b) = 0 / (| 92.3 | + | −38.3 |) = 0
Thus, the moving object detection method is not affected by illumination fluctuations.

JP 2002-259985 A

The apparatus described in Patent Document 1 can cope with illumination fluctuations, but cannot detect when the vertical / horizontal edge ratio does not change, for example, when the edge moves in the horizontal direction.

20A and 20B show an example in which the edge in the 4 × 4 pixel block unit moves in the horizontal direction. In FIG. 20A, the left column has a luminance value of 100 and the remaining one has a luminance value of 50. In FIG. 20B, an object having a luminance value of 100 has moved to the right by two columns, the left column has a luminance value of 100, and the right column has a luminance value of 50. This phenomenon occurs when a sufficiently large moving body moves horizontally with respect to the block.

FIG. 21 (a) and FIG. 21 (b) show the DCT coefficients for FIG. 20 (a) and FIG. 20 (b), respectively. The coefficient arrangement in FIGS. 21A and 21B corresponds to FIG.

When the vertical / horizontal edge ratios BR (c) and BR (d) are calculated for FIGS. 21 (a) and 21 (b), respectively.
BR (c) = 0 / (| 65.3 | + | 50 | + | 27.1 |) = 0
BR (d) = 0 / (| 65.3 | + | −50 | + | 27.1 |) = 0
It becomes. In other words, when the sum of absolute values of either horizontal or vertical is 0, it cannot be detected regardless of the moving object. Further, when the horizontal / vertical edge ratio does not change, for example, when the luminance is inverted, the change cannot be detected even though the pattern is changed.

This invention solves the said subject, and aims at providing the moving body detection method and moving body detection apparatus which can detect a moving body with high precision, without being influenced by illumination fluctuation.

In order to achieve the above object, a moving object detection method according to an aspect of the present invention includes a feature extraction step of extracting a feature amount in block units from an input image, and a normalized feature from the feature amount extracted from the feature extraction step. A normalizing step for calculating the quantity, and a moving object detecting step for determining whether the object is a moving object according to the normalized feature value normalized in the normalizing step.

The moving object detection device according to an aspect of the present invention includes a feature extraction unit that extracts a feature amount in block units from an input image, and a normalization that calculates a normalized feature amount from the feature amount extracted by the feature extraction unit. And a moving object detection unit that determines whether the object is a moving object according to the normalized feature value normalized by the normalization unit.

As described above, the present invention can provide a moving object detection method and a moving object detection apparatus that can detect the presence or absence of a luminance pattern change even when illumination changes and can detect a moving object with high accuracy even when illumination changes occur.

It is a block diagram which shows the structure of the moving body detection apparatus which concerns on Embodiment 1 of this invention. It is an illustration figure of the orthogonal transformation coefficient which concerns on Embodiment 1 of this invention. It is an example of the positional relationship between the block of the input feature-value and background feature-value which concern on Embodiment 1 of this invention. It is a moving body determination method flowchart of the moving body detection part which concerns on Embodiment 1 of this invention. It is a flowchart of the moving body detection method which concerns on Embodiment 1 of this invention. It is a moving body determination method flowchart of the moving body detection part which concerns on Embodiment 2 of this invention. It is a block diagram which shows the structure of the moving body detection apparatus which concerns on Embodiment 3 of this invention. It is a block diagram which shows the structure of the moving body detection apparatus which concerns on Embodiment 4 of this invention. It is a figure which shows the edge pattern which concerns on Embodiment 4 of this invention. It is a flowchart of the pattern extraction method from the normalized feature-value which concerns on Embodiment 4 of this invention. It is a flowchart of the moving body detection method which concerns on Embodiment 4 of this invention. It is an illustration figure which divides | segments the block of a 5x5 pixel unit based on Embodiment 4 of this invention into a detailed block. It is a block diagram which shows the structure of the imaging system which concerns on Embodiment 5 of this invention. It is a figure which shows the example of a display of a moving body detection result. (A) And (b) is a figure which shows the example of a contraction process of the moving body detection result output from the moving body detection apparatus. (A) And (b) is a figure which shows the example of an expansion process of the moving body detection result output from the moving body detection apparatus. It is an illustration figure of the DCT coefficient in the conventional moving body detection apparatus. (A) And (b) is a figure which shows the luminance example of the illumination fluctuation | variation in a 4x4 pixel block unit. (A) And (b) is a figure which shows the DCT coefficient with respect to Fig.18 (a) and FIG.18 (b). (A) And (b) is a figure which shows the example of a brightness | luminance when the edge in a 4x4 pixel block unit moves to a horizontal direction. (A) And (b) is a figure which shows the DCT coefficient with respect to Fig.20 (a) and FIG.20 (b).

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and various modifications can be made.

Embodiment 1
First, the configuration of the moving object detection device 100 according to Embodiment 1 of the present invention will be described. FIG. 1 is a block diagram showing a configuration of a moving object detection apparatus 100 according to Embodiment 1 of the present invention. The moving object detection apparatus 100 according to the present invention includes a block dividing unit 101, an orthogonal transform unit 102, a feature extracting unit 103, a feature accumulating unit 104, a background updating unit 105, and a moving object detecting unit 106.

The block dividing unit 101 divides the input image 150 into predetermined block units, and outputs a predetermined block unit image 151. Here, the input image 150 is a luminance value. The input image 150 may be a color difference, or may be an R value, a G value, or a B value. The predetermined block unit is performed in N × N pixel units. N is a natural number. Smaller objects can be detected as N is smaller, but are more susceptible to noise. Conversely, as N increases, it is less susceptible to noise, but it is difficult to detect objects that are smaller than the block size. In the following description, it is assumed that processing is performed in blocks of 8 × 8 pixels.

In addition, the block division may be divided into blocks by predetermined units, or the blocks may overlap each other.

The orthogonal transform unit 102 performs orthogonal transform on the block unit image 151 divided by the block dividing unit 101 and calculates a transform coefficient 152. The orthogonal transform includes Hadamard transform, DFT (discrete Fourier transform), DCT, wavelet transform, and the like. In particular, the Hadamard transform can calculate a coefficient by the simplest addition / subtraction process, and can realize high-speed processing.

When an orthogonal transformation is performed on a block of 8 × 8 pixel units, an 8 × 8 transformation coefficient 152 can be obtained. FIG. 2 shows an example of the conversion coefficient 152. The coefficient is expressed as F (k, l) (0 ≦ k ≦ 7, 0 ≦ l ≦ 7), k represents a frequency in the horizontal direction, and l represents a frequency in the vertical direction. The higher k and l, the higher the frequency.

The feature extraction unit 103 extracts a necessary coefficient from the transform coefficient 152 obtained from the orthogonal transform unit 102 and outputs it as an input feature quantity 153. The input feature quantity 153 is two feature quantities described below. One is a DC component for normalization. The DC component is 201 in FIG. 2 and is used for normalization at a later stage. Second, the AC component sum (ACsum) is used as the edge feature amount in the block. The AC component sum (ACsum) is a value obtained by integrating the coefficients indicated by 202 in FIG. 2, that is, the sum of the conversion coefficients excluding the DC component.

Here, the sum of the AC components is shown as the edge feature amount. However, when it is desired to detect only the horizontal edge (the pixel value changes when viewed in the horizontal direction), for example, the horizontal noise in an interlaced image is used. When this occurs, only the horizontal component of the AC component (for example, 1702 in FIG. 17) may be used. When it is desired to detect only the vertical edge (the pixel value changes when viewed in the vertical direction), for example, when smear occurs in the image and vertical line noise occurs, only the vertical component of the AC component is detected. (For example, 1703 in FIG. 17) may be used. In an image with a lot of random noise, for example, an image in which the sensor output is raised in a dark place, if the reliability of the high frequency component is low, only a part of the AC component on the low frequency side (eg, 203 in FIG. 2) is used. May be. If the processing amount is not limited, the AC component may be an (N × N−1) -dimensional feature amount.

In addition to the DC component, the average value, median value, maximum value, and minimum value of the input image in the block may be used as values for normalizing the block.

Although normalization is performed in units of blocks, the above values may be calculated from the entire screen to normalize the screen.

The feature storage unit 104 stores an input feature amount 153 and a background feature amount 154. The background feature quantity 154 is a feature quantity generated from an image other than the input image. A past image may be used in time series, or a future image may be obtained by prefetching the image. In this embodiment, when the next input image is processed after the input feature value 153 is used for moving object detection, it can be said that the input feature value 153 is a feature value of the past image, and this is used as the background feature value 154. . Therefore, the background feature quantity 154 is composed of the DC component and the AC component sum as in the case of the input feature quantity 153. Further, if the input feature quantity 153 extracted from the input image for one image is counted as one frame, the number of frames of the background feature quantity 154 is F frames. F is a natural number, and the smaller the F, the smaller the comparison object, so the difference between the background and the moving object is less likely to occur. However, the processing amount is smaller, and the more F, the greater the comparison object. It is easy, but the processing amount becomes large. It is assumed that the background feature quantity 154 is calculated in advance for F frames from the monitoring area image during the initial operation of the moving object detection apparatus 100.

If the background feature quantity 154 does not exist in the initial state, the F-frame feature quantity may be calculated and accumulated by operating the block dividing unit 101 to the feature extraction unit 103 in advance.

In addition, although the example which has the characteristic storage part 104 in the inside of the moving body detection apparatus 100 was shown, you may utilize the external memory of an apparatus.

The background update unit 105 updates the background feature value 154 from the input feature value 153 stored in the feature storage unit 104.

Specific processing of the background update unit 105 will be described. There is a method of overwriting and updating with the input feature value 153 at a certain frame interval. When a new input feature quantity 153 is input to the background update unit 105, the frame that has the longest passage of time in the background feature quantity 154, that is, the frame that is stored the oldest in time is overwritten.

The background update may be performed every time one frame of the input image 150 is input, or the monitoring target moving speed, frame rate, background feature amount F frame, such as once every two frames or once every three frames. The update frequency may be changed according to machine specifications such as the amount of memory to be stored.

It should be noted that the background feature quantity 154 may be calculated in advance from a background image that does not have a monitoring target, and an object that has been left without updating or an object that has been removed may be detected as a moving object.

The moving object detection unit 106 detects a moving object using the input feature value 153 and the background feature value 154 accumulated in the feature accumulation unit 104, and outputs a moving object detection result 155.

When the background feature quantity 154 has F frames, the frame number of the background feature quantity 154 is set to f. Let Cnt be the number of moving objects detected. Of the input feature quantity 153, the sum of the AC components of the block to be processed (hereinafter referred to as a processing block) is ACsum (in), and the DC component is DC (in). For the f-th block at the same position as the processing block in the background feature quantity 154, the AC component sum is ACsum (f), the DC component is DC (f), and 0 ≦ f <F.

FIG. 3 shows the positional relationship between the input feature quantity 153 and the background feature quantity 154 between blocks. 301 is an input feature amount, and 302 is a background feature amount for F frames. In 303, the AC component total ACsum (in) and the DC component DC (in) in the input feature amount are stored. 304, 305, and 306 store the AC component sum and DC component of the block at the same position as 303 in the background feature amount 302. 304 to 306 in the background feature quantity 302 are compared with 303 in the input feature quantity 301, respectively, and it is detected whether a moving object exists in the block by majority vote. The comparison uses the normalized feature value Pat. The normalized feature value Pat is a feature value normalized by dividing the AC component sum by the DC component,
Pat = ACsum / DC
It is.

The moving object determination method of the moving object detection unit 106 will be described. FIG. 4 is a flowchart of a moving object determination method of the moving object detection unit 106 according to the first embodiment of the present invention.

In S401, the frame number f of the background feature quantity is set to 0, and the moving object detection number Cnt is set to 0.

In S402, ACsum (in) of the input feature value is divided by DC (in) to calculate a normalized feature value Pat (in).

In S403, the AC feature (f) of the f-th frame of the background feature value is divided by DC (f) to calculate the normalized feature value Pat (f).

In S404, it is determined whether or not the difference absolute value between Pat (in) and Pat (f) is greater than the normalization threshold TH1.

In S405, when it is determined in S404 that the difference absolute value between Pat (in) and Pat (f) is larger than the normalization threshold TH1, the moving object detection number Cnt is increased by one.

In S406, the frame number f is incremented by 1 in the case where it is determined in S404 that the absolute difference between Pat (in) and Pat (f) is not larger than the normalization threshold TH1, and in the case following S405.

In S407, it is determined whether the frame number f has been compared for all the background feature amount F frames. If it is determined in S407 that all the background feature F frames have not been compared, the process returns to S403.

In S408, when it is determined that the frame number f has been compared for all the background feature amount F frames, it is determined whether the moving object detection number Cnt is larger than the moving object detection threshold MoveTH.

In S409, if it is determined in S408 that the moving object detection number Cnt is larger than the moving object detection threshold MoveTH, a detection result 155 that there is a moving object is output.

In S410, when it is determined that the moving object detection number Cnt is not larger than the moving object detection threshold MoveTH, the detection result 155 that there is no moving object is output.

The normalization threshold value TH1 is a threshold value for determining how large the difference between the input feature value and the background feature value is. The moving object detection threshold value MoveTH is a threshold value that determines how many frames the background feature amount has changed in F frames to be a moving object.

Both TH1 and MoveTH are sensitive to changes if they are small, and insensitive to changes if they are large. It is desirable to change according to the monitoring target.

<Process flow>
The moving body detection method by the moving body detection apparatus 100 configured as described above will be described with reference to FIG. FIG. 5 is a flowchart of the moving object detection method according to Embodiment 1 of the present invention.

First, the input image 150 is divided into 8 × 8 pixel block unit images 151 by the block dividing unit 101 (S501).

The orthogonal transform unit 102 performs orthogonal transform on the block unit image 151 divided by the block dividing unit 101, and calculates a transform coefficient 152 (S502).

The feature extraction unit 103 extracts the input feature quantity 153 from the transform coefficient 152 calculated by the orthogonal transform unit 102, and stores it in the feature storage unit 104 (S503).

The moving object detection unit 106 calculates a normalized feature value from the input feature value 153 and the background feature value 154 stored in the feature storage unit 104 (S504), and performs moving object detection using the normalized feature value (S505). .

Next, the background update unit 105 determines whether or not to update the feature storage unit 104 with the input feature value 153 as the background feature value 154 (S506).

The moving object detection effect by the normalized feature amount according to the first embodiment will be described using DCT.

As described above, the DCT coefficients when the illumination variation occurs in the blocks of FIGS. 18A and 18B are as shown in FIGS. 19A and 19B. If the normalized feature amounts calculated from a) and FIG. 19B are Pat (Ca) and Pat (Cb),
Pat (Ca) = (184.8 + (− 76.5)) / 600≈0.18
Pat (Cb) = (92.3 + (− 38.3)) / 300≈0.18
Thus, there is no change amount even when the illumination fluctuates, and the feature quantity is strong against the illumination fluctuation.

As described above, the DCT coefficients in FIGS. 20 (a) and 20 (b) when the object moves horizontally in the block are as shown in FIGS. 21 (a) and 21 (b). If the normalized feature values calculated from FIGS. 21A and 21B are Pat (Ea) and Pat (Eb), respectively.
Pat (Ea) = (65.3 + 50 + 27.1) /250≈0.57
Pat (Eb) = (65.3 + (− 50) +27.1) /350≈0.12
Thus, since the amount of change can be calculated even when moving in the horizontal direction, it can be detected as a moving object. Similarly, even when an object moves in the block in the vertical direction, it can be detected as a moving object.

In addition, although the block division unit 101, the orthogonal transformation unit 102, and the like are processed inside the moving object detection device 100, they may be processed by an external device.

Note that the feature extraction unit 103 may perform the feature extraction step S503 and the normalization step S504. That is, the normalization feature value is calculated by the moving object detection unit 106, but the feature extraction unit 103 calculates the normalization feature value in advance and outputs it as the normalized input feature value 153. The normalized input feature quantity 153 is accumulated. Accordingly, the background feature amount updated by the background update unit 105 becomes the normalized background feature amount 154, and steps S402 and S403 that are normalized by the moving object detection unit 106 are omitted.

Of course, as an alternative to the block dividing unit 101 and the orthogonal transform unit 102, it is possible to use an encoding device or a decoding device that performs orthogonal transform in units of blocks and encodes transform coefficients.

<< Embodiment 2 >>
Next, the moving object detection apparatus according to the second embodiment will be described. The moving object detection device according to the present embodiment has a configuration substantially similar to that of the moving object detection device 100 according to the first embodiment, but the moving object detection method of the moving object detection unit 106 is different. Hereinafter, the description will be given focusing on the difference.

The moving object detection unit 106 according to the second embodiment can select whether or not to calculate the normalized feature amount. A moving object determination method of the moving object detection unit 106 according to Embodiment 2 will be described with reference to FIG. FIG. 6 is a flowchart of a moving object determination method of the moving object detection unit 106 according to the second embodiment.

In S601, the frame number f of the background feature quantity is set to 0, and the moving object detection number Cnt is set to 0.

In S602, ACsum (in) of the input feature value is divided by DC (in) to calculate a normalized feature value Pat (in).

In S603, it is determined whether or not the difference absolute value between DC (in) and DC (f) is larger than the DC threshold value DCTH.

In S604, if it is determined in S603 that the absolute difference between DC (in) and DC (f) is greater than the DC threshold DCTH, the ACsum (f) of the fth frame of the background feature is divided by DC (f). Then, the normalized feature value Pat (f) is calculated.

In S605, following S604, it is determined whether or not the difference absolute value between Pat (in) and Pat (f) is larger than the normalization threshold TH1.

In S606, when it is determined in S605 that the absolute difference between Pat (in) and Pat (f) is larger than the normalization threshold TH1, the moving object detection number Cnt is increased by one.

In S607, if it is determined in S603 that the absolute difference between DC (in) and DC (f) is not greater than the DC threshold DCTH, the absolute difference between ACsum (in) and ACsum (f) is greater than the threshold TH2. Determine if it is larger.

In S608, when it is determined in S607 that the absolute difference between ACsum (in) and ACsum (f) is greater than the threshold value TH2, the moving object detection number Cnt is increased by one.

In S609, when it is determined in S605 that the absolute difference between Pat (in) and Pat (f) is not larger than the normalization threshold TH1, the continuation of S606, and ACsum (in) and ACsum (f) in S607. The frame number f is incremented by 1 when it is determined that the difference absolute value between is not greater than the threshold value TH2 and after S608.

In S610, it is determined whether the frame number f has been compared for all the background feature amount F frames. If it is determined in S610 that all background feature F frames have not been compared, the process returns to S603.

In S611, when it is determined that the frame number f has been compared for all the background feature amount F frames, it is determined whether or not the moving object detection number Cnt is greater than the moving object detection threshold MoveTH.

In S612, if it is determined in S611 that the moving object detection number Cnt is larger than the moving object detection threshold MoveTH, a detection result 155 that there is a moving object is output.

In S613, when it is determined that the moving object detection number Cnt is not larger than the moving object detection threshold value MoveTH, the detection result 155 that there is no moving object is output.

The necessity for the determination in S603 will be described. When there is almost no DC component difference absolute value, since the brightness has not changed, there is no need to use a normalized feature amount that is not affected by the brightness. Since the normalized feature amount is divided, the amount of processing is large. Therefore, using the DC threshold value DCTH, when the difference absolute value of the DC component is larger than the DC threshold value DCTH, the normalized feature value is calculated, and when not larger than the DC threshold value DCTH, the processing is reduced by using ACsum. .

By using the above method, the division processing can be reduced and the calculation processing can be speeded up by using the AC component summation in the portion that does not need to be normalized.

<< Embodiment 3 >>
The present invention can be realized without the feature accumulation unit 104 and the background update unit 105 in the first and second embodiments. FIG. 7 shows a moving object detection apparatus 700 according to Embodiment 3 of the present invention. The description of the parts described in the first and second embodiments is omitted.

The moving object detection apparatus 700 includes a first block division unit 711, a first orthogonal transformation unit 712, a first feature extraction unit 713, a second block division unit 721, a second orthogonal transformation unit 722, and a second feature extraction. Unit 723 and a moving object detection unit 704.

The first block dividing unit 711 divides the input image 750 into predetermined block unit images 751.

The first orthogonal transform unit 712 performs orthogonal transform on the block unit image 751 divided by the first block dividing unit 711 and calculates a transform coefficient 752.

The first feature extraction unit 713 calculates an input feature amount 753 from the conversion coefficient 752. Here, as in the feature extraction unit 103 of the first embodiment, the AC component sum and the DC component are output as the input feature value 753.

The second block dividing unit 721 divides the background image 760 into predetermined block unit images 761. The background image 760 may be a temporally previous frame of the input image 750 or a subsequent frame.

The second orthogonal transform unit 722 performs orthogonal transform on the block unit image 761 divided by the second block dividing unit 721 to calculate a transform coefficient 762.

The second feature extraction unit 723 outputs the AC component sum and the DC component as the background feature amount 763 from the conversion coefficient 762.

The moving object detection unit 704 receives the input feature value 753 and the background feature value 763 and outputs a moving object detection result 770. Since the process of the moving object detection unit 704 is the same as that of the moving object detection unit 106 of the first and second embodiments, the description thereof is omitted.

In this embodiment, image input for two frames is taken as an example. However, the second block dividing unit 721, the second orthogonal transform unit 722, and the second feature extracting unit 723 are increased by the third and fourth. By doing so, it is possible to compare in a plurality of frames, as in the first and second embodiments.

<< Embodiment 4 >>
A configuration of a moving object detection apparatus 800 according to Embodiment 4 of the present invention will be described. FIG. 8 is a block diagram showing a configuration of a moving object detection device 800 according to Embodiment 4 of the present invention. The moving object detection apparatus 800 according to the present invention includes a first block dividing unit 801, a first normalized feature value calculating unit 802, a first pattern determining unit 803, a feature accumulating unit 804, a background updating unit 805, A two-block dividing unit 807, a second normalized feature value calculating unit 808, a second pattern determining unit 809, and a moving object detecting unit 806 are provided.

The first block dividing unit 801 divides the input image 850 into predetermined block units and outputs a predetermined first block unit image 851. Here, description will be made assuming that processing is performed in blocks of 3 × 3 pixels.

The first normalized feature value calculation unit 802 outputs a first normalized feature value 852 for the first block unit image 851 divided by the first block dividing unit 801. The first normalized feature value 852 is obtained by applying an edge filter to the first block unit image 851 to calculate an edge feature value and normalizing it. A horizontal filter value h and a vertical filter value v are used as edge feature amounts by the edge filter. A Sobel filter is used as the type of edge filter, but the edge filter includes a Canny filter and the like, and is not limited thereto. If the central pixel of a 3 × 3 pixel unit block is a coordinate f (i, j), the horizontal filter value h and the vertical filter value v using the Sobel filter are
h = (− 1) × f (i−1, j−1) + 0 × f (i, j−1) + 1 × f (i + 1, j−1) + (− 2) × f (i−1, j ) + 0 × f (i, j) + 2 × f (i + 1, j) + (− 1) × f (i−1, j + 1) + 0 × f (i, j + 1) + 1 × f (i + 1, j + 1)
v = (− 1) × f (i−1, j−1) + (− 2) × f (i, j−1) + (− 1) × f (i + 1, j−1) + 0 × f (i −1, j) + 0 × f (i, j) + 0 × f (i + 1, j) + 1 × f (i−1, j + 1) + 2 × f (i, j + 1) + 1 × f (i + 1, j + 1)
It is represented by

The normalized horizontal filter value H and the normalized vertical filter value V are obtained by normalizing the horizontal filter value h and the vertical filter value v with the luminance average in the block, and these are used as the first normalized feature value 852. Output. As described in the first embodiment, the value to be normalized is not limited to the luminance average.

The first pattern determination unit 803 outputs an edge pattern as the input feature value 853 from the first normalized feature value 852 obtained from the first normalized feature value calculation unit 802. There are five types of edge patterns, and any one pattern is output.

FIG. 9 shows an edge pattern according to Embodiment 4 of the present invention. Reference numeral 901 denotes a horizontal edge 1, which is a horizontal edge whose luminance value in the left half is lower than that in the right half. Reference numeral 902 denotes a horizontal edge 2, which is a horizontal edge in which the left half luminance value is higher than the right half luminance value. Reference numeral 903 denotes a vertical edge 1, which is a vertical edge whose upper half luminance value is lower than the lower half luminance value. Reference numeral 904 denotes a vertical edge 2, which is a vertical edge in which the upper half luminance value is higher than the lower half luminance value. Reference numeral 905 denotes no edge.

The pattern extraction method is described below. FIG. 10 is a flowchart of a pattern extraction method from normalized feature values according to Embodiment 4 of the present invention.

In S1001, it is determined whether the absolute value of the normalized horizontal filter value H is greater than the horizontal threshold value H_TH or whether the absolute value of the normalized vertical filter value V is greater than the vertical threshold value V_TH.

In S1002, if it is determined in S1001 that the absolute value of the normalized horizontal filter value H is greater than the horizontal threshold value H_TH or the absolute value of the normalized vertical filter value V is greater than the vertical threshold value V_TH, the normalized horizontal filter value It is determined whether the absolute value of H is greater than the absolute value of the normalized vertical filter value V.

In S1003, when it is determined in S1002 that the absolute value of the normalized horizontal filter value H is greater than the absolute value of the normalized vertical filter value V, it is determined whether the normalized horizontal filter value H is greater than 0.

In S1004, when it is determined in S1003 that the normalized horizontal filter value H is greater than 0, the edge pattern is set to horizontal edge 1 (901).

In S1005, when it is determined in S1003 that the normalized horizontal filter value H is not greater than 0, the edge pattern is set to horizontal edge 2 (902).

In S1006, if it is determined in S1002 that the absolute value of the normalized horizontal filter value H is not greater than the absolute value of the normalized vertical filter value V, it is determined whether the normalized vertical filter value V is greater than 0.

In S1007, when it is determined in S1006 that the normalized vertical filter value V is greater than 0, the edge pattern is set to vertical edge 1 (903).

In S1008, when it is determined in S1006 that the normalized vertical filter value V is not larger than 0, the edge pattern is set to the vertical edge 2 (904).

In S1009, if it is determined in S1001 that the absolute value of the normalized horizontal filter value H is not larger than the horizontal threshold value H_TH and the absolute value of the normalized vertical filter value V is not larger than the vertical threshold value V_TH, the edge pattern is No edge (905).

The background update unit 805 generates a new background image 854 from the input image 850 and the background image 854 and outputs it. When the input image is f (t) and the background image is g (t), the background image g (t + 1) generated using the background update rate α (0 ≦ α ≦ 1) is
g (t + 1) = αf (t) + (1−α) g (t)
Is generated. In an initial state where the background image 854 is not stored, α is set to 1, and the input image 850 is output as it is as the background image 854. When α = 0, it can be left behind and can have a removal detection function.

The feature storage unit 804 holds the background image 854 output from the background update unit 805. The background image 854 for at least one frame can be held.

The second block dividing unit 807 divides the background image 854 into predetermined block units, and outputs a predetermined second block unit image 855.

The second normalized feature value calculation unit 808 outputs the second normalized feature value 856 to the second block unit image 855 divided by the second block dividing unit 807. Since the internal processing is the same as that of the first normalized feature value calculation unit 802, description thereof is omitted.

The second pattern determination unit 809 outputs an edge pattern as the background feature value 857 from the normalized feature value 856 obtained from the second normalized feature value calculation unit 808. Since the edge pattern determination process is the same as that of the first pattern determination unit 803, description thereof is omitted.

The moving object detection unit 806 outputs the moving object detection result 858 by comparing the input feature value 853 output from the first pattern determination unit 803 with the background feature value 857 output from the second pattern determination unit 809.

は Check the pattern match to see if the moving objects between the blocks match. Since there are only five edge patterns 901 to 905, the edge pattern is assumed to be a moving object unless it is the same pattern. When F = 1, the detection result 858 is output as a moving object when the edge pattern stored in the background feature value 857 and the edge pattern stored in the input feature value 853 do not match.

When F> 1, if more than half of the F frames do not match, it is determined to be a moving object, and a detection result 858 is output.

<Process flow>
A moving object detection method by the moving object detection apparatus 800 configured as described above will be described with reference to FIG. FIG. 11 is a flowchart of the moving object detection method according to the fourth embodiment of the present invention.

First, the input image 850 is divided by the first block dividing unit 801 into the first block unit image 851 in units of 3 × 3 pixels, and the background image 854 is divided by the second block dividing unit 807 in units of 3 × 3 pixels. The image is divided into second block unit images 855 (S1101).

The first normalized feature value calculating unit 802 extracts edge features by an edge filter from the first block unit image 851 divided by the first block dividing unit 801 (S1102), and the first normalized feature value 852. (S1103), the second normalized feature value calculation unit 808 extracts edge features from the second block unit image 855 divided by the second block division unit 807 (S1102), and the second normalization feature amount calculation unit 808 extracts the second feature. The normalized feature amount 856 is calculated (S1103).

The first pattern determination unit 803 outputs the input feature value 853 from the first normalized feature value 852 calculated by the first normalized feature value calculation unit 802, and the second pattern determination unit 809 outputs the second normalization feature value 805. A background feature value 857 is output from the second normalized feature value 856 calculated by the feature value calculation unit 808 (S1104).

The moving object detection unit 806 detects whether or not the object is a moving object by seeing a match between the input feature value 853 and the background feature value 857 (S1105).

Next, the background update unit 805 updates the new background image 854 using the input image 850 (S1106).

Note that the normalization step S1103 is processed by the first normalized feature value calculation unit 802 and the second normalized feature value calculation unit 808, but may be processed by the first and second

pattern determination units

803 and 809. Good.

Moreover, although the block division unit has been described in units of 3 × 3 pixels, this is divided into 5 × 5 pixel units, and 5 × 5 pixels are divided into detailed blocks. That is, the feature amount to be used is increased from one dimension to multiple dimensions. May be.

FIG. 12 is an exemplary diagram of dividing a block of 5 × 5 pixel units into detailed blocks. The block of 5 × 5 pixel units is further divided into 3 × 3 pixel units, and edge patterns are calculated by the above-described pattern determination method using edge filters at 1201, 1202, 1203, 1204, and 1205, respectively. Moving object detection may be performed considering a feature quantity of a dimension. When there are a plurality of feature quantity dimensions, a moving object is selected if the number of matching patterns exceeds a majority. In the case of five dimensions, for example, three patterns out of five patterns need only match.

<< Embodiment 5 >>
In the fifth embodiment of the present invention, an imaging system including the moving object detection device according to the first to fourth embodiments will be described.

FIG. 13 is a block diagram showing a configuration of an imaging system 1300 according to Embodiment 5 of the present invention. The imaging system 1300 is, for example, a digital still camera, a network camera, a surveillance camera, or the like.

An imaging system 1300 illustrated in FIG. 13 includes an optical system 1301, a sensor 1302, an A / D conversion circuit 1303, an image processing circuit 1304, a recording transfer unit 1305, a reproduction unit 1306, a timing control circuit 1307, a system And a control circuit 1308.

The optical system 1301 focuses the incident image light on the sensor 1302.

The sensor 1302 generates an electrical signal (image signal) by photoelectrically converting the image light imaged by the optical system 1301.

The A / D conversion circuit 1303 converts the electrical signal (analog signal) generated by the sensor 1302 into a digital signal.

The image processing circuit 1304 includes the moving object detection apparatus 100 according to the first embodiment described above. The image processing circuit 1304 performs Y / C processing, edge processing, image enlargement / reduction processing, image compression / decompression processing such as JPEG and MPEG, and image compression on the digital signal converted by the A / D conversion circuit 1303. perform the control and the like of the stream. The moving object detection apparatus 100 detects a moving object based on the digital signal converted by the A / D conversion circuit 1303.

The recording / transferring unit 1305 records the signal processed by the image processing circuit 1304 and the result detected by the moving object detection apparatus 100 on a recording medium or transmits it via the Internet or the like.

The reproduction unit 1306 reproduces the signal recorded or transferred by the recording / transfer unit 1305. Note that the moving object detection result may be displayed over the reproduced image. FIG. 14 is a display example of a moving object detection result. A person 1400 is assumed to be a moving object. Only a block in which a moving object exists can be displayed with a thick frame.

Timing control circuit 1307 controls sensor 1302 and image processing circuit 1304.

The system control circuit 1308 controls the optical system 1301, the recording / transferring unit 1305, the reproducing unit 1306, and the timing control circuit 1307.

Further, in the system control circuit 1308, the captured image is not stable during the AF (Automatic Focus) operation of the optical system 1301 or the AE (Automatic Exposure) operation of the sensor 1302, so the moving object detection device 100 is stopped. Good.

Note that when the image output from the sensor 1302 is saturated by strong light, or when the output image of the sensor 1302 is blacked out in a place with no light, the image pattern changes, so the operation of the moving object detection device 100 is stopped. May be.

In addition, although the moving body detection apparatus 100 which concerns on this invention demonstrated the example used for the camera apparatus etc. which photoelectrically convert the image light from the optical system 1301 by the sensor 1302, and input into the A / D conversion circuit 1303 here, It goes without saying that the moving object detection apparatus 100 according to the present invention may be used in other devices. For example, analog video input from an AV device such as a television may be input directly to the A / D conversion circuit 1303.

The result output from the moving object detection apparatus 100 may be further judged by performing image processing for contraction or expansion.

The shrinking image processing will be described. The block of interest is determined to be the background when N blocks or more of the 8 blocks around the block of interest are not determined to be moving objects. FIG. 15A shows an example of a moving object detection result output from the moving object detection device 100. The shaded portion is a block detected as a moving object by the moving object detection apparatus 100, and the white block is a block determined not to be a moving object. FIG. 15B shows the result of contraction processing with N = 6. In the block surrounded by the thick line, since 6 or more blocks out of the surrounding 8 blocks are not moving objects, they are not displayed as false detections. Through the above processing, false detection that occurs once due to the influence of noise is reduced.

The expansion image processing will be described. A block of interest is determined to be a moving object when N or more blocks are determined to be moving objects among the 8 blocks around the block of interest. FIG. 16A shows an example of a moving object detection result output from the moving object detection device 100. The shaded portion is a block detected as a moving object by the moving object detection device 100, and the white block is a block determined not to be a moving object. FIG. 16B shows the result of the expansion process with N = 6. Since the blocks surrounded by the bold lines are moving bodies in 6 blocks or more out of the surrounding 8 blocks, they are added to the display as moving bodies. Through the above processing, the hollowing out due to the undetected moving object is prevented.

By using the above-described contraction / expansion processing, it is possible to determine again whether the object is moving or background based on the determination result of the surrounding blocks. Furthermore, false detection and non-detection can be reduced by alternately repeating contraction and expansion.

Note that the moving object detection device 100 in FIG. 13 can be replaced with the moving

object detection devices

100, 700, and 800 according to the second to fourth embodiments. These moving

object detection devices

100, 700, and 800 are typically realized as LSIs that are integrated circuits. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

Here, LSI is used, but it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Also, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

Furthermore, if integrated circuit technology that replaces LSI emerges as a result of advances in semiconductor technology or other derived technology, it is naturally also possible to integrate functional blocks using this technology. There is a possibility of adaptation of biotechnology.

The present invention can be applied to a moving object detection method and a moving object detection device, and is particularly useful as a moving object detection method and a moving object detection device for a surveillance camera that detects an intruder.

DESCRIPTION OF SYMBOLS 100 Moving object detection apparatus 101 Block division part 102 Orthogonal transformation part 103 Feature extraction part 104 Feature accumulation part 105 Background update part 106 Moving body detection part 150 Input image 151 Block unit image 152 Conversion coefficient 153 Input feature-value 154 Background feature-value 155 Motion detection result

Claims

A feature extraction step of extracting feature quantities from the input image in units of blocks;
A normalization step of calculating a normalized feature amount from the feature amount extracted by the feature extraction step;
A moving object detection method comprising: a moving object detection step of determining whether or not the object is a moving object in accordance with the normalized feature value normalized in the normalization step.
The moving object detection method according to claim 1,
The moving object detection method, wherein the feature amount is a transform coefficient by orthogonal transform.
The moving object detection method according to claim 2,
The orthogonal detection is a Hadamard transform, a cosine transform, a wavelet transform, or a Fourier transform.
In the moving body detection method of Claim 3,
The moving object detection method, wherein the feature amount is a horizontal component and a vertical component of the AC component of the conversion coefficient.
In the moving body detection method of Claim 3,
The moving object detection method, wherein the feature amount is a sum of AC components of the conversion coefficients.
In the moving body detection method of Claim 3,
The moving object detection method, wherein the feature amount is a part of an AC component of the conversion coefficient.
In the moving body detection method of Claim 3,
The moving object detection method, wherein the normalized feature value is obtained by normalizing the feature value with a DC component of the conversion coefficient.
The moving object detection method according to claim 2,
A moving object detection method further comprising a background update step of updating the feature quantity as a background feature quantity.
The moving object detection method according to claim 8,
In the moving object detection step, a moving object is detected by comparing the normalized feature value with a feature value obtained by normalizing the background feature value.
The moving object detection method according to claim 9, wherein
The moving object detection step includes:
Comparing the DC component of the transform coefficient that normalizes the feature quantity with the DC component of the transform coefficient that normalizes the background feature quantity;
A step of determining whether or not the object is a moving object by comparing the unnormalized feature quantity and the unnormalized background feature quantity when the difference between the two DC components is small. Motion detection method.
The moving object detection method according to claim 2,
A moving object detection method further comprising a background update step of updating the normalized feature value as a normalized background feature value.
The moving object detection method according to claim 1,
The moving object detection method, wherein the feature amount is calculated by an edge filter.
The moving object detection method according to claim 12,
The moving object detection method, wherein the edge filter is a Sobel filter.
The moving object detection method according to claim 12,
The moving object detection method, wherein the edge filter is a Canny filter.
The moving object detection method according to claim 12,
The moving object detection method, wherein the feature amount is a multi-dimensional edge feature amount calculated by an edge filter for each detail block by further dividing the block into detail blocks.
The moving object detection method according to claim 12,
Holding a background image;
A background update step of updating the background image from the input image;
And a step of calculating a background feature amount from the background image.
The moving object detection method according to claim 1,
A moving object detection method further comprising a re-determination step of determining again whether the object is a moving object or a background based on a determination result of peripheral blocks.
The moving object detection method according to claim 1,
The moving object detection method, wherein the normalized feature value is obtained by normalizing the feature value by a luminance average of the input image.
The moving object detection method according to claim 1,
The moving object detection method, wherein the normalized feature amount is obtained by normalizing the feature amount with a maximum value of the input image.
The moving object detection method according to claim 1,
The moving body detection method, wherein the normalized special feature amount is obtained by normalizing the feature amount with a minimum value of the input image.
The moving object detection method according to claim 1,
The moving object detection method, wherein the normalized feature amount is obtained by normalizing the feature amount with a median value of the input image.
The moving object detection method according to claim 1,
The moving object detection method, wherein the input image is a luminance value.
The moving object detection method according to claim 1,
The moving object detection method, wherein the input image is a color difference, an R value, a G value, or a B value.
The moving object detection method according to claim 1,
A moving object detection method further comprising a feature storage step of storing at least one frame of a background image.
The moving object detection method according to claim 1,
A moving object detection method further comprising a result display step of displaying the presence or absence of a moving object for each processing block.
The moving object detection method according to claim 1,
A second feature extraction step of extracting feature quantities in block units from the background image;
A second normalization step of calculating a second normalized feature amount from the feature amount extracted in the second feature extraction step;
In the moving object detection step, it is detected whether the object is a moving object by comparing the normalized feature quantity with the second normalized feature quantity.
A feature extraction unit that extracts feature amounts from the input image in units of blocks;
A normalization unit that calculates a normalized feature amount from the feature amount extracted by the feature extraction unit;
A moving object detection apparatus comprising: a moving object detection unit that determines whether the object is a moving object according to the normalized feature value normalized by the normalization unit.
An optical system for imaging light;
A sensor that converts light imaged by the optical system into an image signal;
An image processing circuit including a moving object detection device according to claim 27, wherein a moving object is detected from the image signal in units of blocks.
An imaging system comprising: a system control circuit that controls the optical system, the sensor, and the image processing circuit.
The imaging system according to claim 28, wherein
An imaging system characterized in that the operation of the moving object detection device is stopped during AF or AE control by the system control circuit.
The imaging system according to claim 28, wherein
An imaging system characterized in that the operation of the moving object detection device is stopped when an image signal output from the sensor is saturated by strong light or blackened by weak light.