JP2008060967A - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that it is difficult to detect an unidentified object or the like from a monitoring object together with a periodical fluctuation such as the sea. <P>SOLUTION: Still image data are evenly divided into rectangular regions. A characteristic amount is calculated per division region. The distance is calculated between the characteristic amount per division region and a predetermined value. By comparing the distance obtained with a threshold value, whether it is a background is determined. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to an image signal processing device, an image processing method, and an image processing program.
More specifically, the present invention relates to a technique suitably applied to detect a suspicious object or a suspicious person from a video obtained by imaging a predetermined monitoring place by a monitoring camera.

2. Description of the Related Art Conventionally, a monitoring system that monitors a predetermined monitoring target by visually recognizing an image captured by a monitoring camera via a predetermined monitoring monitor is known.
In such a monitoring camera, it is difficult for a 24-hour monitor to continue to monitor the monitor as the number of places to be monitored increases. In video recorders connected to surveillance cameras, there is an urgent need to shorten the actual operation time, such as recording only the minimum necessary video.
For this reason, establishment of the technique in which a machine detects a suspicious object or a suspicious person from an input image is calculated | required.
For example, in a bank ATM, the above-described requirements can be satisfied relatively easily if there is a sensor for detecting a human body.
However, such a sensor cannot be used when detecting a suspicious ship or the like from a distant landscape such as the seaside.
JP 2006-14215 A JP-A-10-328226

Conventionally, in such a situation, a method of performing comparison with the immediately preceding image data as described in Patent Document 1 has been often employed.
When an object enters the video, the brightness of the object portion in the video data changes. Therefore, an object can be detected by detecting a region having a difference in luminance value in the video as a difference region. However, in the case of natural scenery such as the sea, desert, grassland, and sky, water, sand, grass, clouds, etc. other than the object to be detected are also moving. For this reason, there has been a problem that they are erroneously detected as being different from the previous image data.

One method for solving this problem is, for example, the technique described in Patent Document 2.
In Patent Document 2, a difference value between an image taken at the current time and an image taken in the past is created and binarized by threshold processing. At that time, a background image in which the threshold value is changed is created based on the accumulated result of past difference values. In this way, attempts are made to reduce false detection of fluctuations caused by trees and water existing in the background image.
However, with this technique, when the change in the luminance value due to the fluctuation of the tree or the like is large, the case where the threshold value is too large is considered. In such a case, there is a possibility that a detection omission may occur when an important intruder or the like enters.

  The present invention has been made in view of the above points, and from image data obtained by a monitoring camera that monitors a place where a photographed image fluctuates mainly due to a natural phenomenon or the like, such as a sea, a desert, or a grassland. It is an object of the present invention to provide an image processing apparatus capable of stably entering a suspicious person or a suspicious object.

  In the present invention for solving the above-described problem, after the input image data is temporarily stored, the input image data is divided into equal areas, and a feature amount is calculated for each divided area. The distance between the feature value obtained by the feature value calculation and a predetermined value is calculated, and it is determined whether or not the background is based on the obtained distance. And based on the determination result obtained by the determination, the presence or absence of a suspicious object etc. is output.

The inventor can compare the background and the suspicious object, etc. with respect to the object to be photographed while the natural scenery such as the sea has a certain regularity by comparison with the immediately preceding image data known in the prior art. The method was judged to be inappropriate. Therefore, the inventor sought a technique for discriminating a background from a suspicious object using only input image data. As a result, as a technique for discriminating between the background and the suspicious object, etc., the idea was to divide the image data into predetermined areas, calculate feature values for each area, and calculate the distance from this value. It was.
If the entire image data is the background, even if the image data is divided, the feature amounts of the respective divided areas hardly change. With the image processing method based on this technical idea, even if there is a change caused by a natural phenomenon such as the sea or sky, it can be determined that the background has a certain characteristic including this.

  According to the present invention, particularly in a monitoring device that monitors a landscape with a certain fluctuation, such as the sea or grassland, excellent image processing that can recognize a natural phenomenon such as a sea wave or a cloud in the sky as a background without erroneous recognition. Equipment can be provided.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS.

[1. Outline of Embodiment]
FIG. 1 is an overall block diagram of a monitoring device that is a common concept of the embodiment of the present invention. FIG. 1A is a block diagram corresponding to the contents of an actual device, and FIG. 1B is a virtual block diagram focusing on functions.
In FIG. 1A, the monitoring apparatus 101 detects the presence / absence of a suspicious object from image data obtained from the imaging camera 102, and performs a binary signal (alarm output) to notify the presence / absence of a suspicious object or a predetermined process. The applied image signal is output. The imaging camera 102 is a well-known camera that outputs an image signal, for example, composed of a CCD imaging device or the like. The output image signal is sent to another host device via a large-capacity storage such as an HDD or a communication line.
The monitoring device 101 is mainly composed of a microcomputer. The CPU 103, ROM 104, RAM 105 and bus 106 constituting the microcomputer perform predetermined processing on the image data obtained from the imaging camera 102 and output a predetermined signal or the like through the output interface 107.
The configuration shown in FIG. 1A is a configuration common to all the embodiments described in detail later.

In FIG. 1B, each unit is a software program executed on a microcomputer except for an image storage unit 112, which is a RAM 105, and a predetermined value 115 including a RAM 105 or a ROM 104. These software are stored in the ROM 104.
An image signal obtained from the imaging camera 102 is stored in the image storage unit 112 as input still image data.
The area dividing unit 113 gives a rectangular divided area having an equal area and shape to the input still image data in the image storage unit 112.
The feature amount calculation unit 114 creates feature amount data for each region of the input still image data in the image storage unit 112 divided by the region division unit 113.
The predetermined value 115 provides feature amount data obtained by the feature amount calculation unit 114 or a comparison target of predetermined data based on the feature amount data. Then, the distance calculation unit 116 calculates the distance between the feature value data obtained by the feature value calculation unit 114 or predetermined data based thereon and the predetermined value 115.
The calculation result of the distance calculation unit 116 is output to a storage, a network or the like as a binary alarm or image data.
The configuration in FIG. 1B is a concept common to all the embodiments described in detail later.

Here, the feature amount is a pattern or texture of the image. It is almost synonymous with what is called “texture” in general drawing software.
In the case of grassland, there are many green shades in a parabolic shape in an oblique direction.
If it is a calm sea surface, there are many white parts such as fine patterns parallel to the horizon that reflects sunlight and white parts such as splashes, based on dark blue.
These natural landscapes do not form a mechanically constant pattern, such as artificially created tiles. However, although it has a random variation, it is recognized that a feature like a certain similar pattern is formed. There is no wave splash on the grassland, and there is no diagonal shading on the calm sea surface.
These natural landscapes move with natural phenomena with time. Therefore, simply comparing a plurality of still images with a time difference as in the prior art, it is misrecognized as a moving object, that is, a suspicious object even though it is essentially a background. .
In the present embodiment, a technique for firmly recognizing the background as the background without being confused by the change on the time axis by capturing the “background texture” with respect to the “moving background” is provided.
Capturing a background texture or background pattern means that it can also capture the characteristics of a foreign object present in the background. This is the calculation of the feature amount.
The input image data is divided into equal fine regions, and the feature amount is calculated for each divided region. Of the calculated feature quantities for each divided area, those indicating values deviating from the background feature quantity are determined to be areas where suspicious objects exist. For this purpose, it is calculated how much the feature quantity of the sample is similar to the feature quantity of the target divided region. This is “distance calculation”. Finally, it is determined whether or not the background is obtained by comparing the obtained distance with a predetermined threshold. This is an extension of the technical idea of calculating the luminance difference between single pixels, which is well known in the art.
The above is a common concept in all the embodiments.
How to obtain the feature amount and how to calculate the distance between the feature amounts will be described later.

[2. First Embodiment]
The first embodiment will now be described. First, an outline of the present embodiment will be described. In the present embodiment, sample image data is given in advance, a background portion is designated manually from the sample image data, and the feature amount is held. After that, when the apparatus is in full operation, it is compared with the feature quantity data of the background sample held for each divided area of the input image data.
FIG. 2 is a block diagram of the monitoring apparatus according to the first embodiment of the present invention. This is based on the content of FIG.
Input still image data obtained from the imaging camera 102 is temporarily stored in the image storage unit 112.
The area dividing unit 113 finely divides the input still image data for each divided area.
The feature amount calculation unit 114 calculates the feature amount of each divided image data.
On the other hand, the background feature amount storage unit 202 stores the feature amount of the background area specified in advance by the operation of the operator.
The distance calculation unit 116 calculates the distance between the feature amount for each divided region obtained from the feature amount calculation unit 114 and the background feature amount stored in the background feature amount storage unit 202. As a result of the distance calculation by the distance calculation unit 116, it is determined that a suspicious object exists in the divided area determined not to be the background, and an alarm is output.

A display unit 203 and an input unit 204 are connected to the background feature quantity storage unit 202.
The display unit 203 is a known display device such as an LCD.
The input unit 204 is a known pointing device such as a mouse.
The background feature amount storage unit 202 stores a background feature amount as a sample. Prior to the actual operation of this apparatus, input image data as a sample is displayed on the display unit 203 in advance, and a range for acquiring a background is input thereto by the input unit 204. And the feature-value calculation part 114 calculates a feature-value, and preserve | saves this.

FIGS. 3A, 3B, and 3C are image diagrams illustrating an outline of the operation of the monitoring apparatus according to the first embodiment by exemplifying input image data.
In FIG. 3A, as input image data, there is image data 301 including a sea 302, a sky 303, and a ship 304 that is a suspicious object. This is stored in the image storage unit 112.
In FIG. 3B, the background feature quantity storage unit 202 displays input image data on the display unit 203 by the operation of the operator, and surrounds the background portion with the input unit 204. A range surrounded by a dotted line in FIG. 3A is a background partial region input by the operation of the input unit 204 by the operator. This area is calculated by the feature amount calculation unit 114 and stored in the background feature amount storage unit 202. In FIG. 3A, two background samples B (1) and B (2) are stored in the sky part and the sea part.
In FIG. 3C, the input image data is finely divided by the region dividing unit 113, and the feature amount calculating unit 114 calculates the feature amount for each region. In FIG. 3C, 25 divided areas S (1) to S (25) are provided.
The distance calculation unit 116 calculates the distance between the background sample and the divided area. This calculation is performed one by one. That means
Calculate the distance between the background sample B (1) and the divided area S (1),
Calculate the distance between the background sample B (2) and the divided area S (1),
Calculate the distance between the background sample B (1) and the divided area S (2),
Calculate the distance between the background sample B (2) and the divided area S (2),
In order, the distance between the background sample B (1) and the divided area S (25) is calculated.
The distance between the background sample B (2) and the divided area S (25) is calculated, and the process ends.

These operations will be described in more detail with reference to FIGS.
FIG. 4 is a flowchart illustrating pre-processing prior to actual operation of the monitoring apparatus according to the first embodiment.
When the process is started (S401), sample input still image data is taken into the RAM (S402).
Next, the input still image data is displayed on the display unit 203 (S403).
While confirming this visually, the operator operates the input unit 204 to input the address range of the background area (S404).
When the input is confirmed, the designated address range is stored in a RAM or the like (S405).
Then, the pixel in the designated address range is extracted from the sample input still image data, the feature amount is calculated, stored (S406), and the process ends (S407).

5 and 6 are flowcharts showing an actual operation state of the monitoring apparatus according to the first embodiment.
When the process is started (S501), sample input still image data is taken into the RAM (S502).
Next, the feature amount for each divided region divided by the region dividing unit 113 is calculated by the feature amount calculating unit 114 (S503).
Subsequent processing is processing for sequentially calculating the distance between the calculated feature amount for each region and the background sample.
First, the variable i is initialized to 1 (S504). The variable i is a variable that specifies a feature amount for each region.
Next, the variable j is initialized to 1 (S505). The variable j is a variable that specifies the background sample.
Then, the distance between the background sample B (j) and the divided region feature S (i) is calculated (S506).
Next, the distance obtained as a result of the distance calculation is compared with a threshold value separately held in advance (S507). If the distance is equal to or greater than the threshold, the flag variable f (i) is increased to 1 (S508). The flag variable f (i) is a variable provided in the same number as that of the divided areas. The flag is raised by 1 and the flag is lowered by 0. If the distance is not greater than or equal to the threshold, the flag f (i) is lowered to 0 (S509).

Next, it is verified whether or not the variable j has reached the maximum value (S610). In the case of FIG. 4, since there are two background samples, it is checked whether j has reached 2. If j is the maximum value, the calculation of the distances between all the background samples and the divided region currently being verified is completed, so the variable i is incremented (S611). That is, the process proceeds to the evaluation of the next divided area. If j is not the maximum value, there is a background sample for which distance calculation still remains, so j is incremented (S612).
After the flag is lowered, the variable i is verified (S613). If it is not the maximum value as a result of the verification, i is incremented (S614). This is because the flag is lowered because it is determined that the divided area is the background as a result of the verification by the distance calculation. In the example of FIG. 4, if it is determined that the distance from the sky background sample is empty first, it is not necessary to calculate the distance from the sea background sample next. For this reason, this process is performed in order to skip the distance calculation process with the next background sample.
If the variable i reaches the maximum value, it means that the distance calculation has been completed for all the divided areas. Therefore, all flag variables f (i) are checked to verify whether there is a variable whose flag is raised (S615).
As a result of the verification, if the flag is not raised at all, it can be determined that all the divided areas are the background and there is no suspicious object.
On the other hand, as a result of the verification, if even one of the flags is raised, it can be determined that the divided area is not the background and the suspicious object exists there.
If the presence of a suspicious object is recognized, an alarm is output (S616) and the process is terminated (S617).

In the present embodiment, a background sample is designated manually in advance. For this reason, there is a drawback in that if the background feature value changes, it cannot follow this. For example, in the case of monitoring the sea, if the sea is rough due to the weather, or if the imaging camera 102 moves the shooting angle and the positional relationship between the sky and the sea changes, a misrecognition will occur.
However, if these features are sufficiently grasped and an appropriate installation is performed for an appropriate monitoring target, the present embodiment can sufficiently exhibit its functions. If the background feature value is a monitoring target that does not change much, or if the imaging camera 102 is fixed or operated, or if the background feature value may be changed, the background sample may be input again. A method may be used.

  By the way, how to determine the size of the divided region is slightly different depending on the monitoring target. For example, even if a too small divided area is set for the background forming a complex pattern, it cannot be said that the feature amount can be calculated appropriately. However, when the size of the suspicious object to be detected is small, even if a too large divided region is set, the change is hardly caused in the obtained feature amount. Therefore, it is necessary to change the optimum divided area according to the monitoring target. For this reason, it is advisable to perform a test operation while changing the divided area area on a trial basis in advance to determine the optimum value before actually operating the monitoring device.

Summarize what has been described above.
The monitoring apparatus 201 of this embodiment is mainly composed of software that runs on a microcomputer.
When the input still image data is input to the monitoring apparatus 201, the input still image data is stored in the image storage unit 112 (actually, the RAM 105). After being divided into equal area shapes by the area dividing unit 113, the feature amount calculating unit 114 creates feature amount data for each divided region. Next, the range of the obtained feature data is set in advance by manual input, the distance from the predetermined value 115 held after the arithmetic processing is calculated, and the obtained “distance” is compared with a predetermined threshold value. . A region having a distance less than the threshold is determined to be the background. Then, the flag indicating the divided area determined to be the background is lowered.
Through the above processing, it is possible to discriminate between the background area and the other area areas from the input still image data through the feature amount calculation and the distance calculation.

[2.1. Feature value calculation method]
Here, a method of calculating the feature amount will be described.
The feature value is obtained by reading image data in a certain range and digitizing features appearing in the image data.
Various features of image data can be considered. Wide range of chromaticity, saturation, and pattern. Therefore, it cannot be a single scalar quantity, and is represented by a matrix composed of many elements.
FIGS. 7A, 7B, and 7C are schematic diagrams for explaining a color histogram, which is one of feature amount calculation methods that can be commonly used in all the embodiments described in the present invention.
FIG. 7A shows a sample of input image data. In the input image data 701, the sea 702 extends over the whole, and a suspicious ship 703 exists therein. Suspicious objects consist of black pixels.
FIG. 7B shows an enlarged portion of the sea portion of the input image data sample shown in FIG. Waves 704 are generated in the sea 702. The seawater 705 portion is a blue pixel and the wave splash 704 portion is a white pixel.
FIG. 7C shows all color components in a matrix. Each pixel existing in the input image data in FIG. 7A is added to the elements corresponding to the matrix 706 one by one. Then, the accumulated points are concentrated in a white region 707 that expresses splashes, a blue region 708 that expresses seawater, and a black region 709 that expresses a suspicious ship. In other words, you should think that a curved surface such as a well-known Gaussian surface is formed at these three locations.
As described above, the color histogram is obtained by counting the color appearance frequency for each pixel in the color image data. You may think that the color particles that make up the image are scattered and collected for the same pigment. For this reason, the pattern concept is lost in the generated data.

FIGS. 8A, 8B, 8C, and 8D are schematic diagrams illustrating frequency analysis, which is one of feature amount calculation methods that can be commonly used in all the embodiments described in the present invention. It is.
FIG. 8A shows a part 801 of input image data. This is the same as that shown in FIG. 7 (b), but shows an enlarged portion of the sea among the samples of input image data. Further, FIG. 8B further enlarges a part thereof to show a part of the wave splash. Pixels corresponding to wave splash are bright and pixels corresponding to seawater are dark. The brightness of each pixel 802 is acquired.
FIG. 8C shows an example in which the brightness of each pixel constituting the input image data is plotted in order from the left side to the right side. It can be seen that the portion corresponding to seawater has low luminance, and the portion corresponding to wave splash has high luminance. Then, the frequency analysis is to treat the figure obtained in this way as if it were a signal and analyze the frequency component.
FIG. 8D shows an example of the result of the frequency analysis. The waveform plotted in FIG. 8C is subjected to Fourier analysis, and the horizontal axis represents frequency and the vertical axis represents level. Finally, the data obtained in this way is used as a matrix. That is, it is a matrix composed of frequency components of the waveform of the luminance of the image data.
As described above, the frequency analysis is a Fourier transform for each pixel in the image data in order to analyze the frequency component by regarding the change in luminance as a waveform. For this reason, a shading pattern, in turn, a pattern of a predetermined pattern is expressed as a frequency component.

FIG. 9 is a diagram for explaining the outline of the co-occurrence probability matrix.
In FIG. 9A, P1 and P2 are any two pixels in the image data. The pixel P1 is separated from the pixel P2 by a distance r and an angle θ. Let r and θ be the relative position function δ. δ = (r, θ).
In FIG. 9B, when δ1 = (1,0 °), the pixel P2 indicates the next adjacent pixel with respect to the pixel P1.
In FIG. 9C, when δ1 = (1, 90 °), the pixel P2 indicates the adjacent pixel immediately above the pixel P1.
The co-occurrence probability matrix is a mechanism that adds a combination of the luminances of two pixels separated by a certain relative position function δ to a square matrix having luminance as the number of elements.

FIG. 10 shows simplified image data and a simplified co-occurrence probability matrix for explanation.
In FIG. 10A, each pixel in the image data composed of 4 × 4 16 pixels has four levels of brightness from 0 to 3.
FIG. 10B is a co-occurrence probability matrix which is a 4 × 4 square matrix having four levels of luminance. Here, a procedure for creating a co-occurrence probability matrix from the image data of FIG. 10A in the relationship of relative position function δ = (r, θ) = (1,0) is shown below.
When (x, y) = (0, 0) in FIG. 10A is the pixel P1, the pixel P2 is (x, y) = (1, 0). At this time, the luminance i of the pixel P1 is 0, and the luminance j of the pixel P2 is 0. Therefore, 1 is added to (i, j) = (0,0). Furthermore, since round trip counting is performed, 1 is added to (i, j) = (0, 0).
If (x, y) = (1, 0) in FIG. 10A is the pixel P1, the pixel P2 is (x, y) = (2, 0). At this time, the luminance i of the pixel P1 is 0, and the luminance j of the pixel P2 is 1. Therefore, 1 is added to (i, j) = (0,1). Furthermore, since round trip counting is performed, 1 is added to (i, j) = (1, 0).
As described above, the operation of counting up the elements of the matrix corresponding to the combination of the luminance of each pixel separated by the relative position function δ is performed for all the pixels. That is, this square matrix is a matrix of counters indicating the number of appearances of the combination of the luminance values of two pixels.
Note that it is up to the designer to perform a round-trip count.

  Due to the nature of the co-occurrence probability matrix, it is desired to take a plurality of relative position functions and generate a plurality of matrices for each. This is because the pattern constituting the image is not always horizontal or vertical. It is desirable to take at least the vertical and horizontal relative position functions.

By these feature amount calculation methods, feature amounts composed of a matrix composed of a large number of numerical data are created for each divided region of the input image data.
A “distance” is calculated by comparing the obtained feature quantity with a predetermined sample value or other feature quantity held in advance. This calculation is called feature amount distance calculation, which will be described later.
Note that the designer determines which of the feature quantity calculation methods listed above is adopted based on a trade-off between the accuracy of the obtained feature quantity and the calculation quantity.

[2.2. Feature distance calculation method]
With the feature amount calculation method described above, a feature amount is created for each divided region of the input image data. These are all matrices consisting of a lot of numerical data.
How to compare matrix data Here, the value to be obtained is the similarity between the two matrix data. In the first embodiment described above, the matrix data constituting the feature amount of the sample background data and the matrix data constituting the feature amount of the divided area.
Various methods for obtaining the similarity of matrix data are conventionally known. Examples are listed below.
(1) A method of taking the difference for each element of the matrix and taking the sum of each element of the obtained matrix (SAD: Sum of Absolute Difference).
(2) A method of taking a difference for each element of the matrix, squaring each element of the obtained matrix, and then summing each element (SSD: Sum of Squared Difference).
(3) A method of taking each element of a matrix as a vector component and taking an inner product of the matrices. That is, a method of taking the product of each element of the matrix and taking the sum of each element of the obtained matrix (inner product: Normalization).

By the comparison operation of these feature amounts, a “distance” consisting of a single scalar value is finally obtained.
A predetermined determination is made by comparing the obtained distance with a predetermined threshold that is held in advance.
Note that the designer determines which of the feature quantity distance calculation methods listed as examples is based on a trade-off between the accuracy of the obtained distance and the calculation amount.

  The feature amount calculation method and the feature amount distance calculation method described above are selected and used in common in the first embodiment described above and in all second, third, fourth, and fifth embodiments described later. It is a possible technology. That is, in any embodiment, any of color histogram, frequency analysis, and co-occurrence probability matrix may be adopted as the feature amount calculation method, and any of SAD, SSD, and inner product is adopted as the feature amount distance calculation method. May be.

[3. Second Embodiment]
FIG. 11 is a block diagram of a monitoring apparatus according to the second embodiment of the present invention. This is based on the content of FIG. In addition, about the part which is common in FIG. 2 explaining 1st Embodiment, it abbreviate | omits description in order to avoid duplication.
The second embodiment is different from the first embodiment in that an average value storage unit 1102 is provided instead of the background feature amount storage unit 202.
The average value storage unit 1102 stores the average value of feature amounts for each divided region obtained from the feature amount calculation unit 114.
The distance calculation unit 116 calculates the distance between the feature amount for each divided region obtained from the feature amount calculation unit 114 and the average value stored in the average value storage unit 1102. As a result of the distance calculation by the distance calculation unit 116, it is determined that a suspicious object exists in the divided area determined not to be the background, and an alarm is output.

FIGS. 12A, 12B, and 12C are image diagrams illustrating the outline of the operation of the monitoring apparatus according to the second embodiment by exemplifying input image data.
In FIG. 12A, as input image data, there is image data 1201 including a sea 1202 and a ship 1204 which is a suspicious object.
In FIG. 12B, the average value storage unit 1102 stores the average values of the feature amounts S (1) to S (25) for each divided region, which are calculated by the feature amount calculation unit 114. That is, it is a value obtained by dividing the sum of the feature amounts of all the divided areas by the number of divided areas.
The distance calculation unit 116 calculates the distance between the average value and the divided area. This distance calculation is performed one by one. That means
Calculate the distance between the average value B and the divided area S (1),
Calculate the distance between the average value B and the divided area S (2),
In order, and finally,
The distance between the average value B and the divided area S (25) is calculated, and the process ends.

13 and 14 are flowcharts illustrating the operation of the monitoring apparatus according to the second embodiment.
When processing is started (S1301), sample input still image data is taken into the RAM (S1302).
Next, the feature amount for each divided region divided by the region dividing unit 113 is calculated by the feature amount calculating unit 114 (S1303).
Then, an average value is calculated from the feature values for all the obtained divided regions (S1304).
From this point onward, the distance between the calculated feature value and the average value for each region is calculated in order.
First, the variable i is initialized to 1 (S1305). The variable i is a variable that specifies a feature amount for each region.
Then, the distance between the average value B and the divided region feature amount S (i) is calculated (S1306).
Next, the result of the distance calculation is compared with a threshold value separately held in advance (S1307). If the distance is greater than or equal to the threshold, the flag variable f (i) is increased to 1 (S1308). The flag variable f (i) is a variable provided in the same number as that of the divided areas. The flag is raised by 1 and the flag is lowered by 0.

Next, the variable i is verified (S1409). If it is not the maximum value as a result of the verification, i is incremented (S1410). If the variable i reaches the maximum value, it means that the distance calculation has been completed for all the divided areas. Therefore, all flag variables f (i) are checked to verify whether there is a variable whose flag is raised (S1411).
As a result of the verification, if the flag is not raised at all, it can be determined that all the divided areas are the background and there is no suspicious object.
On the other hand, as a result of the verification, if even one of the flags is raised, it can be determined that the divided area is not the background and the suspicious object exists there.
If the presence of a suspicious object is recognized, an alarm is output (S1412) and the process ends (S1413).

Summarize what has been described above.
The monitoring device 1101 of this embodiment is mainly composed of software that runs on a microcomputer.
When the input still image data is input to the monitoring device 1101, it is stored in the image storage unit 112 (substantially the RAM 105). After being divided into equal area shapes by the area dividing unit 113, the feature amount calculating unit 114 creates feature amount data for each divided region. Thereafter, the average of the feature amount data of all the divided areas is calculated and stored in the average value holding unit 1102 (the entity is the RAM 105). Next, a distance between the obtained feature amount data and the average value in the average value holding unit 1102 is calculated, and the distance obtained as a result is compared with a predetermined threshold value. A region having a distance less than the threshold is determined to be the background. Then, the flag indicating the divided area determined to be the background is lowered.
Through the above processing, it is possible to discriminate between the background area and the other area areas from the input still image data through the feature amount calculation and the distance calculation.

[4. Third Embodiment]
FIGS. 15A and 15B are block diagrams of a monitoring device according to the third embodiment of the present invention. This is based on the content of FIG. In addition, about the part which is common in FIG. 11 explaining 2nd Embodiment, description is omitted in order to avoid duplication.
In FIG. 15A, the third embodiment is different from the second embodiment in that a clustering processing unit 1503 is provided between the feature amount calculation unit 114 and the comparison unit 1504, and the comparison unit 1504 The clustering processing unit 1503 and the threshold 1502 are compared.
The clustering processing unit 1503 calculates the distance between the feature amounts for each of the divided regions obtained from the feature amount calculating unit 114, and images that are similar between the divided regions that can be determined to be close to each other, that is, the divided regions that can be determined to be similar to each other. A process called “clustering” is performed in which grouping is performed assuming that By this processing, each similar divided region is classified as a set belonging to the same image portion.
The comparison unit 1504 compares the number of elements of each divided region set obtained from the clustering processing unit 1503 with a threshold 1502. As a result of the comparison by the comparison unit 1504, the set of divided areas determined not to be the background is determined to have a suspicious object, and an alarm is output.

FIG. 15B is a block diagram showing an internal configuration of the clustering processing unit 1503 of FIG.
The distance calculation target setting unit 1513 determines which feature amounts of the feature amounts 1512 obtained from the feature amount calculation unit 114 are to be distance calculation targets.
The distance calculation unit 1514 calculates the distance between the two feature quantities selected by the distance calculation target setting unit 1513.
The comparison unit 1516 compares the distance obtained from the distance calculation unit 1514 with a threshold value 1515. This comparison result leads to the determination of whether or not to merge the two regions.
The grouping processing unit 1517 merges the two regions according to the comparison result of the comparison unit 1516, or leaves the regions as they are.
As described above, through a series of processes, the feature amounts for each divided region are classified into groups of several similar feature amounts. After this processing, the data representing the number of groups of feature values is compared with the threshold 1502 by the comparison unit 1504, leading to a determination as to whether or not it is the background.

FIGS. 16A, 16 </ b> B, 16 </ b> C, and 16 </ b> D are image diagrams illustrating the outline of the operation of the monitoring apparatus according to the third embodiment, illustrating input image data.
As input image data, there is image data 1601 including the sea and a ship that is a suspicious object.
The clustering processing unit 1503 compares all the feature amounts S (1) to S (25) for each divided region calculated by the feature amount calculating unit 114, and performs a process of collecting similar regions.
First, the distance calculation unit 1514 calculates the distance between the upper left region S (1) of the region obtained by dividing the image data 1601 in FIG. 16A and the adjacent region S (2). The distance between the two feature amounts obtained as a result of the distance calculation is compared with a threshold 1502 prepared separately in advance, and it is determined whether or not the divided areas are similar to each other. In FIG. 16A, since the regions S (1) and S (2) are both sea portions, their feature quantities are similar, so the distance is short. Therefore, they are considered to be the same pattern and are combined into one. This is the process shown in FIG. That is, it is a concept of erasing the boundary line by combining the areas S (1) and S (2).
From FIG. 16B, the similarity determination is further advanced by calculating the distance between adjacent regions and comparing the obtained distance with the threshold 1502. That is, as shown in FIG. 16C, the boundary lines of similar image segmented areas are erased one after another.
Finally, as shown in FIG. 16 (d), the divided regions S (12), S (13), and S (14) whose feature amounts are significantly different from those of the other divided regions and that are recognized as having a long distance remain. . This is an area where suspicious objects exist.

FIG. 17 is a flowchart illustrating the operation of the monitoring apparatus according to the third embodiment.
When processing is started (S1701), sample input still image data is taken into the RAM (S1702).
Next, the feature amount for each divided region divided by the region dividing unit 113 is calculated by the feature amount calculating unit 114 (S1703).
Then, a clustering process is performed based on the obtained feature values for each divided region (S1704). As a result of the clustering process, the divided areas are grouped into similar areas.
Next, the area area, that is, the number of divided areas constituting a similar area is compared with the threshold 1502, and it is verified whether there is a narrow area that cannot be recognized as a background (S1705).
As a result of the verification, if the number of divided areas constituting all the areas is equal to or greater than the threshold 1502, it can be determined that all the divided areas are the background and that there is no suspicious object.
On the other hand, as a result of the verification, if the number of the divided areas constituting the specific area is less than the threshold 1502, it can be determined that the area is not the background and the suspicious object exists there.
If the presence of a suspicious object is recognized, an alarm is output (S1706) and the process ends (S1707).

Summarize what has been described above.
The monitoring device 1501 of this embodiment is mainly composed of software that runs on a microcomputer.
When the input still image data is input to the monitoring device 1501, it is stored in the image storage unit 112 (substantially the RAM 105). Next, after being divided into equal area shapes by the area dividing unit 113, the feature quantity calculating unit 114 creates feature quantity data for each divided area. After that, the clustering processing unit 1503 calculates the distance between the feature amount data of all the divided areas, determines that the obtained distances are the areas having the same pattern, and integrates them. Do. Thus, the plurality of divided regions are classified into a collection of several regions.
Next, the number of divided areas constituting the aggregate of the plurality of divided areas is compared with a predetermined threshold 1502. A set consisting of a number of divided areas equal to or greater than the threshold 1502 is determined to be the background. Then, the flag indicating the divided area determined to be the background is lowered.
Through the above processing, it is possible to discriminate between the background area and the other area areas from the input still image data through the feature amount calculation and the comparison calculation.

[5. Fourth Embodiment]
FIGS. 18A, 18B, and 18C are an image diagram and an overall block diagram illustrating the outline of the operation of the monitoring apparatus according to the fourth embodiment of the present invention.
In FIG. 18A, the input image data 1801 includes a sea 1802, a sky 1803, and a ship 1804 as a suspicious object floating in the sea. This sea 1802 is a landscape obtained by photographing a distant place, and the actual distance differs greatly between the lower part of the sea part in the input image data 1801 and the vicinity of the horizontal line. For this reason, the wave splash on the sea surface appears to be large in the foreground and small in the distance. That is, the sea does not form a uniform pattern as a whole on the captured still image data.
If such input image data 1801 is evaluated as a whole, it is difficult to obtain precise results.
Therefore, as shown in FIG. 18B, the input image data 1801 is divided into a plurality of parts in the horizontal line direction, and horizontal divided areas 1805, 1806, 1807, 1808, 1809 and 1810 are created.
Feature amount calculation, comparison, and the like are performed on each horizontal division region of the divided image data.

A block diagram for realizing this technical idea is shown in FIG. This is almost the same as FIG. The monitoring apparatus 1811 shown in FIG. 18C is different from the monitoring apparatus 101 in FIG. 1B in that the area dividing unit 1813 is divided into a horizontal area dividing unit 1813a and a small area dividing unit 1813b. It is.
In the actual calculation processing, the processing described in the first embodiment, the second embodiment, and the third embodiment is performed on the horizontally divided regions 1805, 1806, 1807, 1808, 1809, and 1810. It becomes.
It should be noted that how to set the division width of the horizontal region dividing unit 1813a depends on the trade-off between the state of the input image data 1801 to be processed and the amount of calculation required for the processing, or the actual performance of the apparatus. Determined before operation.
The processing flow may be to try this embodiment when the results of the processing according to the first to third embodiments are not satisfactory.

Summarize what has been described above.
The monitoring device 1811 of this embodiment is mainly composed of software that runs on a microcomputer.
When the input still image data is input to the monitoring device 1811, it is stored in the image storage unit 112 (the substance is the RAM 105). Next, after being divided in a direction parallel to the horizontal line by a horizontal region dividing unit 1813a in the region dividing unit 113, it is divided into small regions having an equal area shape by a small region dividing unit 1813b. Thereafter, the feature amount calculation by the feature amount calculation unit 114 and the distance calculation by the distance calculation unit 116 are performed for each horizontal division region.
For each horizontal divided area, it is determined whether or not the small divided area to be evaluated is the background. Then, the flag indicating the small divided area determined to be the background is lowered.
Through the above processing, it is possible to discriminate between the background area and the other area areas from the input still image data through the feature amount calculation and the distance calculation.

[6. Fifth embodiment]
FIG. 19 is a block diagram of a monitoring apparatus according to the fifth embodiment of the present invention.
In FIG. 19, the divided region determination unit 1902 in the monitoring device 1901 is the same as the monitoring device 101 in FIG. That is, in this part, the same function as any of the first, second, third, and fourth embodiments described above operates.
A set of flag data indicating the presence / absence of a suspicious object is output from the divided area determination unit 1902 for each divided area of the input image data divided by the area dividing unit 113 through the distance calculation unit 116. .
The timer 1903 outputs a trigger for operating the divided region determination unit 1902 at regular intervals.
The divided area flag storage unit 1904 includes a RAM, and stores a plurality of sets of flag data output from the divided area determination unit 1902.
The suspicious object region movement amount / direction calculation unit 1905 compares a plurality of flag data sets stored in the divided region flag storage unit 1904 to determine whether or not the region considered to be a suspicious object has moved, If it is moving, its moving direction is calculated, and if it is recognized that it is moving, an alarm is output.

FIGS. 20A and 20B are image diagrams showing an outline of the operation of the monitoring apparatus according to the fifth embodiment, illustrating input image data. It also shows an image of a collection of flags.
The divided area determination unit 1902 outputs a set of flags indicating whether or not the input image data is a suspicious object for each divided area. In FIG. 20A, 0 indicates a divided area that is determined not to be a suspicious object, that is, a background, and 1 indicates a divided area that is determined to be a suspicious object.
Normally, suspicious objects move. There cannot be a suspicious object that does not move. Therefore, when the divided region determination unit 1902 is operated at regular time intervals and the sets of flags to be output are compared, it can be seen that the divided region indicating the suspicious object moves. In the figure, the area determined to be a suspicious object moves with the passage of time.
That is, the comparison between still images with a time interval, which is performed in the prior art, is compared with the flag of the divided region based on the background determination technique according to the present embodiment instead of the raw still image.
By determining the background, it is possible to eliminate misrecognition due to the movement of the background and to detect the movement of the suspicious object with much higher accuracy than in the prior art.

FIG. 21 is a flowchart illustrating the operation of the monitoring apparatus according to the fifth embodiment.
When the process is started (S2101), sample input still image data is taken into the RAM (S2102).
Next, the divided region determination process described in the first, second, third, and fourth embodiments is performed (S2103). That is, the input still image data is divided by the region dividing unit 113, the feature amount is calculated for each divided region, and the distance is calculated based on the feature amounts for all the divided regions. In this way, it is determined for each divided area whether it is a background or a suspicious object. Then, the set of flags for each divided area obtained as a result of the determination is stored in the divided area flag storage unit 1904 (RAM 105) (S2104).
Next, it is confirmed whether or not there is a previous set of flags in the RAM 105 (S2105). If there is no previous set of flags, the determination result is the first, so that after a time interval by the timer 1903, the input still image data is taken in again and a series of processing is performed.
If there is a previous set of flags, each set of flags is compared (S2106). If there is a difference of a certain value or more as a result of the comparison (S2107), it indicates the movement of a suspicious object, so an alarm is output (S2108).

Summarize what has been described above.
The monitoring device 1811 of this embodiment is mainly composed of software that runs on a microcomputer.
The divided region determination unit 1902 stores input still image data in the image storage unit 112 (substantially the RAM 105) in response to a trigger generated by the timer 1903. The input still image data is divided into equal area shapes by the region dividing unit 113, and then feature amount data is created for each divided region by the feature amount calculating unit 114. Next, the distance between the obtained feature data and the predetermined value 115 is calculated, and the obtained distance is compared with a predetermined threshold. A region having a distance less than the threshold is determined to be the background. Then, the flag indicating the divided area determined to be the background is lowered.
The set of flags obtained in the above process is sequentially stored in the divided region flag storage unit 1904 in response to the trigger issued by the timer 1903. The suspicious object region movement amount / direction calculation unit 1905 compares the set of saved flags at regular time intervals to determine whether there is a moving suspicious object.
Through the above processing, it is possible to discriminate between the background area and the other area areas from the input still image data through the feature amount calculation and the comparison calculation.

The following application examples can be considered in the present embodiment.
(1) Instead of mounting by a microcomputer, a programmable logic device (PLD) may be used.
(2) In the first, second, third, and fourth embodiments described above, it is possible to add an exclusion process that excludes a background portion that is easily recognized as a suspicious object from the target of the feature amount calculation process.
(3) A plurality of the first, second, third, fourth and fifth embodiments described above can be simultaneously operated in a single monitoring device. The set of flags for each region obtained as a result of the arithmetic processing can be all merged by logical product or compared with a predetermined threshold after the addition processing.

In the present embodiment, an object different from the background can be determined from still image data. This processing is different from the method based on comparison with the immediately preceding image data, which is well known in the prior art, and is determined using only the input image data. For this reason, it is particularly suitable in the monitoring mode in which the imaging camera is installed on the swivel device and rotated to the left and right.
In the present embodiment, for discrimination, the image data is divided into predetermined areas, and feature amounts are calculated for each divided area. Then, the distance between the feature amount for each divided region and a predetermined value is calculated. For this reason, even if there is a change caused by a natural phenomenon such as the sea or the sky, it can be determined as a background having certain characteristics including this.
Therefore, by combining the image processing apparatus according to the present embodiment with an imaging camera, an excellent monitoring apparatus that can recognize a natural phenomenon such as a sea wave or a cloud in the sky as a background without being misrecognized as compared with the prior art. Can be provided.

  The embodiment of the present invention has been described above. However, the present invention is not limited to the above-described embodiment, and other modifications may be made without departing from the gist of the present invention described in the claims. It goes without saying that application examples are included.

1 is an overall block diagram of an image processing apparatus according to a technical idea common to embodiments of the present invention. 1 is an overall block diagram of an image processing apparatus according to a first embodiment. It is an image figure which shows schematically operation | movement of 1st Embodiment. It is a flowchart which shows pre-processing among the processing contents of the monitoring apparatus of 1st Embodiment. It is a flowchart which shows the process at the time of real operation among the processing content of the monitoring apparatus of 1st Embodiment. It is a flowchart which shows the process at the time of real operation among the processing content of the monitoring apparatus of 1st Embodiment. It is the schematic explaining the color histogram which is one of the feature-value calculation methods. It is the schematic explaining frequency analysis which is one of the feature-value calculation methods. It is the schematic explaining the co-occurrence probability matrix which is one of the feature-value calculation methods. It is the schematic explaining the relationship between image data and a co-occurrence probability matrix. It is a whole block diagram of the image processing apparatus by 2nd Embodiment. It is an image figure which shows schematically operation | movement of 2nd Embodiment. It is a flowchart which shows the processing content of the monitoring apparatus of 2nd Embodiment. It is a flowchart which shows the processing content of the monitoring apparatus of 2nd Embodiment. It is a whole block diagram of the image processing apparatus by 3rd Embodiment. It is an image figure which shows schematically operation | movement of 3rd Embodiment. It is a flowchart which shows the processing content of the monitoring apparatus of 3rd Embodiment. It is the image figure which shows schematically operation | movement of the image processing apparatus by 4th Embodiment, and an entire block diagram. It is a whole block diagram of the image processing apparatus by 5th Embodiment. It is an image figure which shows schematically operation | movement of 5th Embodiment. It is a flowchart which shows the processing content of the monitoring apparatus of 5th Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Monitoring apparatus 102 ... Imaging camera 103 ... CPU, 104 ... ROM, 105 ... RAM, 106 ... Bus, 107 ... Output interface, 112 ... Image storage part, 115 ... Predetermined value, 113 ... Area division part, 114 ... Feature amount calculation unit, 115 ... predetermined value, 116 ... distance calculation unit

Claims (8)

An image storage unit for storing input image data;
A region dividing unit that divides the input image data in the image storage unit into an equal area and a fixed shape;
A feature amount calculating unit that calculates a feature amount for each divided region divided by the region dividing unit;
A distance calculation unit that calculates a distance between the feature value for each divided region obtained by the feature value calculation unit and a predetermined value, and determines whether the background is based on the obtained distance; A featured image processing apparatus.   The image processing apparatus according to claim 1, wherein the predetermined value is a background feature amount of a sample.   The image processing apparatus according to claim 1, wherein the predetermined value is an average value of feature amounts of the entire input image data.   The image processing according to claim 1, wherein the predetermined value is a feature amount for each divided region, and grouping processing is performed by determining whether the divided regions are similar based on the obtained distance. apparatus. The area dividing unit includes:
A horizontal region dividing unit that divides parallel to the horizontal line;
A small area dividing unit that further divides the horizontal divided area divided by the horizontal area dividing unit;
The image processing apparatus according to claim 1, wherein the distance calculation unit performs an operation for each horizontal division region. Furthermore,
A determination result for each divided region obtained from the distance calculation unit, a divided region flag storage unit that stores the determination result every predetermined time,
The image processing apparatus according to claim 1, further comprising: a suspicious object region movement amount / direction calculating unit that compares the plurality of determination results stored in the divided region flag storage unit. Dividing the input image data into a uniform area and a fixed shape;
Calculating a feature amount from the divided input image data;
Calculating a distance between a feature value for each divided region and a predetermined value;
An image processing method comprising: determining whether each of the divided areas is a background based on a distance obtained by distance calculation. A function of dividing input image data into equal divided areas;
A function of calculating a feature value for each of the divided regions;
A function for calculating a distance between a feature value for each divided region and a predetermined value;
An image processing program comprising a function that compares the distance with a predetermined threshold and uses the comparison result as a basis for determining whether or not the divided region is a background.

Images