JP2003304531A - Image processing apparatus, image processing method, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To properly particularize an existing area of a monitoring object even when illumination or the like gives effect on the monitoring. <P>SOLUTION: An image processing apparatus is provided with: a difference absolute value calculation means 12 for calculating a difference absolute value between frame image data at a first time captured by an image acquisition means 11 and frame image data at a time earlier than the first time; a binary processing means 13 for applying binary processing to the difference absolute value; a monitoring object area extract means 14 for dividing the area of the object on the basis of the obtained binary processing data and extracting the area of the monitoring object at the first time from the divided areas; a control means 10 for using the frame image data at least at one past time different from the earlier time above, and causing the means 12, 13, 14 to perform the calculation of the difference absolute value, the binary processing, and extraction of the monitoring object as to extract a plurality of the areas of the monitoring object at the first time; and a composite means 15 for composing a plurality of the areas. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, an image processing method and an image processing program suitable for use in a road monitoring system or the like.

[0002]

2. Description of the Related Art As a conventional road monitoring system and an image processing apparatus used for it, for example, Japanese Patent Laid-Open No. 10-3079 is known.
88, JP 9-97335, and JP 11
The one shown in Japanese Patent No. 353581 is known.

By the way, in the road monitoring system, it is difficult to specify the existing area of the monitored object based on the surrounding environment where the brightness changes. Also, in an environment where the brightness due to lighting is not constant and there is a large unevenness in the spatial brightness distribution, such as in a tunnel, light and dark stripes occur on the road surface, and the vehicle that is the monitoring target has the light and dark stripes. When the vehicle runs on a road where the light is generated, bright and dark stripes are transferred to the vehicle, which makes it difficult to detect the vehicle. Especially for vehicles such as trucks whose surface is made of aluminum with no pattern,
There is a problem that it is difficult to trace the contour due to a change in the reflection portion other than the occurrence of the bright and dark stripes.

[0004]

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems in image processing in the related art, and its purpose is to stripe a monitored object due to the influence of illumination such as in a tunnel. It is possible to provide an image processing device, an image processing method, and an image processing program capable of appropriately specifying the existing area of the monitored object even if the monitored object is changed or the reflection portion is changed due to the monitored object. .

[0005]

An image processing apparatus according to the present invention comprises image acquisition means for taking in monitoring target image data,
Difference absolute value calculation means for calculating a difference absolute value between the frame image data at the first time captured by the image acquisition means and the frame image data at a time earlier than the first time, and the difference absolute value calculation The binarization processing means for binarizing the difference absolute value calculated by the means, and the area division of the object based on the binarized data obtained by the binarization processing means The difference absolute value calculation is performed by using a monitoring target area extraction unit that extracts the area of the monitoring target at the first time from the area, and at least one frame image data at the past time different from the past time. The calculation is performed by the means, and the calculation result is processed by the binarization processing means and the monitoring target area extraction means, and the monitoring at the first time is performed. Characterized by comprising a control means for a plurality extracting an area of an object, and a combining means for performing regional composite for a plurality areas of the monitored object in the extracted first time.

Further, in the image processing apparatus according to the present invention, the synthesizing means superimposes a plurality of areas of the extracted monitoring target object at the first time to form a monitoring target area. To do.

Further, in the image processing apparatus according to the present invention, the synthesizing means obtains an area of the monitored object by superimposing a plurality of areas of the extracted monitored object at the first time, and at the same time, the monitored object is monitored. It is characterized in that a region where the target object cannot exist is removed to be a region of the monitoring target object.

According to the image processing method of the present invention, the image acquisition step of capturing the image data to be monitored, the frame image data at the first time captured in the image acquisition step, and the frame image data before the first time. A difference absolute value calculation step of calculating a difference absolute value of the frame image data at a time; a binarization processing step of performing a binarization process on the difference absolute value calculated by the difference absolute value calculation step; A monitoring object region extraction step of performing a region division of the object based on the binarized data obtained by the binarization processing step, and extracting a region of the monitoring target object at the first time from the divided region; The calculation in the difference absolute value calculation step is performed using at least one frame image data at a past time different from the time A repeated processing step of causing the binarization processing step and the monitoring target area extraction step to perform the calculation result on the calculation result, and extracting a plurality of monitoring target areas at the first time; And a synthesizing step of synthesizing regions for a plurality of regions of the monitored object at the first time.

An image processing program according to the present invention is an image processing program used in a computer for performing processing by a central control unit based on a program stored in a main memory to obtain a result. An image acquisition step of capturing image data, and a difference absolute for calculating an absolute difference value between the frame image data at the first time captured at the image acquisition step and the frame image data at a time earlier than the first time. A binarization process is performed on the value calculation step and the difference absolute value calculated by the difference absolute value calculation step 2
A monitoring target that performs a binarization processing step and area division of an object based on the binarized data obtained by the binarization processing step, and extracts the area of the surveillance target object at the first time from the divided area. The object area extracting step and the frame image data at least one past time different from the past time are used to perform the calculation in the difference absolute value calculating step, and the calculation result is converted into the binarizing processing step. Iterative processing step of causing the processing by the monitoring target area extraction step to extract a plurality of monitoring target areas at the first time, and a plurality of extracted monitoring target areas at the first time. And a synthesizing step of synthesizing the regions.

[0010]

BEST MODE FOR CARRYING OUT THE INVENTION An image processing apparatus, an image processing method and an image processing program according to the present invention will be described below with reference to the accompanying drawings. In each drawing, the same constituents are designated by the same reference numerals, and duplicated description will be omitted. FIG. 1 shows the configuration of the image processing apparatus. This image processing apparatus is an image processing unit computer 2 of the monitoring system shown in FIG.
It is realized by.

As shown in FIG. 2, the monitoring system includes an image processing unit computer 2, an integrated processing unit computer 3, an event determination unit computer 4, and a notification board 5-1 to 5-n for displaying via a network 1. Are connected. Image processing computer 2
Are connected to cameras 6-1 to 6-m (m is an integer).

The above system operates as follows. The cameras 6-1 to 6-m capture images of the monitoring target and send the image data to the image processing unit computer 2. The image processing unit computer 2 executes the identification of the existence area of the monitoring target as described later, and sends the processing result information to the integrated processing unit computer 3. The integrated processing unit computer 3 uses, for example, the cameras 6-1 to 6-1 based on the information sent from the image processing unit computer 2.
6-m, such as determining how to use the result of processing by the image processing unit computer 2 for the image data obtained, extracting or combining the image data necessary for the next event determination, etc. Perform processing. The result is sent from the integrated processing unit computer 3 to the event determination unit computer 4.
The event determination unit computer 4 uses the sent image data and the result processed by the integrated processing unit computer 3 to determine the event that has occurred. Here, the event is, for example, in a road monitoring system, an accident, a falling object, a traffic jam, a fire, or the like.

Based on the result of the judgment, the event judging section computer 4 judges to which one of the notification boards 5-1 to 5-n the information of what kind is to be sent, and the like. Send to the required alarm board. Thus, the alarm board 5-1
The required information is notified by ~ 5-n.

In the image processing section computer 2 described above, an image processing apparatus is constituted by the respective means shown in FIG.
This image processing apparatus includes a control unit 10 and an image acquisition unit 1.
1, a difference absolute value calculation unit 12, a binarization processing unit 13, a monitoring target area extraction unit 14, and a combination unit 15. The image acquisition means 11 takes in the monitoring target image data from the cameras 6-1 to 6-m. The difference absolute value calculation means 12 calculates the difference absolute value between the frame image data at the first time taken in by the image acquisition means 11 and the frame image data at a time earlier than the first time. .

The binarization processing means 13 performs binarization processing on the difference absolute value calculated by the difference absolute value calculation means 12. Monitoring target area extraction means 14
Is to perform area division of the object based on the binarized data obtained by the binarization processing means 13 and extract the area of the monitored object at the first time from the divided area.

The control means 10 causes the difference absolute value calculation means 12 to perform calculation using the frame image data at at least one past time different from the past time, and the calculation result is binarized. Processing means 1
3 and the monitoring target region extraction means 14 to perform a process to extract a plurality of regions of the monitoring target at the first time. The synthesizing unit 15 performs region synthesizing on a plurality of regions of the monitored object at the extracted first time.

The image processing section computer 2 constituting the above image processing apparatus is constituted by, for example, a personal computer, a workstation or other computer, and is constituted by, for example, the components shown in FIG. That is, the computer shown in FIG. 3 has a CPU 51 that controls the apparatus as a whole,
A main storage device 52 for storing information such as programs and data used by the CPU 51 is connected to the CPU 51. Further, a keyboard control unit 54, a display control unit 55, a printer control unit 56, a communication interface 57, a mouse control unit 58, and a magnetic disk control unit 59 are connected to the CPU 51 via a system bus 53. The keyboard control unit 54 is connected to a keyboard input device 60 capable of inputting various information by keys, the display control unit 55 is connected to a CRT display device 61 for displaying information, and the printer control unit 56 prints information. A printer device 62 for output is connected, a communication processing unit 63 for performing communication via a line is connected to the communication interface 57,
A mouse 64 which is a pointing device is connected to the mouse controller 58, and a magnetic disk device 65 which is an auxiliary storage device is connected to the magnetic disk controller 59. The image processing unit computer 2 is provided with at least a CPU 51, a main storage device 52, a magnetic disk control unit 59, a magnetic disk device 65, a communication interface 57, and a communication control unit 63. Further, a flexible disk drive, a magnetic card or IC card reader, an MO (magneto-optical disk) drive, etc. are provided as needed.

Each means shown in FIG.
1 is realized by performing the image processing method according to the present invention by the program corresponding to the flowchart shown in FIG. 4, the operation of the image processing apparatus according to the present invention will be described with reference to this flowchart. The program corresponding to the above flowchart is an image processing program according to the present invention, and is stored in, for example, the magnetic disk device 65,
It is read out to the main memory 52 and executed. Here, a case will be described in which the first time is set as the current time and the existing area of the monitored object at the current time is extracted. First, the operation is started, and image data is captured from the cameras 6-1 to 6-m (S2: image acquisition means 11).

Here, the processing for the image data captured from the camera 6-1 will be described, but the processing for the image data captured from another camera is also performed in the same manner. Then, the image at the current time captured from the camera 6-1 is an image inside the tunnel as shown in FIG.
A vehicle S traveling on the road R is included. Stripes appear on the vehicle S due to the influence of illumination, and the reflection portion changes with time.

Therefore, the absolute difference value between the frame image at the current time and the frame image at the past time i is calculated (S2: absolute difference value calculating means 12). Here, the time i in the past is, for example, t from the current time.
The time is seconds before. In the calculation of the absolute value of the difference between the frame image at the current time and the frame image at the time i in the past, as shown in FIG.
A frame image at the current time composed of data represented by brightness values of 6 gradations (0 to 255) (see FIG. 6).
Similarly to (a)), for example, 256 gradations (0 to 0) are set for each pixel.
The difference between the frame images (FIG. 6 (b)) at the time i in the past composed of the data represented by 255) is calculated, and the absolute value of the difference is calculated by 256 gradations forming one screen as shown in FIG. It is obtained as a set of expressed pixel data.

Next, the data for one screen, in which the pixel data of the absolute difference values obtained above are collected, is binarized for each pixel using a predetermined threshold value, and the data of each pixel is obtained. It is set to either 1 or 0 (S3: Binarization processing means 3). As a result, for example, as shown in FIG. 8, it is divided into a portion where “1” s are gathered and a portion where “0” is not described in FIG.

The area of the object is divided based on the binarized data as described above, and the area of the monitored object at the current time is extracted from the divided area (S4: monitored object area extracting means 14). That is, as shown in FIG. 8, a labeling process for grouping consecutive "1" s and numbering (No.1, No.2, No.3, ...) To the grouped regions. Then, the area of the monitored object or the area of other objects is extracted. In the example of FIG. 8, three areas are detected and correspond to the vehicle S as is clear by comparison with FIG.
In addition to the No. 1 area, unnecessary No. 2 and No. 3 areas are extracted.

Next, it is detected whether a predetermined number of past frame image data has been used (S5), and when not used, the past time i is set to i + 1 (S6), and the processing from step S2 to step S4 is performed. Repeat (control means 10). That is, the past time i is, for example, 2 from the current time.
t seconds before, the same processing as above is executed, the past time i is set to, for example, 3t seconds before the current time, the same processing as above is executed, and only a predetermined number of past frame images are used. Process.

As a result, a plurality of areas (No. 1 areas) of the monitored object at the current time as shown in FIG. 8 are extracted. For example, by using the frame images t, 2t, and 3t seconds past the current time, three areas of the monitored object at the current time are extracted. Areas other than the monitored object at the current time (areas No. 2 and No. 3 in FIG. 8) appear at different positions by the above three processes unless the vehicle as the monitored object actually exists. The process of removing is performed.

When the area of the vehicle S at the current time shown in FIG. 5 is the area surrounded by a thick frame in FIGS. 9A, 10A and 11A, the current time is As shown in FIG. 9A, for example, the result of using the frame image that is older than t seconds is specified as a region in which “1” is collected.
Therefore, the area as shown in FIG. 9B is set as the area of the monitored object at the current time. Further, the result of using the frame image 2t seconds past the current time is, for example, as shown in FIG.
As shown in, the area is identified as a group of "1" s. Therefore, the area as shown in FIG. 10B is set as the area of the monitored object at the current time. Further, the result of using the frame image 3t seconds past the current time is shown in FIG.
As shown in (a), it is specified as an area in which "1" is collected. Therefore, the area as shown in FIG. 11B is set as the area of the monitored object at the current time.

If there are three preset frame images set in advance in this way, the process branches to YES in step S5, and a plurality of monitored objects (here, three patterns) are obtained at the present time. Area is synthesized (S
7: Synthesis means 15). In the synthesis processing here, first, the synthesis is simply performed by an OR (logical sum) operation. Figure 9
When the regions shown in (b), FIG. 10 (b), and FIG. 11 (b) are being extracted, the vehicle at the current time shown in FIG. 5 as shown in FIG. 12 by the above OR (logical sum) operation. The area of S is specified.

The information of the specific area is used by the image processing unit computer 2 to specify the position of the vehicle and used for necessary processing.
It is sent to the integrated processing unit computer 3 and the event determination unit computer 4 and used for event determination.

The synthesizing process is shown to be a simple OR (logical sum) operation (superposition), but the result of this operation is compared with, for example, the shape data of the vehicle prepared in advance, O if the result is irrational
The target used for the R calculation is limited. For example, instead of using all three combinations, the result is a reasonable combination of the two combinations. In addition, the result that the part where the vehicle cannot exist, such as the upper part of the wall or the ceiling part in the tunnel, is extracted as the region, and the result exists that the vehicle exists, such as the upper part or the ceiling part of the wall in the tunnel. It is removed using the area database (included in the image processing apparatus) of the impossible part. Further, a speed sensor is installed, the vehicle position at the past time is predicted based on the speed data obtained from this sensor, and the result that the position where the vehicle cannot exist is extracted as a region is removed. The final result is output using one or more of these correction processes. Also, although the above has shown that the area of the monitored object at the current time is specified, the present invention can be generally used to specify the area of the monitored object at a certain time.

[0029]

As described above, according to the present invention, the absolute difference value between the captured frame image data at the first time and the frame image data at a time earlier than the first time is calculated, The calculated absolute difference value is binarized, the object area is divided based on the obtained binarized data, and the area of the monitored object at the first time is extracted from the divided area. Process
Further, the difference absolute value calculation is performed using at least one frame image data at the past time different from the past time, and the calculation result is subjected to the binarization process and the process related to the monitoring target region extraction. Then, a plurality of regions of the monitored object at the first time are extracted, and regions are combined for the plurality of extracted regions of the monitored object at the first time. As a result, it is possible to extract an area that cannot be obtained by one difference process, and it is possible to appropriately specify the existing area of the monitored object.

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of an image processing apparatus according to the present invention.

FIG. 2 is a configuration diagram of a road monitoring system configured by using the image processing apparatus according to the present invention.

FIG. 3 is a block diagram showing a practical configuration example of an image processing apparatus according to the present invention.

FIG. 4 is a flowchart for explaining the operation of the image processing apparatus according to the present invention.

FIG. 5 is a diagram showing an example of a screen on which image processing is performed by the image processing apparatus according to the present invention.

FIG. 6 is a diagram showing image data at a current time and image data at a past i time for obtaining an absolute difference value by the image processing apparatus according to the present invention.

FIG. 7 is a diagram showing image data in a state where absolute difference values are obtained by the image processing apparatus according to the present invention.

FIG. 8 is binarized by the image processing apparatus according to the present invention,
The figure which shows the image data in the state where labeling was performed.

FIG. 9 is a diagram showing a main part of a region specifying processing result for explaining image processing using image data of the present time and t seconds past the present time by the image processing apparatus according to the present invention.

FIG. 10 is a diagram showing a main part of a region specifying processing result for explaining image processing using image data of a current time and a time 2t seconds before the current time by the image processing apparatus according to the present invention.

FIG. 11 is a diagram showing a main part of a region specifying processing result for explaining image processing using image data of a current time and a time 3t seconds before the current time by the image processing apparatus according to the present invention.

FIG. 12 is a diagram showing a process of synthesizing a plurality of region specifying process results by the image processing apparatus according to the present invention to obtain a final region of a monitoring object.

[Explanation of symbols]

1 network 2 Image processing computer 3 Integrated processing unit computer 4 Event judgment part calculator 5-1 to 5-n Alarm board 6-1 to 6-m camera 10 Control means 11 Image acquisition means 12 Difference absolute value calculation means 13 Binarization processing means 14 Monitoring target area extraction means 15 Synthetic means

Front page continuation (51) Int.Cl. ⁷ identification code FI theme code (reference) G06T 7/20 200 G06T 7/20 200A G08G 1/04 G08G 1/04 C F term (reference) 5B057 AA16 BA02 CA08 CA12 CA16 CB06 CB12 CB16 CE08 CE12 DA08 DB02 DB09 DC32 DC36 5C054 CA04 CC03 CD03 EA01 FC05 FC12 FC13 HA26 5H180 AA01 CC04 JJ26 5L096 AA06 BA02 CA02 CA04 DA01 EA43 FA14 GA08 HA03

Claims (5)

[Claims] 1. An image acquisition unit for capturing monitoring target image data, absolute difference between frame image data at a first time captured by the image acquisition unit and frame image data at a time earlier than the first time. A difference absolute value calculating means for calculating a value; a binarization processing means for performing a binarization process on the difference absolute value calculated by the difference absolute value calculating means; and a binarization processing means. The area of the object is divided based on the binarized data, and the first area is divided from the divided area.
Using the monitoring target area extraction means for extracting the monitoring target area at the time of, and the frame image data at least one past time different from the past time, the difference absolute value calculation means performs the calculation. In addition, a control unit that causes the binarization processing unit and the monitoring target region extraction unit to perform processing on the calculation result to extract a plurality of monitoring target regions at the first time, An image processing apparatus, comprising: a synthesizing unit that synthesizes a region of a plurality of regions of the monitoring target at the first time. 2. The image processing according to claim 1, wherein the synthesizing unit superimposes a plurality of regions of the extracted monitoring target object at the first time to form a monitoring target object region. apparatus. 3. The combining means obtains an area of the monitored object by superimposing a plurality of areas of the extracted monitored object at the first time, and removes an area where the monitored object cannot exist. The image processing apparatus according to claim 1, wherein the area is the area to be monitored. 4. An image acquisition step of capturing monitoring target image data, a difference between the frame image data at the first time captured at the image acquisition step and the frame image data at a time earlier than the first time. A difference absolute value calculation step of calculating an absolute value; a binarization processing step of performing a binarization process on the difference absolute value calculated by the difference absolute value calculation step; and a binarization processing step Monitoring object region extraction step of performing region segmentation of the object based on the binarized data and extracting the region of the monitoring target region at the first time from the segmented region, and at least one different from the past time Using the frame image data at the past time, the calculation in the difference absolute value calculation step is performed, and the calculation result is The repetitive processing step of causing the binarization processing step and the monitoring target area extraction step to perform a plurality of extraction of the monitoring target area at the first time, and the extracted first An image processing method, which comprises a region combining process for a plurality of regions of the monitoring target at a time. 5. An image processing program used in a computer, wherein a central control unit performs processing based on a program stored in a main memory to obtain a result, wherein an image acquisition step of capturing monitoring target image data into the central processing unit. And a difference absolute value calculation step of calculating a difference absolute value between the frame image data at the first time captured in the image acquisition step and the frame image data at a time earlier than the first time, the difference A binarization processing step of performing a binarization processing on the difference absolute value calculated by the absolute value calculation step, and a region division of the object based on the binarized data obtained by the binarization processing step, A monitoring target area extraction step of extracting a monitoring target area at the first time from the partitioned area; The calculation in the difference absolute value calculation step is performed using frame image data at least one past time different from the past time, and the binarization processing step and the monitoring target object area extraction are performed on the calculation result. And a repeating processing step of extracting a plurality of regions of the monitoring target object at the first time, and performing region combination for the plurality of extracted regions of the monitoring target object at the first time. An image processing program for executing a combining step.