JP4065137B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a monitoring image processing apparatus that issues a notification when a change caused by the movement of an object occurs in a captured image, and in particular, the image is divided into a plurality of blocks according to the state of noise generation for each block. The present invention relates to an image processing apparatus, an image processing method, and a program for determining whether to issue a report.
[0002]
[Prior art]
FIG. 12 is a diagram showing a configuration of a conventional image processing apparatus used for monitoring. In the figure, reference numeral 1 denotes a video input unit connected to an imaging apparatus (not shown), which inputs video data to be processed by the image processing apparatus. Reference numeral 2 denotes an A / D converter that quantizes video data, which is an analog signal input from the video input unit 1, and converts it into a digital signal. A current image storage unit 3 inputs and stores current video data (hereinafter referred to as current image data) converted into a digital signal by the A / D converter 2. Reference numeral 4 denotes a background image accumulating unit that performs a predetermined operation on the current image data input from the A / D converter 2 to generate and store background image data for comparison. Reference numeral 5 denotes a background update unit, which updates the data content (background) by performing a predetermined calculation on the background image data stored in the background image storage unit 4. For example, the average of the pixel values of the current image data corresponding to the past five scenes is used as the updated background image data. The current image storage unit 3, the background image storage unit 4, and the background update unit 5 can be realized, for example, by causing a computer device to execute a program having these functions.
[0003]
Reference numeral 6 denotes a background difference unit (changed area extraction means) that compares the image data stored in the current image storage unit 3 and the background image storage unit 4 and calculates the difference image data. Reference numeral 7 denotes a threshold value calculation unit (change area extraction unit) that calculates a threshold value for binarizing the difference image data calculated by the background difference unit 6. In general, an optimum threshold value is obtained by a simple rule based on difference image data, and for example, an average of pixel values of difference image data is set as the threshold value. Reference numeral 8 denotes a binarization unit (change area extraction means), which binarizes the difference image data with the threshold obtained by the threshold calculation unit 7. Reference numeral 9 denotes a feature amount calculation unit which receives the binary data calculated by the binarization unit 8 and calculates a feature amount. The parameters that define the feature amount include, for example, the area of the change area indicating the pixel area that has changed over time in the image, the vertical and horizontal dimensions, and the like. The background difference unit 6, the threshold value calculation unit 7, the binarization unit 8, and the feature value calculation unit 9 can be realized by causing a computer device to execute a program having these functions, for example.
[0004]
Reference numeral 10 denotes a feature parameter setting value (predetermined setting value), which is a parameter value for each feature parameter set in advance under a predetermined condition that defines the report target. The feature parameter setting value 10 is stored in a storage device that can be appropriately read out by an arithmetic processing unit of a computer device functioning as the recognition processing unit 11. A recognition processing unit 11 compares the feature amount data calculated by the feature amount calculation unit 9 with the feature amount parameter setting value 10 to be notified to determine whether or not an event has occurred. Reference numeral 12 denotes a notification informing the event discovery. Reference numeral 13 denotes an alarm processing unit that receives the alarm 12 and issues an alarm to the operator. As a reporting method, for example, the reporting processing unit 13 may be configured together with an appropriate sound source, and a beep sound may be generated for reporting.
[0005]
FIG. 13 shows a display screen of current image data. This figure shows a video of the main gate of the company. As shown in the figure, a person is moving near the center of the video. The position of the moving person becomes the change area as it is. FIG. 14 is a diagram showing a display screen of current image data in which a black solid area is superimposed on the change area in FIG. As a method for extracting (cutting out) a specific image area in a video, as shown in FIG. 14, a black solid area is set in the specific image area (change area) to emphasize the difference from the background image. Thereby, the change with time can be easily obtained.
[0006]
Next, the operation will be described.
First, when an external image pickup device (not shown) picks up an image of a company's main gate as shown in FIG. 13, for example, the image signal is input to the video input unit 1 as current image data. The video input unit 1 sends the current image data input in time series from the imaging device to the A / D converter 2. The A / D converter 2 converts the received current image data, which is an analog signal, into a digital signal. For example, it is converted into multi-value data of 8 bits per pixel.
[0007]
Thereafter, the current image data digitally converted by the A / D converter 2 is sent to the current image storage unit 3 and the background image storage unit 4. At this time, the current image storage unit 3 stores the input current image data and prepares for subsequent processing. That is, the current image storage unit 3 accumulates the current image data captured by the imaging device in time series. The background image storage unit 4 creates and stores background image data based on the input current image data.
[0008]
Further, the background update unit 5 periodically updates the background image data stored in the background image storage unit 4 according to a certain rule. In general, arithmetic processing is performed based on current image data sampled periodically. For example, a method of using an average image obtained by averaging pixel values of the current image data of the past five scenes as updated background image data, a method of replacing current image data of a predetermined time, and the like can be considered.
[0009]
Subsequently, the background difference unit 6 compares the image data stored in the current image storage unit 3 and the background image storage unit 4, extracts pixel value difference image data for each pixel, and extracts pixel value difference data. Is output (hereinafter referred to as difference image data). The difference image data represents, for example, a portion where the luminance has changed due to the movement of the object or the change in the amount of light in the video. That is, the difference image data specifies a multi-value change region including pixels that indicate different pixel values between the current image data and the background image data. In addition, moving objects such as people and vehicles are usually the main factors that cause changes in brightness in the video, but rarely, "appearance of shadows due to sunlight" or "nighttime car headlights". In some cases, it may depend on something other than the regular target. In the example of FIG. 13, an image region indicating the person at the center of the image is extracted.
[0010]
Next, the threshold value calculation unit 7 calculates a threshold value for binarizing the difference image data that is the multi-value change region input from the background difference unit 6. As described above, the multi-value changing area is multi-value data indicating different pixel values between the current image data and the background image data. For this reason, in order to perform the recognition process as it is, many calculations must be performed, which is inaccurate. Therefore, the difference image data is converted into binary data and recognition processing is performed. As this threshold value, for example, an average value of pixel values of difference image data is used.
[0011]
The binarization unit 8 binarizes the difference image data that is the multi-value change region using the threshold value received from the threshold value calculation unit 7. For example, a pixel having a pixel value greater than or equal to the threshold is a logical value 1, and a pixel having a pixel value less than the threshold is a logical value 0. As a result, the difference image data that is the change area becomes a binary change area. However, depending on the device, there is a case where the multi-value change area is directly recognized without binarization. Here, binarization is performed. With this binarization, taking the image of FIG. 13 as an example, image data in which the solid area of the change region in FIG. 14 is present on a white background can be obtained. The operation so far corresponds to the change region extraction step.
[0012]
Subsequently, when receiving the difference image data indicating the binarized change area, the feature amount calculation unit 9 calculates the feature amount. Here, for example, “continuity”, “area”, “vertical / horizontal dimension”, and “speed” are provided as parameters for defining the feature amount. More specifically, “continuity” represents how many frame change areas have survived in the video, that is, the continuous occurrence time of the change areas. “Area” represents the number of pixels in the binary change region. The “vertical / horizontal dimension” represents the vertical length or horizontal length of the circumscribed rectangle that defines the binary change region. “Speed” represents the moving speed of the change area in the video.
[0013]
On the other hand, as described above, in this apparatus, the feature amount range and condition of a regular report target (for example, a person in FIG. 13) that the operator desires to report is set as the feature parameter setting value 10. age In advance Is remembered. Therefore, when the recognition processing unit 11 receives the feature amount data from the feature amount calculation unit 9, the recognition processing unit 11 determines whether or not the feature amount matches the range or condition indicated in the feature amount parameter setting value 10. At this time, if it is determined that they match, an event is found. That is, the recognition processing unit 11 outputs a notification 12 indicating that a regular notification target has been found to the notification processing unit 13.
[0014]
Upon receiving this event report 12, the report processing unit 13 performs a predetermined report process. Examples of the reporting process include beep sound output, lamp lighting, and reporting / communication to a host machine.
[0015]
FIG. 15 is a flowchart showing the operation of the recognition processing unit in FIG. 12, and a comparison process between the feature parameter setting value 10 and the actually measured feature data will be described with reference to this diagram.
First, when the recognition processing unit 11 acquires feature amount data from the feature amount calculation unit 9 (step ST11), the recognition processing unit 11 determines whether or not the “continuity” matches the feature amount parameter setting value 10 (step ST12). . For the determination of “continuity”, for example, a continuous time indicating how many frames the change area in the difference image data to be processed survives in the video is used. Specifically, when the range of the feature parameter setting value 10 is “minimum 4 frames, maximum 10 frames”, it is determined that the change areas match if the continuous time in which the change area survives is 4 to 10 frames. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. If it is determined that the range matches the range of the feature parameter setting value 10, the process proceeds to step ST13, and if not, the process ends.
[0016]
In step ST <b> 13, the recognition processing unit 11 determines whether or not the “area” of the feature amount data obtained by the feature amount calculating unit 9 matches the feature amount parameter setting value 10. For the determination of the “area”, for example, the number of pixels in the binary change area of the difference image data is used. Specifically, when the range of the feature parameter setting value 10 is “minimum 500 pixels, maximum 1000 pixels”, it is determined that the binary change area matches if the area is 500 to 1000 pixels. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. Here, if it is determined that it matches the range of the feature parameter setting value 10, the process proceeds to step ST14, and if not, the process ends.
[0017]
Next, the recognition processing unit 11 determines whether or not the “vertical and horizontal dimensions” of the feature amount data obtained by the feature amount calculating unit 9 matches the feature amount parameter setting value 10 (step ST14). For the determination of the “vertical and horizontal dimensions”, for example, the vertical length or horizontal length of a circumscribed rectangle that defines a binary change region is used. Specifically, when the range of the feature parameter setting value 10 is “minimum vertical 50 pixels, maximum 100 pixels” and “horizontal minimum 50 pixels, maximum 100 pixels”, both the vertical and horizontal dimensions of the binary change area are 50 to 50. If it is 100 pixels, it is determined that they match. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. Here, if it is determined that it matches the range of the feature parameter setting value 10, the process proceeds to step ST15, and if not, the process ends.
[0018]
Subsequently, the recognition processing unit 11 determines whether or not the “speed” of the feature amount data obtained by the feature amount calculating unit 9 matches the feature amount parameter setting value 10 (step ST15). For the determination of the “speed”, for example, the moving speed in the video of the change area is used. Specifically, when the range of the feature parameter setting value 10 is “minimum 50 pixels / second, maximum 100 pixels / second”, the moving speed of the binary change area in the video may be 50 to 100 pixels / second. Are determined to match. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. Here, if it is determined that the value matches the range of the feature parameter setting value 10, the process proceeds to step ST16, and if not, the process ends.
[0019]
Note that the “area”, “vertical / horizontal dimensions”, and “moving speed” described above are calculated by calculating the average value of the feature values of each image during the lifetime of the binary change area used for the determination in step ST12. It is common to compare with the quantity parameter set value of 10.
[0020]
In step ST <b> 16, the recognition processing unit 11 determines that it matches all the feature parameter setting values 10 that specify the regular notification target, and outputs the event notification 12 to the notification processing unit 13. In addition, the process from step ST11 to step ST16 is realizable by executing the software which has the function of the recognition process part 11 with a computer apparatus.
[0021]
[Problems to be solved by the invention]
Since the conventional image processing apparatus is configured as described above, even if it is a change area due to what is not a regular report target, the report is issued only by accidental movement matching the feature parameter setting value 10. There was a problem of being judged as a target.
[0022]
The above problem will be specifically described.
In the conventional image processing apparatus, as described above, the extraction of the change region to be reported in the video depends on whether or not the feature amount matches the feature parameter setting value 10. For this reason, for example, when shaking a tree branch that is not a regular issue target, the parameter parameter settings include the time when the tree branch is swung, the area change of the change area due to the swing amplitude, the speed of the shake, etc. If the value 10 is met, it is determined that the report is to be issued.
[0023]
Such a report by a change area that is not a regular report target (hereinafter referred to as “unnecessary detection”) has been a serious problem in conventional image processing apparatuses.
[0024]
The present invention has been made to solve the above-described problems, and calculates a prohibited block for specifying an image area to be excluded from a report target according to a certain rule according to a noise occurrence state, and a condition by the prohibited block. It is possible to obtain an image processing apparatus, an image processing method, and a program capable of preventing unnecessary detection even for an accidental movement of a thing that is not a regular notification target by making a determination and determining whether or not to issue the report. Objective.
[0025]
[Means for Solving the Problems]
An image processing apparatus according to the present invention includes a change area extraction unit that sequentially inputs image data obtained by imaging a target area and extracts a change area in the image caused by the movement of an object in the target area; To determine whether or not the feature quantity specifying the change area matches a predetermined set value, and is involved in the change area that does not match the set value among the plurality of divided block areas. Prohibited block candidate setting means for setting a block area to be set as a prohibited block candidate; The frequency of occurrence of the prohibited block candidate set by the prohibited block candidate setting means is measured in the sequentially input image data, and when the measured occurrence frequency is a predetermined value or more, the prohibited block candidate is set as a prohibited block and measured. Cancel the setting as a prohibited block when the occurrence frequency is less than the predetermined value Change area extracted by prohibited block setting means and change area extraction means Or when there is a prohibited block set by the prohibited block setting means in the circumscribed rectangle of the change area Recognition processing means for setting as a non-notification target.
[ 0026 ]
The image processing method according to the present invention includes a change area extraction step of sequentially inputting image data obtained by imaging a target area and extracting a change area in the image caused by the movement of an object in the target area, and a plurality of entire images. To determine whether or not the feature quantity specifying the change area matches a predetermined set value, and is involved in the change area that does not match the set value among the plurality of divided block areas. Forbidden block candidate setting step for setting a block area to be a forbidden block candidate; The frequency of occurrence of prohibited block candidates set by the prohibited block candidate setting step in the sequentially input image data is measured, and when the measured occurrence frequency is a predetermined value or more, the prohibited block candidate is set as a prohibited block and measured. Cancel the setting as a prohibited block when the occurrence frequency is less than the predetermined value Change area extracted in the prohibited block setting step and change area extraction step Or when there is a prohibited block set by the prohibited block setting step in the circumscribed rectangle of the change area A recognition processing step set as a non-notification target.
[ 0027 ]
A program according to the present invention sequentially inputs image data obtained by imaging a target area and extracts a change area in the image caused by the movement of an object in the target area, and the entire image is divided into a plurality of block areas. Block areas related to a change area that does not match the set value out of a plurality of divided block areas, and determines whether or not the feature quantity specifying the change area matches a predetermined set value. Forbidden block candidate setting means for setting as a forbidden block candidate, The frequency of occurrence of the prohibited block candidate set by the prohibited block candidate setting means is measured in the sequentially input image data, and when the measured occurrence frequency is a predetermined value or more, the prohibited block candidate is set as a prohibited block and measured. Cancel the setting as a prohibited block when the occurrence frequency is less than the predetermined value Change area extracted by prohibited block setting means and change area extraction means Or when there is a prohibited block set by the prohibited block setting means in the circumscribed rectangle of the change area The computer is caused to function as a recognition processing unit that is set as a non-notification target.
[ 0028 ]
DETAILED DESCRIPTION OF THE INVENTION
An embodiment of the present invention will be described below.
Embodiment 1 FIG.
FIG. 1 is a diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention. In the figure, 11a is a recognition processing unit (recognition processing means), which operates in the same manner as the recognition processing unit 11 in FIG. It is determined whether it is a target. Reference numeral 14 denotes a prohibited block calculation unit (prohibited block candidate setting means, prohibited block setting means) for calculating a prohibited block for removing a change area caused by “noise” included in the current image data. In addition, the same code | symbol is attached | subjected to the same component as FIG. 12, and the overlapping description is abbreviate | omitted.
[ 0029 ]
FIG. 2 is a diagram showing the configuration of the prohibited block calculation unit in FIG. In the figure, reference numeral 15 denotes a noise determination unit (prohibited block candidate setting means) which divides the current image data into blocks and the feature amount does not match the range or condition of the feature amount parameter setting value 10 from the change region. Screen change areas. Here, the block is data for specifying an image of a certain part in the display screen when the current image data is displayed on a display unit (not shown). Reference numeral 16 denotes a prohibited block candidate determination unit (prohibited block candidate setting unit), which selects a block of current image data including a change area due to “noise” discovered by the noise determination unit 15 as a “prohibited block candidate”. Reference numeral 17 denotes a frequency counting unit (prohibited block setting means), which counts the frequency at which “prohibited block candidates” appear. Reference numeral 18 denotes a prohibited block determination unit (prohibited block setting means) that determines a prohibited block that specifies a change area that is not to be reported based on the count value of the frequency counting unit 17. Moreover, the noise determination part 15, the prohibition block candidate determination part 16, the frequency counting part 17, and the prohibition block determination part 18 are realizable by making a computer apparatus run the program which has these functions, for example.
[ 0030 ]
Next, the operation will be described.
Here, processing from when a video signal captured by an external imaging device (not shown) is input as current image data via the video input unit 1 until the feature amount calculation unit 9 calculates the feature amount of the difference image data. Since this is the same as the prior art, a duplicate description is omitted.
[ 0031 ]
FIG. 3 is a flowchart showing the operation of the recognition processing unit in FIG. 1, and the processing after the recognition processing unit 11a receives the feature amount of the difference image data will be described along this drawing.
First, when the recognition processing unit 11a acquires feature amount data from the feature amount calculation unit 9 (step ST1), the recognition processing unit 11a determines whether or not the “continuity” matches the feature amount parameter setting value 10 (step ST2). . For the determination of “continuity”, for example, a continuous time indicating how many frames the change area in the difference image data to be processed survives in the video is used. Specifically, when the range of the feature parameter setting value 10 is “minimum 4 frames, maximum 10 frames”, it is determined that the change areas match if the continuous time in which the change area survives is 4 to 10 frames. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. Here, if it is determined that the value matches the range of the feature parameter setting value 10, the process proceeds to step ST3, and if not, the process ends.
[ 0032 ]
In step ST <b> 3, the recognition processing unit 11 a determines whether the “area” of the feature amount data obtained by the feature amount calculation unit 9 matches the feature amount parameter setting value 10. For the determination of the “area”, for example, the number of pixels in the binary change area of the difference image data is used. Specifically, when the range of the feature parameter setting value 10 is “minimum 500 pixels, maximum 1000 pixels”, it is determined that the binary change area matches if the area is 500 to 1000 pixels. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. Here, if it is determined that it matches the range of the feature parameter setting value 10, the process proceeds to step ST4, and if not, the process ends.
[ 0033 ]
Next, the recognition processing unit 11a determines whether or not the “vertical and horizontal dimensions” of the feature amount data obtained by the feature amount calculating unit 9 matches the feature amount parameter setting value 10 (step ST4). For the determination of the “vertical and horizontal dimensions”, for example, the vertical length or horizontal length of a circumscribed rectangle that defines a binary change region is used. Specifically, when the range of the feature parameter setting value 10 is “minimum vertical 50 pixels, maximum 100 pixels” and “horizontal minimum 50 pixels, maximum 100 pixels”, both the vertical and horizontal dimensions of the binary change area are 50 to 50. If it is 100 pixels, it is determined that they match. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. Here, if it is determined that it matches the range of the feature parameter setting value 10, the process proceeds to step ST5, and if not, the process ends.
[ 0034 ]
Subsequently, the recognition processing unit 11a determines whether or not the “speed” of the feature amount data obtained by the feature amount calculating unit 9 matches the feature amount parameter setting value 10 (step ST5). For the determination of the “speed”, for example, the moving speed in the video of the change area is used. Specifically, when the range of the feature parameter setting value 10 is “minimum 50 pixels / second, maximum 100 pixels / second”, the moving speed of the binary change area in the video may be 50 to 100 pixels / second. Are determined to match. Otherwise, it is determined that it does not match the range of the feature parameter setting value 10. Here, if it is determined that it matches the range of the feature parameter setting value 10, the process proceeds to step ST5A, and if not, the process ends.
[ 0035 ]
Note that the “area”, “vertical / horizontal dimensions”, and “moving speed” described above are calculated from, for example, the average values of the feature values of each image during the survival time of the binary change region used in the determination process of step ST2.
[ 0036 ]
Next, the recognition processing unit 11a compares the data related to the change area displayed on the display screen with the data related to the prohibition block created and held by the prohibition block calculation unit 14, and the change area is the subject of notification. (Step ST5A, recognition processing step). This determination is performed for each change region. At this time, if it is determined that the change area being processed is a report target, the process proceeds to step ST6, and if not matched, the process ends.
[ 0037 ]
In step ST <b> 6, the recognition processing unit 11 a determines that it matches all the feature parameter setting values 10 that specify the regular notification target, and outputs the event notification 12 to the notification processing unit 13. In addition, the process from step ST1 to step ST6 is realizable by executing the software which has the function of the recognition process part 11a with a computer apparatus.
[ 0038 ]
Here, the operation of the prohibited block calculation unit 14 will be described in detail.
FIG. 4 is a flowchart showing the operation of the prohibited block calculation unit in FIG. 2. The operation of the prohibited block calculation unit will be described with reference to FIG.
First, the noise determination unit 15 in the prohibited block calculation unit 14 acquires the current image data (step ST1a), and divides it into blocks of a certain size (step ST2a).
[ 0039 ]
FIG. 5 is a diagram showing a display screen of current image data. This figure shows a video of the festival venue, with lanterns near the top of the video and trees at the bottom. These lanterns and tree movements become the change areas. FIG. 6 is a view showing a display screen obtained by dividing the current image data in FIG. 5 into blocks. As shown in the figure, the size and arrangement of the blocks are desirably uniform, but these may be non-uniform. The block size may be defined as the size of one pixel. Here, the description will be made on the assumption that uniform square blocks having a size of 32 pixels × 32 pixels are uniformly arranged.
[ 0040 ]
In addition to the current image data, the prohibited block calculation unit 14 inputs difference image data (binary change area) between the current image data and the background image data. Here, it is assumed that the binary change area is binary data of logical values “0” and “1”. The logical values “0” and “1” are shown for each pixel. For example, a changed portion is defined as the logical value “1”.
[ 0041 ]
In the image shown in FIG. 5, for example, when the lantern or the tree branch is shaken by the wind, the binarized result of the corresponding pixel is assigned to the logical value “1” as the changed area. The Rukoto.
[ 0042 ]
FIG. 7 is a diagram showing a display screen in which pixels in which the current image data in FIG. 6 has changed are filled. In the figure, reference numeral 19 denotes a change area composed of pixels having a change in pixel value and assigned to a logical value “1”, and image data for dyeing the pixels all over is superimposed. As shown in the figure, the change area 19 appears on the left and right side surfaces of the five lanterns in front and the tip of the leaf of the branch of the tree. When a lantern or a tree branch is shaken by the wind and a fine movement occurs, differential image data is generated in the moving portion and cut out as a change area. Here, it is not desirable for the operator to report these lanterns and tree branch movements. That is, it is assumed that the change area by these is not a regular report object.
[ 0043 ]
Next, the noise determination unit 15 selects the change region whose feature amount does not match the range / condition of the feature amount parameter setting value 10 and sets it as a change region due to “noise” that is not a regular issue target. (Step ST3a). Here, the “change area that does not match” refers to, for example, a change area determined as NO in any of steps ST2 to ST5 by the recognition processing unit 11a illustrated in FIG. That is, the noise determination unit 15 receives the feature amount data and the determination result for each change region from the recognition processing unit 11a, and determines whether the noise is due to “noise”. Further, in the example of FIG. 7, the change area 19 due to the lantern and the shaking of the tree branch is not a regular notification target, and thus is determined as “noise”.
[ 0044 ]
Note that the noise determination by the noise determination unit 15 is performed even when the noise determination unit 15 determines NO from step ST2 to step ST5 in FIG. 3 when using the determination result by the recognition processing unit 11a as described above. It is also possible to perform the process of proceeding to the next determination step without ending and to determine that the change area determined as NO in a plurality of steps is “noise”.
[ 0045 ]
In addition, it is not based on whether or not NO is determined in the processing from step ST2 to step ST5 in FIG. “Noise” judgment based on a limited situation of “below” or “above” may be independently performed. In other words, in the present invention, it is only necessary to select a change area that is determined to be “noise” according to some rule from factors that cause changes that are not regular notification targets.
[ 0046 ]
Subsequently, the forbidden block candidate determination unit 16 acquires data related to the block defined by the noise determination unit 15 by dividing the current image data into blocks, and includes a block including a change area due to “noise” obtained as described above. Are selected (step ST4a). Specifically, for example, a block including a change area due to “noise” when the position coordinate data on the display screen of the pixels belonging to the change area due to “noise” and the pixels defined by the block match. Is determined. Data relating to the blocks thus selected is defined as “prohibited block candidates” and held in a memory (not shown). The operation so far corresponds to the prohibited block candidate setting step.
[ 0047 ]
Whether or not the block contains a change area due to “noise” is determined by whether the change area due to “noise” is “contacted” or “is nearby”, etc. It may be a standard. In addition, it is not determined whether or not the block related to the current image data includes a change area due to “noise” on the same screen, but the block related to the current image data and the change area due to “noise” are temporally different data. It may be calculated separately based on the above and compared with each other.
[ 0048 ]
As described above, when the prohibited block candidate determination unit 16 obtains the “prohibited block candidate”, the frequency counting unit 17 counts how often the “prohibited block candidate” appears (step ST5a). .
FIG. 8 is a view showing a display screen in which the frequency count value of the prohibited block candidate is superimposed on the block in FIG. As shown in the figure, the appearance frequency of “prohibited block candidates” is counted for each block, and is counted up and down as necessary.
[ 0049 ]
Here, the frequency counting process by the frequency counting unit 17 will be described in detail.
First, rules regarding the count-up of the frequency count value will be described.
The frequency counting unit 17 counts how many times a “prohibited block candidate” is determined within a predetermined unit time. For example, assuming that the unit time is 1 minute, the frequency count value of the block is 3 if “forbidden block candidate” occurs 3 times per minute.
[ 0050 ]
At this time, if it is determined that the same block is a “prohibited block candidate” within a predetermined unit time, the frequency count value is counted up. For example, when the unit time = 1 minute, if a block that has once become a “prohibited block candidate” becomes a “prohibited block candidate” again within one minute, the frequency count value of that block is incremented.
[ 0051 ]
Further, the frequency counting unit 17 counts up the frequency count value even when a predetermined number of “prohibited block candidates” exist around the block that has become the “prohibited block candidate”. For example, when the predetermined number = 3, if there are three or more “prohibited block candidates” around the “prohibited block candidate”, the frequency count value of the block is incremented.
[ 0052 ]
Next, rules regarding the countdown of the frequency count value will be described.
The frequency counting unit 17 counts down when the predetermined unit time “prohibited block candidate” does not appear. For example, when unit time = 1 minute, if no “prohibited block candidate” appears for one minute, the frequency count value of that block is decremented.
[ 0053 ]
At this time, the frequency count value is counted down even when there is no predetermined number of “prohibited block candidates” around the block that has become the “prohibited block candidate”. For example, when the predetermined number = 3, if there are less than three “prohibited block candidates” around the “prohibited block candidate”, the frequency count value of the block is decremented. Alternatively, the frequency count value may be simply decremented at regular intervals.
[ 0054 ]
Note that the above conditions are not limited to the above as long as the frequency count value is counted up or down depending on the temporal generation frequency or spatial generation density of the “prohibited block candidates”.
[ 0055 ]
The frequency counting process of the “prohibited block candidate” described above may be performed as follows.
First, rules regarding the count-up of the frequency count value will be described.
When it is determined that the area in the block of the change area existing in the “prohibited block candidate” is larger (or smaller) than the predetermined value, the frequency counting unit 17 counts up the frequency count value of the block. For example, when the predetermined value = 512 pixels, if the change area occupies more than half of the entire block (32 × 32 = 1024), the frequency count value of the block is incremented.
[ 0056 ]
Here, when one change region exists across a plurality of “forbidden block candidates”, when the area of the change region in each “forbidden block candidate” is determined to be larger (or smaller) than a predetermined value, The frequency count value of the block is counted up. For example, when the predetermined value = 2048 pixels, if the area of the change area applied to the block is 2048 (approximately 2 blocks) pixels or more, the frequency count value of that block is incremented.
[ 0057 ]
Further, when a predetermined number of “prohibited block candidates” satisfying the above-described two conditions exist around the block that has become the “prohibited block candidate”, the frequency counter is incremented by the frequency counter value. For example, when the predetermined number = 3, if there are three or more “prohibited block candidates” in which the change area occupies half of the block around the “prohibited block candidate” to be frequency counted, the frequency count of that block The value is incremented.
[ 0058 ]
Next, rules regarding the countdown of the frequency count value will be described.
When it is determined that the area in the block of the change area existing in the “prohibited block candidate” is smaller (or larger) than the predetermined value, the frequency counting unit 17 counts down the frequency count value of the block. For example, when the predetermined value = 512 pixels, if the area of the change region occupies less than half of the entire block (32 × 32 = 1024), the frequency count value of the block is decremented.
[ 0059 ]
When one change area exists across a plurality of “prohibited block candidates”, if it is determined that the area of the change area in each “prohibited block candidate” is smaller (or larger) than the predetermined value, Count down the frequency count value. For example, when the predetermined value = 2048 pixels, if the area of the change area applied to the block is 2048 (approximately 2 blocks) or less, the frequency count value of the block is decremented. Also at this time, the frequency count value may be simply decremented at regular intervals in the same manner as described above.
[ 0060 ]
Note that the above conditions are not limited to the above as long as the frequency count value of the “prohibited block candidate” is counted up / down depending on the increase / decrease state of the area of the change area related to the “prohibited block candidate”.
[ 0061 ]
The frequency count value determined as described above is a large number in a block where noise frequently occurs, and a small number in a block where noise is not generated much or a block where noise generation is reduced. Further, as shown in FIG. 8, the frequency count value is superimposed on the display screen of the current image data divided into blocks and is made visible to the operator.
[ 0062 ]
Note that one of the two frequency count processing methods described above may be performed separately, or both may be performed in parallel and used for determining a prohibited block.
[ 0063 ]
Subsequently, when the prohibited block determining unit 18 receives the frequency count value from the frequency counting unit 17, the prohibited block determining unit 18 determines a prohibited block based on the value (step ST6a). Here, a “prohibited block candidate” having a count value equal to or greater than a predetermined value is set as a prohibited block. For example, when the predetermined value is 7, a “prohibited block candidate” having a count value of 7 or more is determined as a prohibited block.
[ 0064 ]
FIG. 9 is a diagram showing a display screen in which prohibited blocks determined based on the frequency count value in FIG. 8 are filled. In the example shown in the figure, the “prohibited block candidate” whose frequency count value exceeds 6 is determined as a prohibited block, and solid blocks are superimposed on this. When the prohibited block determining unit 18 determines the prohibited block, the prohibited block determining unit 18 stores data on the prohibited block such as the position coordinates on the display screen in a memory (not shown).
[ 0065 ]
In addition, the determination of the prohibited block is made by displaying “prohibited block candidate” instead of setting the predetermined value fixedly in addition to setting the “prohibited block candidate” having a count value equal to or greater than a predetermined value as the prohibited block. A variation value that varies according to the position on the screen and the situation may be set as the predetermined value.
[ 0066 ]
Further, the determination of the prohibited block may be based on the frequency count value of the blocks located around the “prohibited block candidate” in addition to the frequency count value of the “prohibited block candidate”. Further, the prohibition block may be determined based on the frequency count value of the “prohibition block candidate” obtained from the image data that is separated in time.
The operation so far corresponds to the prohibited block setting step.
[ 0067 ]
Next, a process for recognizing a notification target using the prohibited block obtained as described above will be described in detail. Here, it is assumed that a prohibited block is created by the prohibited block calculation unit 14 and data related thereto is stored in the memory.
In step ST5A in FIG. 3, the recognition processing unit 11a uses the change area and the prohibited block based on the data related to the changed area displayed on the display screen and the data related to the prohibited block created and held by the prohibited block calculating unit 14. It is determined whether or not the change area is a report target from the mutual arrangement. This determination is made for each change region.
[ 0068 ]
Here, the process in step ST5A in FIG. 3 will be described in detail.
First, the recognition processing unit 11a defines a circumscribed rectangle that surrounds the change area when comparing the change area on the display screen with the prohibited block. The circumscribed rectangle is provided so as to surround the change area at an isolated position, and is provided so as to surround the change area when the plurality of change areas are at a fixed position.
[ 0069 ]
If there is a forbidden block in the change area on the display screen or its circumscribed rectangle, the recognition processing unit 11a determines that the change area is not a report target. On the other hand, if there is no prohibited block in the change area or its circumscribed rectangle, it is determined that the change area is a report target.
[ 0070 ]
In addition to the determination criteria described above, whether or not to issue a report may be determined based on whether or not there is a prohibited block within a predetermined distance from the change area or its circumscribed rectangle. In this case, if there is a prohibited block within a predetermined distance from the change area or its circumscribed rectangle, it is determined that the change area is not a report target. For example, when the predetermined distance = 1 block, if there is a prohibited block within one block range around the change area, it is determined that the change area is not the subject of notification.
[ 0071 ]
Note that the present invention is not limited to the above conditions as long as the change area can be reported according to the distance from the change area or its circumscribed rectangle to the prohibited block.
[ 0072 ]
Further, the processing by the recognition processing unit 11a described above may be performed as follows.
First, if there are a predetermined number or more of prohibited blocks in the change area or its circumscribed rectangle, the recognition processing unit 11a determines that the change area is not the subject of notification. Otherwise, it is determined to be a report target. For example, when the predetermined number = 3, if there are three forbidden blocks in the circumscribed rectangle of the change area, it is determined that the change area is not the subject of notification.
[ 0073 ]
Further, the recognition processing unit 11a may determine whether there is a predetermined number or more of prohibited blocks within a predetermined distance from the change area or its circumscribed rectangle. In this case, if there are a predetermined number or more of prohibited blocks within a predetermined distance, it is determined that the change area is not a subject of notification. For example, when the predetermined distance = 1 block and the predetermined number = 3, if there are three forbidden blocks within the range of one block around the change area, the change area is not to be reported.
[ 0074 ]
Note that the conditions are not limited to the above as long as the change area can be reported based on the number of prohibited blocks in and around the change area or its circumscribed rectangle.
[ 0075 ]
Further, the processing by the recognition processing unit 11a may be as follows.
First, the recognition processing unit 11a determines that a change area is not a notification target if there is a prohibited block that continues to exist for a predetermined time or longer in the change area or its circumscribed rectangle. Otherwise, it is determined to be a report target. Here, survival is the time from the establishment of a prohibited block to the present. For example, when the predetermined time = 3 seconds, if there is a forbidden block that remains alive for 3 seconds or more in the circumscribed rectangle of the change area, it is determined that the change area is not an issue target.
[ 0076 ]
In addition to the above conditions, if a predetermined number or more of prohibited blocks are continuously alive within a predetermined distance from the change area or its circumscribed rectangle, it is determined that the change area is not the subject of notification. You may make it do. For example, when the predetermined time = 1 second and the predetermined number = 3, if there are 3 or more prohibited blocks that have been alive for 2 seconds or more within the range of one block around the change area, the change area is not subject to the alert. It is determined that there is not.
[ 0077 ]
Note that the present invention is not limited to the above conditions as long as the change area can be reported based on the length of the lifetime of the prohibited blocks inside and around the change area or its circumscribed rectangle.
[ 0078 ]
The above-described condition determination by the prohibition block is performed for each change area appearing on the display screen. These condition determinations may be performed separately or in parallel.
[ 0079 ]
Here, a specific example is given to summarize the recognition target object recognition processing using the prohibited block.
FIG. 10 is a diagram showing a display screen of current image data in which prohibited blocks are superimposed, and the display screen of the image data to be processed displays the prohibited blocks created in the process from FIG. 5 to FIG. . In the image shown in FIG. 10, pedestrians, lanterns swaying in the wind, and tree branches swaying in the wind are mixed, creating six change areas.
[ 0080 ]
FIG. 11 is a diagram showing a display screen in which the change area is solid-dyed on the current image data in FIG. 10 and the circumscribed rectangle is overwritten. In the figure, A is a circumscribed rectangle defined in the change area by “pedestrian”, B to E are circumscribed rectangles defined in the change area by “the lantern swayed by the wind”, and F is “the tree branch swayed by the wind” A circumscribed rectangle defined in the change area. Here, in the condition determination by the prohibition block, the recognition processing unit 11a determines that the simplest rule is “if there is a prohibition block in the circumscribed rectangle of the change area, the change area is not a report target”. . Further, the “pedestrian” that is a factor of the change area defined by the circumscribed rectangle A is an object that the operator wants to be notified of. On the other hand, “wind lanterns swaying by the wind” and “branches swaying by the wind”, which are the factors of the change area defined by the circumscribed rectangles B to F, are not objects that the operator desires to be notified. That is, it is desirable that the change area defined by the circumscribed rectangles B to F is not a regular report target and is not reported.
[ 0081 ]
In the example of FIG. 11, there are 2 to 14 forbidden blocks in the circumscribed rectangles B to F. For this reason, the recognition processing unit 11a determines that the change area defined by the circumscribed rectangles B to F is not a report target. Therefore, the recognition processing unit 11a does not output the event report 12 for the change area defined by the circumscribed rectangles B to F to the report processing unit 13, and does not report.
[ 0082 ]
On the other hand, the circumscribed rectangle A defined in the change area by “pedestrian” has no prohibited block. For this reason, the recognition processing unit 11a determines that the change area defined by the circumscribed rectangle A is a report target. Accordingly, the recognition processing unit 11a outputs the event report 12 for the change area defined by the circumscribed rectangle A to the report processing unit 13 and reaches the report.
[ 0083 ]
What should be noted here is the relationship between the sizes of the change areas defined by the circumscribed rectangles A and F. That is, the circumscribed rectangle A is smaller. Originally, in an image processing apparatus that emphasizes only the area of a change area, it is difficult to issue a change area with a small area without reporting a change area with a large area.
[ 0084 ]
As described above, in the image processing apparatus of the present invention, even when there are “an object that frequently fluctuates such as a lantern or a tree branch” or “an object that continues to move for a while such as a puddle”, these are regarded as noise. Judgment is made to create a forbidden block, thereby judging whether or not to issue a change area. Therefore, it is possible to prevent unnecessary detection of a large area change area that is not a report target, and it is possible to reliably perform a report process for a small area change area that is a report target.
[ 0085 ]
Moreover, according to the rules for creating a prohibited block, if the noise of the prohibited block decreases (in the example of FIG. 11, if the wind subsides and the lanterns and the branches of the tree disappear), the noise disappears naturally and the noise increases. Naturally generated. Therefore, when there is little noise, there is no prohibited block, and it is possible to report the change area equally over the entire screen. On the contrary, when there is a lot of noise, a prohibition block is generated, and it is possible to prohibit notification only for a change area in a noisy area.
[ 0086 ]
As described above, the most important feature of the present invention is to prevent unnecessary detection by limiting the notification according to the increase or decrease of noise.
[ 0087 ]
As described above, according to the first embodiment, the forbidden block calculation unit 14 calculates a forbidden block with a certain rule according to the noise occurrence state, and the recognition processing unit 11a sets the feature parameter setting value for each change area. In addition to the determination by 10, the condition determination by the prohibited block is performed to determine whether or not the notification is possible. Therefore, an image region in which noise that happens to coincide with the feature parameter setting value 10 accidentally occurs is detected from the notification target. Except for this, it is possible to determine whether or not to issue. Thereby, unnecessary detection can be reliably prevented.
[0093]
【The invention's effect】
As described above, according to the present invention, image data obtained by capturing a target area is sequentially input, a change area in the image caused by the movement of an object in the target area is extracted, and the entire image is divided into a plurality of block areas. Block areas related to a change area that does not match the set value out of a plurality of divided block areas, and determines whether or not the feature quantity specifying the change area matches a predetermined set value. Is set as a forbidden block candidate, The frequency of occurrence of prohibited block candidates set in sequentially input image data is measured, and when the measured occurrence frequency is a predetermined value or more, the prohibited block candidate is set as a prohibited block, and the measured occurrence frequency is predetermined. If it is less than the value, cancel the setting as a prohibited block. , Extracted change area Or when there is a prohibited block set in the circumscribed rectangle of the change area Since it is set as a non-notification target, it is possible to determine whether it is possible to report by excluding the image area where noise that might be accidentally determined to be a notification target, and to prevent unnecessary detection reliably. There is an effect that can be. Moreover, regardless of the size of the change area, for example, unnecessary detection of a large area change area that is not a report target can be prevented, and a notification process for a small area change area that is a report target can be performed. It can be done reliably.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an image processing apparatus according to Embodiment 1 of the present invention.
FIG. 2 is a diagram showing a configuration of a prohibited block calculation unit in FIG. 1;
FIG. 3 is a flowchart showing the operation of the recognition processing unit in FIG. 1;
FIG. 4 is a flowchart showing the operation of the prohibited block calculation unit in FIG. 2;
FIG. 5 is a diagram showing a display screen of current image data.
6 is a diagram showing a display screen obtained by dividing the current image data in FIG. 5 into blocks.
7 is a diagram showing a display screen in which pixels in which current image data in FIG. 6 has changed are filled in. FIG.
8 is a diagram showing a display screen in which the frequency count value of the prohibited block candidate is superimposed on the block in FIG. 7. FIG.
FIG. 9 is a diagram showing a display screen in which prohibited blocks determined based on the frequency count value in FIG. 8 are filled.
FIG. 10 is a diagram showing a display screen of current image data on which prohibited blocks are superimposed.
11 is a diagram showing a display screen in which a change area is solid-colored on the current image data in FIG. 10 and a circumscribed rectangle is overwritten.
FIG. 12 is a diagram illustrating a configuration of a conventional image processing apparatus.
FIG. 13 is a diagram showing a display screen of current image data.
14 is a diagram showing a display screen of current image data in which a black solid area is superimposed on the change area in FIG.
15 is a flowchart showing the operation of the recognition processing unit in FIG. 12. FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Video input part, 2 A / D converter, 3 Current image storage part, 4 Background image storage part, 5 Background update part, 6 Background difference part (Change area extraction means), 7 Threshold calculation part (Change area extraction means), 8 Binarization unit (change area extraction unit), 9 feature amount calculation unit, 10 feature amount parameter setting value (predetermined set value), 11a recognition processing unit (recognition processing unit), 12 alerting, 13 alerting processing , 14 Prohibited block calculation unit (prohibited block candidate setting unit, prohibited block setting unit), 15 Noise determining unit (prohibited block candidate setting unit), 16 Prohibited block candidate determining unit (prohibited block candidate setting unit), 17 Frequency counting unit (Prohibited block setting means), 18 prohibited block determining section (prohibited block setting means), 19 change area.

Claims (3)

Change area extraction means for sequentially inputting image data obtained by imaging a target area and extracting a change area in the image caused by movement of an object in the target area;
The entire image is divided into a plurality of block areas, and it is determined whether or not a feature value for specifying the change area matches a predetermined set value, and the set value among the divided block areas is determined. Forbidden block candidate setting means for setting a block area related to a change area that does not match as a forbidden block candidate;
Measure the frequency of occurrence of the prohibited block candidate set by the prohibited block candidate setting means in the sequentially input image data, and set the prohibited block candidate as a prohibited block when the measured occurrence frequency is a predetermined value or more, A prohibited block setting means for canceling the setting as a prohibited block when the measured occurrence frequency is less than a predetermined value ;
A recognition processing means for setting as a non-notification target when there is a prohibited area set by the prohibited block setting means in the change area extracted by the change area extraction means or a circumscribed rectangle of the change area; apparatus. A change area extraction step of sequentially inputting image data obtained by imaging a target area and extracting a change area in the image caused by the movement of an object in the target area;
The entire image is divided into a plurality of block areas, and it is determined whether or not a feature value for specifying the change area matches a predetermined set value, and the set value among the divided block areas is determined. A forbidden block candidate setting step for setting a block area related to a change area that does not match as a forbidden block candidate;
The occurrence frequency of the prohibited block candidate set by the prohibited block candidate setting step in the sequentially input image data is measured, and when the measured occurrence frequency is a predetermined value or more, the prohibited block candidate is set as a prohibited block, A prohibited block setting step for canceling the setting as a prohibited block when the measured occurrence frequency is less than a predetermined value ;
A recognition processing step that is set as a non-notification target when there is a prohibited block set by the prohibited block setting step in the changed region extracted by the changed region extraction step or a circumscribed rectangle of the changed region. Processing method. Change area extraction means for sequentially inputting image data obtained by imaging a target area and extracting a change area in the image caused by movement of an object in the target area;
The entire image is divided into a plurality of block areas, and it is determined whether or not a feature value for specifying the change area matches a predetermined set value, and the set value among the divided block areas is determined. Prohibited block candidate setting means for setting, as a prohibited block candidate, a block area related to a change area that does not match
Measure the frequency of occurrence of the prohibited block candidate set by the prohibited block candidate setting means in the sequentially input image data, and set the prohibited block candidate as a prohibited block when the measured occurrence frequency is a predetermined value or more, Prohibited block setting means for canceling the setting as a prohibited block when the measured occurrence frequency is less than a predetermined value ,
A program that causes a computer to function as a recognition processing unit that is set as a non-notification target when there is a prohibited block set by the prohibited block setting unit in a changed region extracted by the changed region extracting unit or a circumscribed rectangle of the changed region .

Images